Bug 145681 - [iOS] Arabic text styled with Georgia is rendered as boxes
Summary: [iOS] Arabic text styled with Georgia is rendered as boxes
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Myles C. Maxfield
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2015-06-04 20:13 PDT by Myles C. Maxfield
Modified: 2015-06-23 14:10 PDT (History)
11 users (show)

See Also:


Attachments
Patch (10.11 KB, patch)
2015-06-04 20:16 PDT, Myles C. Maxfield
no flags Details | Formatted Diff | Diff
Patch (10.18 KB, patch)
2015-06-04 20:17 PDT, Myles C. Maxfield
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews106 for mac-mavericks-wk2 (594.99 KB, application/zip)
2015-06-04 21:08 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews101 for mac-mavericks (552.20 KB, application/zip)
2015-06-04 21:27 PDT, Build Bot
no flags Details
Patch (11.53 KB, patch)
2015-06-05 12:11 PDT, Myles C. Maxfield
no flags Details | Formatted Diff | Diff
Patch (8.12 KB, patch)
2015-06-05 21:17 PDT, Myles C. Maxfield
darin: review+
Details | Formatted Diff | Diff
Patch for landing (7.54 KB, patch)
2015-06-08 21:35 PDT, Myles C. Maxfield
no flags Details | Formatted Diff | Diff
Patch (8.83 KB, patch)
2015-06-19 20:41 PDT, Myles C. Maxfield
no flags Details | Formatted Diff | Diff
Patch (94.30 KB, patch)
2015-06-19 23:37 PDT, Myles C. Maxfield
no flags Details | Formatted Diff | Diff
Patch (94.37 KB, patch)
2015-06-19 23:38 PDT, Myles C. Maxfield
darin: review+
Details | Formatted Diff | Diff
Patch for landing (94.21 KB, patch)
2015-06-22 11:46 PDT, Myles C. Maxfield
commit-queue: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Myles C. Maxfield 2015-06-04 20:13:29 PDT
Arabic text styled with Georgia is rendered as boxes
Comment 1 Myles C. Maxfield 2015-06-04 20:16:39 PDT
Created attachment 254334 [details]
Patch
Comment 2 Myles C. Maxfield 2015-06-04 20:17:59 PDT
Created attachment 254335 [details]
Patch
Comment 3 Myles C. Maxfield 2015-06-04 20:18:39 PDT
<rdar://problem/21169844>
Comment 4 mitz 2015-06-04 20:27:38 PDT
Times New Roman has been working with Arabic for a while, but last time we tried allowing it, it was prohibitively slow. Has that changed?
Comment 5 Myles C. Maxfield 2015-06-04 20:31:31 PDT
(In reply to comment #4)
> Times New Roman has been working with Arabic for a while, but last time we
> tried allowing it, it was prohibitively slow. Has that changed?

I don't know; I didn't test performance. How was it tested before? Were there any radars filed about it? Do you have any other information?
Comment 6 Build Bot 2015-06-04 21:08:45 PDT
Comment on attachment 254335 [details]
Patch

Attachment 254335 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.appspot.com/results/4670180730863616

New failing tests:
fast/text/arabic-times-new-roman.html
Comment 7 Build Bot 2015-06-04 21:08:49 PDT
Created attachment 254337 [details]
Archive of layout-test-results from ews106 for mac-mavericks-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews106  Port: mac-mavericks-wk2  Platform: Mac OS X 10.9.5
Comment 8 mitz 2015-06-04 21:10:22 PDT
(In reply to comment #5)
> (In reply to comment #4)
> > Times New Roman has been working with Arabic for a while, but last time we
> > tried allowing it, it was prohibitively slow. Has that changed?
> 
> I don't know; I didn't test performance. How was it tested before?

It was tested using Apple’s PLT.

> Were there any radars filed about it? Do you have any other information?

<rdar://problem/12096835> is the Apple bug tracking allowing Times New Roman to be used for Arabic, and it contains the most recent measurements.
Comment 9 Build Bot 2015-06-04 21:27:34 PDT
Comment on attachment 254335 [details]
Patch

Attachment 254335 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/6693017448611840

New failing tests:
fast/text/arabic-times-new-roman.html
Comment 10 Build Bot 2015-06-04 21:27:37 PDT
Created attachment 254339 [details]
Archive of layout-test-results from ews101 for mac-mavericks

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews101  Port: mac-mavericks  Platform: Mac OS X 10.9.5
Comment 11 Myles C. Maxfield 2015-06-05 12:11:15 PDT
Created attachment 254371 [details]
Patch
Comment 12 Darin Adler 2015-06-05 14:58:37 PDT
Comment on attachment 254371 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=254371&action=review

> Source/WebCore/platform/graphics/cocoa/FontCocoa.mm:-89
> -    // Times New Roman contains Arabic glyphs, but Core Text doesn't know how to shape them. <rdar://problem/9823975>
> -    // FIXME <rdar://problem/12096835> remove this function once the above bug is fixed.
> -    // Arial and Tahoma are have performance issues so don't use them as well.

Have we tested both the correctness of shaping Times New Roman, and the performance of shaping in Arial and Tahoma? Where can I see some details of how we tested and what results we got?
Comment 13 Myles C. Maxfield 2015-06-05 15:39:59 PDT
Comment on attachment 254371 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=254371&action=review

>> Source/WebCore/platform/graphics/cocoa/FontCocoa.mm:-89
>> -    // Arial and Tahoma are have performance issues so don't use them as well.
> 
> Have we tested both the correctness of shaping Times New Roman, and the performance of shaping in Arial and Tahoma? Where can I see some details of how we tested and what results we got?

I'm testing the performance right now.

I probably don't actually need to change Arial and Tahoma for this bug, since it only concerns Times New Roman. I'll revert that part.
Comment 14 Myles C. Maxfield 2015-06-05 15:46:22 PDT
The patch, as posted, is a slowdown by almost 1 standard deviation. So that's unacceptable.
Comment 15 Myles C. Maxfield 2015-06-05 15:47:46 PDT
(In reply to comment #14)
> The patch, as posted, is a slowdown by almost 1 standard deviation. So
> that's unacceptable.

(Just tested on kooora.com)
Comment 16 Myles C. Maxfield 2015-06-05 20:22:00 PDT
Calling CTFontCreatePhysicalFontDescriptorForCharactersWithLanguage() on TimesNewRomanPSMT returns GeezaPro
Comment 17 Myles C. Maxfield 2015-06-05 21:13:35 PDT
Comment on attachment 254371 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=254371&action=review

>>> Source/WebCore/platform/graphics/cocoa/FontCocoa.mm:-89
>>> -    // Arial and Tahoma are have performance issues so don't use them as well.
>> 
>> Have we tested both the correctness of shaping Times New Roman, and the performance of shaping in Arial and Tahoma? Where can I see some details of how we tested and what results we got?
> 
> I'm testing the performance right now.
> 
> I probably don't actually need to change Arial and Tahoma for this bug, since it only concerns Times New Roman. I'll revert that part.

The smaller version of this patch has no regression on iPhone 5s
Comment 18 Myles C. Maxfield 2015-06-05 21:17:40 PDT
Created attachment 254405 [details]
Patch
Comment 19 mitz 2015-06-05 21:23:22 PDT
Comment on attachment 254405 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=254405&action=review

> Source/WebCore/ChangeLog:10
> +        Georgia doesn't support Arabic, so we ask CoreText what font does support Arabic. It returns
> +        TimesNewRomanPSMT. In the past, this font used to be broken with Arabic, so we explicitly disallowed it.

Are there any fonts that don’t support Arabic for which Core Text returns Arial or Tahoma as a fallback?
Comment 20 Myles C. Maxfield 2015-06-05 21:40:56 PDT
(In reply to comment #19)
> Comment on attachment 254405 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=254405&action=review
> 
> > Source/WebCore/ChangeLog:10
> > +        Georgia doesn't support Arabic, so we ask CoreText what font does support Arabic. It returns
> > +        TimesNewRomanPSMT. In the past, this font used to be broken with Arabic, so we explicitly disallowed it.
> 
> Are there any fonts that don’t support Arabic for which Core Text returns
> Arial or Tahoma as a fallback?

Just wrote a tiny iOS program, looks like the answer is no. The only fonts that CTFontCreatePhysicalFontDescriptorForCharactersWithLanguage() will return for Arabic will be:

CourierNewPS-BoldMT
CourierNewPSMT
GeezaPro
GeezaPro-Bold
TimesNewRomanPS-BoldMT
TimesNewRomanPSMT
Comment 21 Myles C. Maxfield 2015-06-05 21:45:34 PDT
Comment on attachment 254405 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=254405&action=review

>>> Source/WebCore/ChangeLog:10
>>> +        TimesNewRomanPSMT. In the past, this font used to be broken with Arabic, so we explicitly disallowed it.
>> 
>> Are there any fonts that don’t support Arabic for which Core Text returns Arial or Tahoma as a fallback?
> 
> Just wrote a tiny iOS program, looks like the answer is no. The only fonts that CTFontCreatePhysicalFontDescriptorForCharactersWithLanguage() will return for Arabic will be:
> 
> CourierNewPS-BoldMT
> CourierNewPSMT
> GeezaPro
> GeezaPro-Bold
> TimesNewRomanPS-BoldMT
> TimesNewRomanPSMT

Tahoma doesn't even seem to be installed on iOS.
Comment 22 Darin Adler 2015-06-06 22:30:50 PDT
Comment on attachment 254405 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=254405&action=review

> Source/WebCore/ChangeLog:24
> +        * platform/graphics/Font.cpp:
> +        (WebCore::Font::Font): Deleted.
> +        (WebCore::createAndFillGlyphPage): Deleted.
> +        * platform/graphics/Font.h:
> +        (WebCore::Font::shouldNotBeUsedForArabic): Deleted.
> +        * platform/graphics/cocoa/FontCascadeCocoa.mm:
> +        (WebCore::FontCascade::fontForCombiningCharacterSequence): Deleted.
> +        * platform/graphics/cocoa/FontCocoa.mm:
> +        (WebCore::fontFamilyShouldNotBeUsedForArabic): Deleted.
> +        (WebCore::Font::platformInit): Deleted.

This is now almost entirely wrong. Need to regenerate the list of affected functions.
Comment 23 Myles C. Maxfield 2015-06-08 21:21:21 PDT
Here's my raw data of performance of this patch:

iPhone 5s:      Time Std. Dev.
With patch:    1.496     0.102
Without patch: 1.523     0.105
With patch:    1.497     0.098
Without patch: 1.499     0.102
With patch:    1.507     0.104
Without patch: 1.490     0.103

iPad Mini:      Time Std. Dev.
With patch:    3.674     0.182
Without patch: 3.638     0.176
With patch:    3.610     0.144
Without patch: 3.630     0.249
With patch:    3.665     0.187
Without patch: 3.637     0.204
Comment 24 Myles C. Maxfield 2015-06-08 21:35:51 PDT
Created attachment 254545 [details]
Patch for landing
Comment 25 Myles C. Maxfield 2015-06-08 21:37:38 PDT
Metz: Given that the font works as expected, perf seems to be okay, and the patch has been positively reviewed, I'm going to land this if I don't hear from you within a day or two.
Comment 26 Myles C. Maxfield 2015-06-08 21:37:52 PDT
(In reply to comment #25)
> Metz: Given that the font works as expected, perf seems to be okay, and the
> patch has been positively reviewed, I'm going to land this if I don't hear
> from you within a day or two.

*Mitz
Comment 27 Myles C. Maxfield 2015-06-10 14:33:17 PDT
(In reply to comment #25)
> Metz: Given that the font works as expected, perf seems to be okay, and the
> patch has been positively reviewed, I'm going to land this if I don't hear
> from you within a day or two.

I have now heard from Mitz.
Comment 28 Myles C. Maxfield 2015-06-10 16:35:36 PDT
(In reply to comment #27)
> (In reply to comment #25)
> > Metz: Given that the font works as expected, perf seems to be okay, and the
> > patch has been positively reviewed, I'm going to land this if I don't hear
> > from you within a day or two.
> 
> I have now heard from Mitz.

Dan and I discussed a better way to fix this bug.
Comment 29 Myles C. Maxfield 2015-06-19 20:41:41 PDT
Created attachment 255273 [details]
Patch
Comment 30 Myles C. Maxfield 2015-06-19 23:37:16 PDT
Created attachment 255282 [details]
Patch
Comment 31 Myles C. Maxfield 2015-06-19 23:38:19 PDT
Created attachment 255283 [details]
Patch
Comment 32 WebKit Commit Bot 2015-06-19 23:40:07 PDT
Attachment 255283 [details] did not pass style-queue:


ERROR: Source/WebCore/platform/graphics/ios/FontCacheIOS.mm:450:  Multi line control clauses should use braces.  [whitespace/braces] [4]
Total errors found: 1 in 10 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 33 Darin Adler 2015-06-22 10:41:01 PDT
Comment on attachment 255283 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=255283&action=review

> Source/WebCore/platform/graphics/Font.h:72
> +#if PLATFORM(IOS)
> +bool fontFamilyShouldNotBeUsedForArabic(CFStringRef);
> +#endif

Pretty ugly to have a super-platform-specific function using a platform-specific type right here at the top of Font.h. Is there a better home for this?

> Source/WebCore/platform/graphics/ios/FontCacheIOS.mm:449
> +    if (RetainPtr<CFStringRef> foundFontName = adoptCF(static_cast<CFStringRef>(CTFontDescriptorCopyAttribute(fallbackFontDescriptor.get(), kCTFontNameAttribute)))) {

Could use auto instead of RetainPtr<CFStringRef>.

> Source/WebCore/platform/graphics/ios/FontCacheIOS.mm:451
> +            RetainPtr<CFStringRef> familyName = adoptCF(static_cast<CFStringRef>(CTFontDescriptorCopyAttribute(fallbackFontDescriptor.get(), kCTFontFamilyNameAttribute)));

Could use auto instead of RetainPtr<CFStringRef>.
Comment 34 Myles C. Maxfield 2015-06-22 11:44:25 PDT
Comment on attachment 255283 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=255283&action=review

>> Source/WebCore/platform/graphics/Font.h:72
>> +#endif
> 
> Pretty ugly to have a super-platform-specific function using a platform-specific type right here at the top of Font.h. Is there a better home for this?

I tried to find a better place, and I can't seem to find a better one. We don't have any platform-specific headers for font stuff (rather, we have platform-specific implementation files). I think the best I can do is to move this to the bottom of the file.
Comment 35 Myles C. Maxfield 2015-06-22 11:46:15 PDT
Created attachment 255358 [details]
Patch for landing
Comment 36 WebKit Commit Bot 2015-06-22 11:48:21 PDT
Comment on attachment 255358 [details]
Patch for landing

Rejecting attachment 255358 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.appspot.com', '--bot-id=webkit-cq-02', 'validate-changelog', '--check-oops', '--non-interactive', 255358, '--port=mac']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

ChangeLog entry in LayoutTests/ChangeLog contains OOPS!.

Full output: http://webkit-queues.appspot.com/results/5839308690817024
Comment 37 WebKit Commit Bot 2015-06-22 11:48:48 PDT
Attachment 255358 [details] did not pass style-queue:


ERROR: Source/WebCore/platform/graphics/ios/FontCacheIOS.mm:450:  Multi line control clauses should use braces.  [whitespace/braces] [4]
Total errors found: 1 in 10 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 38 Myles C. Maxfield 2015-06-22 13:54:14 PDT
Committed r185842: <http://trac.webkit.org/changeset/185842>
Comment 39 Alex Christensen 2015-06-23 10:27:24 PDT
fast/text/arabic-times-new-roman.html always fails on Windows.
Comment 40 Myles C. Maxfield 2015-06-23 14:10:36 PDT
Committed r185883: <http://trac.webkit.org/changeset/185883>