Bug 186010

Summary: REGRESSION (Mountain Lion-Mavericks): Incorrect font used to render decomposed grapheme cluster in some cases
Product: WebKit Reporter: mitz
Component: TextAssignee: Myles C. Maxfield <mmaxfield>
Status: NEW ---    
Severity: Normal CC: ap, benjamin, cdumez, cmarcelo, dbates, ews-watchlist, mmaxfield, rniwa
Priority: P2 Keywords: Regression
Version: Safari Technology Preview   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=185933
https://bugs.webkit.org/show_bug.cgi?id=68287
https://bugs.webkit.org/show_bug.cgi?id=186007
Attachments:
Description Flags
Mountain Lion
none
Mavericks
none
macOS Browsers
none
Edge
none
Edge
none
WIP
ews-watchlist: commit-queue-
Archive of layout-test-results from ews101 for mac-sierra
none
Archive of layout-test-results from ews104 for mac-sierra-wk2
none
Archive of layout-test-results from ews123 for ios-simulator-wk2
none
Archive of layout-test-results from ews117 for mac-sierra
none
Archive of layout-test-results from ews116 for mac-sierra none

Description mitz 2018-05-25 22:52:38 PDT
In the following test case:

<div style="font-family: verdana, times;">
    i&#x0302; and i&#x033f;
</div>

the first grapheme cluster (lowercase I with combining circumflex) is rendered in Times, but it should be rendered in Verdana.

This works correctly in OS X Mountain Lion 10.8.5, after <https://trac.webkit.org/r95391>, the fix for bug 68287, but got broken some time between then and r232220. The problem is that Font::canRenderCombiningCharacterSequence() started returning false for Verdana and the i+circumflex sequence.
Comment 1 mitz 2018-05-25 23:17:21 PDT
This got broken as soon as OS X 10.9 Mavericks.
Comment 2 mitz 2018-05-25 23:21:57 PDT
If this is a WebCore regression, this puts it between the tags Safari-536.30.2 and Safari-537.70.
Comment 3 mitz 2018-05-25 23:35:19 PDT
This doesn’t seem to be a WebKit regression: the bug happens in Safari 7.0 on Mavericks (537.71) but doesn’t happen in Safari 6.1.1 on Mountain Lion (537.73.11).

Unless there’s some platform-specific #ifs in the code (which there don’t seem to be), the regression is probably due to a change in Core Text’s behavior between Mountain Lion and Mavericks.
Comment 4 mitz 2018-05-25 23:50:14 PDT
Results of CTLineCreateWithUniCharProvider:

Mountain Lion:

CTLine: run count = 1, string range = (0, 2), width = 12.8975, A/D/L = 47.2524/9.86816/0, glyph count = 1
{

CTRun: string range = (0, 2), string = "i\u0302", attributes = <CFBasicHash 0x7fcdca035400 [0x7fff77f34110]>{type = mutable dict, count = 3,
entries =>
	0 : <CFString 0x7fff76938810 [0x7fff77f34110]>{contents = "NSKern"} = <CFNumber 0x7fcdca0531b0 [0x7fff77f34110]>{value = +0.0, type = kCFNumberFloat32Type}
	1 : <CFString 0x7fff76938830 [0x7fff77f34110]>{contents = "NSLigature"} = <CFNumber 0x87 [0x7fff77f34110]>{value = +0, type = kCFNumberSInt32Type}
	2 : <CFString 0x7fff769387f0 [0x7fff77f34110]>{contents = "NSFont"} = CTFont <name: Verdana, size: 47.000000, matrix: 0x0>
CTFontDescriptor <attributes: <CFBasicHash 0x7fcdca0257a0 [0x7fff77f34110]>{type = mutable dict, count = 1,
entries =>
	2 : <CFString 0x7fff76939ef0 [0x7fff77f34110]>{contents = "NSFontNameAttribute"} = <CFString 0x7fff77f182a0 [0x7fff77f34110]>{contents = "Verdana"}
}
>
}

}

Mavericks:

<CTLine: 0x7fd1b3c6f0d0>{run count = 1, string range = (0, 2), width = 56.2256, A/D/L = 40.8413/12.25/0.430664, glyph count = 1, runs = (

<CTRun: 0x7fd1b3c5bfd0>{string range = (0, 2), string = "i\u0302", attributes = <CFBasicHash 0x7fd1b3d54830 [0x7fff7c97aeb0]>{type = mutable dict, count = 3,
entries =>
	0 : <CFString 0x7fff7b915b30 [0x7fff7c97aeb0]>{contents = "NSKern"} = <CFNumber 0x7fd1b3c46160 [0x7fff7c97aeb0]>{value = +0.0, type = kCFNumberFloat32Type}
	1 : <CFString 0x7fff7b915b50 [0x7fff7c97aeb0]>{contents = "NSLigature"} = <CFNumber 0x27 [0x7fff7c97aeb0]>{value = +0, type = kCFNumberSInt32Type}
	2 : <CFString 0x7fff7b915b10 [0x7fff7c97aeb0]>{contents = "NSFont"} = <CTFont: 0x7fd1b3c61a20>{name = LastResort, size = 49.000000, matrix = 0x0, descriptor = <CTFontDescriptor: 0x7fd1b3d36330>{attributes = <CFBasicHash 0x7fd1b3d37520 [0x7fff7c97aeb0]>{type = mutable dict, count = 1,
entries =>
	2 : <CFString 0x7fff7b917390 [0x7fff7c97aeb0]>{contents = "NSFontNameAttribute"} = <CFString 0x7fff7bd5c150 [0x7fff7c97aeb0]>{contents = "LastResort"}
}
>}}
}
}

)
}
Comment 5 mitz 2018-05-25 23:51:10 PDT
(Ignore the sizes being 47 and 49 above, I used versions of the test case that had those sizes instead of the default).
Comment 6 Myles C. Maxfield 2018-05-26 11:27:28 PDT
I believe the right way to solve this is to normalize the string.
Comment 7 Myles C. Maxfield 2018-05-26 11:54:20 PDT
(In reply to Myles C. Maxfield from comment #6)
> I believe the right way to solve this is to normalize the string.

In the example in https://bugs.webkit.org/show_bug.cgi?id=186010#c0 Verdana doesn't support U+0302, but it does support U+00EE
Comment 8 mitz 2018-05-26 12:09:25 PDT
(In reply to Myles C. Maxfield from comment #6)
> I believe the right way to solve this is to normalize the string.

Is that how Core Text arrives at the right result?
Comment 9 mitz 2018-05-26 13:44:51 PDT
Created attachment 341407 [details]
Mountain Lion
Comment 10 mitz 2018-05-26 13:45:10 PDT
Created attachment 341408 [details]
Mavericks
Comment 11 mitz 2018-05-26 13:49:24 PDT
Attached screenshots showing how even the test case from bug 68287 regressed between Mountain Lion and Mavericks.

In Mountain Lion, Core Text had the ability to tell us that we could use Verdana for the combining character sequence i+circumflex, even though Verdana didn’t have a glyph for the combining circumflex alone. The fix for bug 68287 was to take advantage of this ability, which was already benefiting the Cocoa text system.

In Mavericks, that Core Text ability was gone. It regressed WebKit, but also the Cocoa text system, as can be seen with TextEdit.
Comment 12 mitz 2018-05-26 13:49:56 PDT
(In reply to mitz from comment #8)
> Is that how Core Text arrives at the right result?

Used to arrive :(
Comment 13 Myles C. Maxfield 2018-05-26 21:50:00 PDT
See also: <rdar://problem/40583639>
Comment 14 Myles C. Maxfield 2018-05-26 21:53:51 PDT
Created attachment 341418 [details]
macOS Browsers
Comment 15 Myles C. Maxfield 2018-05-26 22:03:26 PDT
Created attachment 341419 [details]
Edge
Comment 16 mitz 2018-05-26 22:04:50 PDT
(In reply to Myles C. Maxfield from comment #15)
> Created attachment 341419 [details]
> Edge

😑

Is it the same Verdana though? It’s possible that the version that ships with Windows these days does have a standalone glyph for combining circumflex.
Comment 17 Myles C. Maxfield 2018-05-27 22:47:05 PDT
(In reply to mitz from comment #16)
> (In reply to Myles C. Maxfield from comment #15)
> > Created attachment 341419 [details]
> > Edge
> 
> 😑
> 
> Is it the same Verdana though?

Nope!

> It’s possible that the version that ships
> with Windows these days does have a standalone glyph for combining
> circumflex.

You are correct - Windows's Verdana has a glyph for U+0302. I'll upload a new screenshot.
Comment 18 Myles C. Maxfield 2018-05-27 22:51:07 PDT
Created attachment 341447 [details]
Edge
Comment 19 Myles C. Maxfield 2018-05-28 21:19:11 PDT
Created attachment 341469 [details]
WIP
Comment 20 EWS Watchlist 2018-05-28 21:20:13 PDT
Attachment 341469 [details] did not pass style-queue:


ERROR: Source/WebCore/ChangeLog:8:  You should remove the 'No new tests' and either add and list tests, or explain why no new tests were possible.  [changelog/nonewtests] [5]
Total errors found: 1 in 10 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 21 EWS Watchlist 2018-05-28 22:16:18 PDT
Comment on attachment 341469 [details]
WIP

Attachment 341469 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/7841629

New failing tests:
imported/w3c/web-platform-tests/encoding/legacy-mb-japanese/shift_jis/sjis-encode-form-errors-misc.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-misc.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-errors-misc.html
fast/text/midword-break-before-surrogate-pair-2.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-href-errors-misc.html
fast/regex/dom/unicodeCaseInsensitive.html
fast/url/idna2008.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-href-errors-misc.html
imported/w3c/web-platform-tests/css/css-text/word-break/word-break-break-all-007.html
imported/w3c/web-platform-tests/encoding/legacy-mb-japanese/shift_jis/sjis-encode-href-errors-misc.html
imported/blink/fast/text/international/complex-text-leading-space-wrapping.html
Comment 22 EWS Watchlist 2018-05-28 22:16:20 PDT
Created attachment 341470 [details]
Archive of layout-test-results from ews101 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews101  Port: mac-sierra  Platform: Mac OS X 10.12.6
Comment 23 EWS Watchlist 2018-05-28 22:42:13 PDT
Comment on attachment 341469 [details]
WIP

Attachment 341469 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/7841733

New failing tests:
imported/w3c/web-platform-tests/encoding/legacy-mb-japanese/shift_jis/sjis-encode-form-errors-misc.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-misc.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-errors-misc.html
fast/text/midword-break-before-surrogate-pair-2.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-href-errors-misc.html
fast/regex/dom/unicodeCaseInsensitive.html
fast/url/idna2008.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-href-errors-misc.html
imported/w3c/web-platform-tests/css/css-text/word-break/word-break-break-all-007.html
imported/w3c/web-platform-tests/encoding/legacy-mb-japanese/shift_jis/sjis-encode-href-errors-misc.html
imported/blink/fast/text/international/complex-text-leading-space-wrapping.html
Comment 24 EWS Watchlist 2018-05-28 22:42:15 PDT
Created attachment 341472 [details]
Archive of layout-test-results from ews104 for mac-sierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-sierra-wk2  Platform: Mac OS X 10.12.6
Comment 25 EWS Watchlist 2018-05-28 23:16:43 PDT
Comment on attachment 341469 [details]
WIP

Attachment 341469 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/7841777

New failing tests:
imported/w3c/web-platform-tests/encoding/legacy-mb-japanese/shift_jis/sjis-encode-form-errors-misc.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-misc.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-errors-misc.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-href-errors-misc.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-href-errors-misc.html
fast/regex/dom/unicodeCaseInsensitive.html
fast/url/idna2008.html
imported/w3c/web-platform-tests/css/css-text/word-break/word-break-break-all-007.html
imported/w3c/web-platform-tests/encoding/legacy-mb-japanese/shift_jis/sjis-encode-href-errors-misc.html
imported/blink/fast/text/international/complex-text-leading-space-wrapping.html
Comment 26 EWS Watchlist 2018-05-28 23:16:45 PDT
Created attachment 341476 [details]
Archive of layout-test-results from ews123 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews123  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.4
Comment 27 EWS Watchlist 2018-05-28 23:22:25 PDT
Comment on attachment 341469 [details]
WIP

Attachment 341469 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/7841795

New failing tests:
imported/w3c/web-platform-tests/encoding/legacy-mb-japanese/shift_jis/sjis-encode-form-errors-misc.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-misc.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-errors-misc.html
fast/text/midword-break-before-surrogate-pair-2.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-href-errors-misc.html
fast/regex/dom/unicodeCaseInsensitive.html
fast/url/idna2008.html
imported/w3c/web-platform-tests/encoding/legacy-mb-japanese/shift_jis/sjis-encode-href-errors-misc.html
imported/blink/fast/text/international/complex-text-leading-space-wrapping.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-href-errors-misc.html
imported/w3c/web-platform-tests/css/css-text/word-break/word-break-break-all-007.html
Comment 28 EWS Watchlist 2018-05-28 23:22:27 PDT
Created attachment 341477 [details]
Archive of layout-test-results from ews117 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews117  Port: mac-sierra  Platform: Mac OS X 10.12.6
Comment 29 EWS Watchlist 2018-05-29 01:12:10 PDT
Comment on attachment 341469 [details]
WIP

Attachment 341469 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/7842513

New failing tests:
imported/w3c/web-platform-tests/encoding/legacy-mb-japanese/shift_jis/sjis-encode-form-errors-misc.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-misc.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-errors-misc.html
fast/text/midword-break-before-surrogate-pair-2.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-href-errors-misc.html
fast/regex/dom/unicodeCaseInsensitive.html
fast/url/idna2008.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-href-errors-misc.html
imported/w3c/web-platform-tests/css/css-text/word-break/word-break-break-all-007.html
imported/w3c/web-platform-tests/encoding/legacy-mb-japanese/shift_jis/sjis-encode-href-errors-misc.html
imported/blink/fast/text/international/complex-text-leading-space-wrapping.html
Comment 30 EWS Watchlist 2018-05-29 01:12:11 PDT
Created attachment 341481 [details]
Archive of layout-test-results from ews116 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116  Port: mac-sierra  Platform: Mac OS X 10.12.6