Bug 17965 – Font fallback broken for some Chinese PUA characters

• Status: NEW
• Component: Text

[Alexey Proskuryakov]

Steps to reproduce:
1. Open <http://bugs.webkit.org/attachment.cgi?id=19895>
2. Note that some characters in the right column are displayed as empty boxes (the characters in the left column are all displayed as boxes, that's bug 17964).
3. Copy one of those characters to TextEdit. It will be displayed with LastResort font, which is already better than an empty box.
4. Using Character Pallette or UnicodeChecker, verify that Mac OS X has several fonts that support this character.

These characters are all part of windows-936 Chinese encoding.

[Alexey Proskuryakov]

<rdar://problem/5810055>

[Myles C. Maxfield]

When we open a webpage, we unconditionally translate the page from whatever character encoding the page is in to Unicode. GBK is an encoding used for Chinese, and includes 14 characters which do not have associated Unicode codepoints, so they are assigned codepoints in the private use area. Chinese fonts have glyphs for these particular codepoints. We should assume that all documents using the GBK character set should be using a chinese “lang” attribute (unless otherwise specified). That would change our fallback font to one that has these particular glyphs.

[Myles C. Maxfield]

Created attachment 242006 [details]
Patch

[Alexey Proskuryakov]

Comment on attachment 242006 [details]
Patch

I think that we have a separate bug about deducing language from charset.

[Myles C. Maxfield]

Created attachment 242008 [details]
Patch

[Build Bot]

Comment on attachment 242008 [details]
Patch

Attachment 242008 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/5184212248297472

Number of test failures exceeded the failure limit.

[Build Bot]

Created attachment 242011 [details]
Archive of layout-test-results from webkit-ews-05 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-05  Port: mac-mountainlion  Platform: Mac OS X 10.8.5

[Myles C. Maxfield]

Created attachment 242012 [details]
Patch

[Build Bot]

Comment on attachment 242012 [details]
Patch

Attachment 242012 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.appspot.com/results/4825826419802112

New failing tests:
fast/text/international/003.html
fast/css/line-height-determined-by-primary-font.html
fast/text/backslash-to-yen-sign-euc.html
fast/text/international/001.html

[Build Bot]

Created attachment 242014 [details]
Archive of layout-test-results from webkit-ews-15 for mac-mountainlion-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: webkit-ews-15  Port: mac-mountainlion-wk2  Platform: Mac OS X 10.8.5

[Build Bot]

Comment on attachment 242012 [details]
Patch

Attachment 242012 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/5104930574565376

New failing tests:
fast/text/international/003.html
fast/css/line-height-determined-by-primary-font.html
fast/text/backslash-to-yen-sign-euc.html
fast/text/international/001.html

[Build Bot]

Created attachment 242015 [details]
Archive of layout-test-results from webkit-ews-07 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-07  Port: mac-mountainlion  Platform: Mac OS X 10.8.5