Bug 112488 – REGRESSION (r145854): Mongolian characters are not upright in vertical writing mode

• Status: REOPENED
• Component: Layout and Rendering

[SiqinBilige]

Created attachment 193415 [details]
Actual screenshot

On Mac OS 7 r145930 (-webkit-writing-mode: vertical-lr)
Webkit completely can not display and edit Traditional Mongolian Language vertical way.

[SiqinBilige]

Created attachment 193416 [details]
evidence

[SiqinBilige]

Created attachment 193417 [details]
evidence

[Ryosuke Niwa]

These screenshots are not self-evident. Please describe what's expected and what WebKit does incorrectly.

[SiqinBilige]

Created attachment 193418 [details]
Expected screenshot

glyphs rotate 90 deg.

Caused by <http://trac.webkit.org/r145854>. Note that this has been broken before and then fixed in <http://trac.webkit.org/r124654>, but the test from that change didn’t detect this recent regression.

[Radar WebKit Bug Importer]

<rdar://problem/13436413>

[Enrica Casucci]

Prior to http://trac.webkit.org/r145854 we did not have the logic to avoid rotation for some characters.
When text is rendered using the fast path we choose the correct font and the appropriate glyph depending on whether or not the font has vertical glyphs.
Unfortunately this information doesn't filter down when we must use the complex path to render the text.
I'll fix it.

[SiqinBilige]

The latest version of Google Chrome on Mac OS and Android also can not display Traditional Mongolian Language vertical way.

If the same thing happen on next version of Safari, there is no browser on Mac OS X , iOS and Android can correctly display Traditional Mongolian Language vertical way.

The Traditional Mongolian UNICODE range is from 1800 to 18AF.
http://www.unicode.org/charts/PDF/U1800.pdf
The Traditional Mongolian Language is vertical only language and can not have space between glyphs in one word.

Any one can help on this bug for 10 million mongolian people who using Traditional Mongolian Language in the world ?

Thank you.

[SiqinBilige]

I read the latest (2013/05/24) source code of trunk/Source/WebCore/platform/graphics/FontFastPath.cpp.

It can fixed if change
    if (isInRange(character, 0x01100, 0x011FF) || isInRange(character, 0x01401, 0x0167F) || isInRange(character, 0x01800, 0x018FF))
        return true;
to
    if (isInRange(character, 0x01100, 0x011FF) || isInRange(character, 0x01401, 0x0167F) || isInRange(character, 0x018B0, 0x018FF))
        return true;
in the shouldIgnoreRotation function.

Traditional Mongolian(0x01800-0x018AF) Character need to Rotate in vertical mode(both lr and rl).

[Koji Ishii]

Mongolian and Phags Pa are a bit tricky in UTR#50 and CSS Writing Modes Level 3.

UTR#50 defines glyph orientation against Unicode code chart. Because Mongolian and Phags Pa are vertical only scripts, the code chart has glyphs for vertical flow, so UTR#50 must define them as U (upright against the code chart.)

However, some font systems have pre-rotated glyphs for these scripts. This is mentioned in CSS Writing Modes Level 3:
 # In some systems (e.g. when using OpenType fonts), to correctly orient
 # a character belonging to the Mongolian or Phags-pa script upright, the
 # UA must actually typeset it sideways.
http://dev.w3.org/csswg/css-writing-modes/#vertical-orientations

So such systems must change the interpretation of UTR#50 U value to R for Mongolian and Phags Pa, specifically:
U+1800-18AF
U+A840-A87F

[SiqinBilige]

Created attachment 202904 [details]
Patch

[SiqinBilige]

The Traditional Mongolian Language's code and glyph not one to one mapping.
For each code there is 4(isol,init,medi,fina)X3(fvs1,fvs2,fvs3) glyphs(using private area) in maximum.
For a history reason(from DOS ?), the glyph in the font is horizontal way, not vertical way. so it need to rotate 90 clockwise in vertical mode(lr,rl).

[SiqinBilige]

Created attachment 202907 [details]
The Traditional Mongolian font structure

[SiqinBilige]

The Traditional Mongolian font can download from www.mongolfont.com.
For Windows 7,8 (OpenType)
   http://www.mongolfont.com/jAlmas/cms/documents/mongolfont/font/mnglwhiteotf.ttf
For Mac OS X (AAT)
  http://www.mongolfont.com/jAlmas/cms/documents/mongolfont/font/mnglwhiteaat.ttf

[SiqinBilige]

The shouldIgnoreRotation function removed from FontFastPath.cpp to FontFallbackList.cpp by [e-mail] in 2013/05/27 04:05.
What can I do next ?  Delete the patch and upload new one ?

[SiqinBilige]

The Phags Pa ( A type of Mongolian since 1269 ) has same layout  (top to bottom and left to right) as Traditiona Mongolian.

[SiqinBilige]

05/29/13, antti, Rename FontFallbackList.cpp/h (whitch include the shouldIgnoreRotation function ) to FontGlyphs.cpp/h.

[SiqinBilige]

Created attachment 203666 [details]
Patch

[Build Bot]

Comment on attachment 203666 [details]
Patch

Attachment 203666 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.appspot.com/results/660659

New failing tests:
platform/mac/fast/text/vertical-no-sideways.html

[Build Bot]

Created attachment 203668 [details]
Archive of layout-test-results from webkit-ews-11 for mac-mountainlion-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: webkit-ews-11  Port: mac-mountainlion-wk2  Platform: Mac OS X 10.8.3

[Build Bot]

Comment on attachment 203666 [details]
Patch

Attachment 203666 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/714509

New failing tests:
platform/mac/fast/text/vertical-no-sideways.html

[Build Bot]

Created attachment 203714 [details]
Archive of layout-test-results from webkit-ews-01 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-01  Port: mac-mountainlion  Platform: Mac OS X 10.8.3

[SiqinBilige]

Created attachment 203770 [details]
Patch

[Build Bot]

Comment on attachment 203770 [details]
Patch

Attachment 203770 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/769057

New failing tests:
platform/mac/fast/text/vertical-no-sideways.html

[Build Bot]

Created attachment 203775 [details]
Archive of layout-test-results from webkit-ews-07 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-07  Port: mac-mountainlion  Platform: Mac OS X 10.8.3

[SiqinBilige]

Created attachment 203788 [details]
Patch

[Build Bot]

Comment on attachment 203788 [details]
Patch

Attachment 203788 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/784055

New failing tests:
platform/mac/fast/text/vertical-no-sideways.html

[Build Bot]

Created attachment 203858 [details]
Archive of layout-test-results from webkit-ews-04 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-04  Port: mac-mountainlion  Platform: Mac OS X 10.8.3

[Build Bot]

Comment on attachment 203788 [details]
Patch

Attachment 203788 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.appspot.com/results/695429

New failing tests:
platform/mac/fast/text/vertical-no-sideways.html

[Build Bot]

Created attachment 203898 [details]
Archive of layout-test-results from webkit-ews-16 for mac-mountainlion-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: webkit-ews-16  Port: mac-mountainlion-wk2  Platform: Mac OS X 10.8.3

[SiqinBilige]

Created attachment 203899 [details]
Patch

[Darin Adler]

Comment on attachment 203899 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=203899&action=review

> LayoutTests/ChangeLog:9
> +        * platform/mac/fast/text/vertical-no-sideways.html: Removed 0x01800 and 0x01801.
> +        * platform/mac/platform/mac/fast/text/vertical-no-sideways-expected.txt: Removed 0x01800 and 0x01801.

Why is removing this the right thing to do? It seems like we are testing less now. Why not still test this to make sure things are working properly?

A patch that fixes bugs normally needs to include new tests that cover what was fixed. Tests that fail before the fix and succeed after the fix.

[Build Bot]

Comment on attachment 203899 [details]
Patch

Attachment 203899 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.appspot.com/results/745375

New failing tests:
platform/mac/fast/text/vertical-no-sideways.html

[Build Bot]

Created attachment 203902 [details]
Archive of layout-test-results from webkit-ews-06 for mac-mountainlion

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: webkit-ews-06  Port: mac-mountainlion  Platform: Mac OS X 10.8.3

[Erdene-Ochir Tuguldur]

(In reply to comment #8)
> The latest version of Google Chrome on Mac OS and Android also can not display Traditional Mongolian Language vertical way.
> 
> If the same thing happen on next version of Safari, there is no browser on Mac OS X , iOS and Android can correctly display Traditional Mongolian Language vertical way.
> 
> The Traditional Mongolian UNICODE range is from 1800 to 18AF.
> http://www.unicode.org/charts/PDF/U1800.pdf
> The Traditional Mongolian Language is vertical only language and can not have space between glyphs in one word.
> 
> Any one can help on this bug for 10 million mongolian people who using Traditional Mongolian Language in the world ?
> 
> Thank you.

Hallo SiqinBilige,

you have to use "-webkit-text-orientation: sideways-right;" as workaround so that Mongolian are shaped correctly. Here is my demo page: https://dl.dropboxusercontent.com/u/16982795/mongol/test1.html

I have tested with Chrome and OperaNext on Mac. Both browsers can shape the Mongolian correctly (there is also another bug with 0x202F, see: https://code.google.com/p/chromium/issues/detail?id=155334).

Tuguldur

[SiqinBilige]

(In reply to comment #35)
> (In reply to comment #8)
> > The latest version of Google Chrome on Mac OS and Android also can not display Traditional Mongolian Language vertical way.
> > 
> > If the same thing happen on next version of Safari, there is no browser on Mac OS X , iOS and Android can correctly display Traditional Mongolian Language vertical way.
> > 
> > The Traditional Mongolian UNICODE range is from 1800 to 18AF.
> > http://www.unicode.org/charts/PDF/U1800.pdf
> > The Traditional Mongolian Language is vertical only language and can not have space between glyphs in one word.
> > 
> > Any one can help on this bug for 10 million mongolian people who using Traditional Mongolian Language in the world ?
> > 
> > Thank you.
> 
> Hallo SiqinBilige,
> 
> you have to use "-webkit-text-orientation: sideways-right;" as workaround so that Mongolian are shaped correctly. Here is my demo page: https://dl.dropboxusercontent.com/u/16982795/mongol/test1.html
> 

Tuguldur

Thank you for your advise.
You are right, using "-webkit-text-orientation: sideways-right;" can display Traditional Mongolian Vertical way on Webkit like browsers.
But it is not good solution for the writing-mode:vertica-lr.
We are not only consider Traditional Mongolian only document.
We are must consider Traditional Mongolian mixed with other language document too. such as with CJK and Other symbols used in Traditional Mongolian document.

We believe you can understand our considering, after carefully look over screenshots below.

1. On Windows IE, correct CJK and symbols layout.
  mongolian_mixed_win_ie.png

2. On Windows Chrome, not correct CJK and symbols layout. (I can not download latest version of Webkit for windows)
  mongolian_mixed_win_chrome_no.png

3. On Windows Chrome (with "-webkit-text-orientation: sideways-right;"), not correct CJK and symbols layout.
  mongolian_mixed_win_chrome.png

4. On Mac OS Webkit, correct CJK and symbols layout.
  mongolian_mixed_mac_webkit_no.png

5. On Mac OS Webkit (with "-webkit-text-orientation: sideways-right;"), not correct CJK and symbols layout.
  mongolian_mixed_mac_webkit.png

It is what we are struggling here to fix this bug or want to fixed by some one else first.

> I have tested with Chrome and OperaNext on Mac. Both browsers can shape the Mongolian correctly (there is also another bug with 0x202F, see: https://code.google.com/p/chromium/issues/detail?id=155334).
> 
There is no bug about 0x202F.
You can confirm it look over the last line of screenshots.

But, there is a lot of other bugs about Traditional Mongolian(-webkit-writing-mode: vertical-lr;) forms in Webkit.
  https://bugs.webkit.org/show_bug.cgi?id=117228
  https://bugs.webkit.org/show_bug.cgi?id=117229
  https://bugs.webkit.org/show_bug.cgi?id=117231
  https://bugs.webkit.org/show_bug.cgi?id=117233

[SiqinBilige]

Created attachment 203998 [details]
mongolian_mixed_win_ie.png

[SiqinBilige]

Created attachment 203999 [details]
mongolian_mixed_win_chrome.png

[SiqinBilige]

Created attachment 204000 [details]
mongolian_mixed_win_chrome_no.png

[SiqinBilige]

Created attachment 204001 [details]
mongolian_mixed_mac_webkit.png

[SiqinBilige]

Created attachment 204002 [details]
mongolian_mixed_mac_webkit_no.png

[Erdene-Ochir Tuguldur]

> > I have tested with Chrome and OperaNext on Mac. Both browsers can shape the Mongolian correctly (there is also another bug with 0x202F, see: https://code.google.com/p/chromium/issues/detail?id=155334).
> > 
> There is no bug about 0x202F.
> You can confirm it look over the last line of screenshots.

Well I think you have testet on Mac with Safari which uses AAT font. When you open the test page with Chrome on Mac or Linux, you can see that the word suffixes are shaped wrong. Pango/Linux got also same problem: https://bugzilla.gnome.org/show_bug.cgi?id=701652

[SiqinBilige]

Created attachment 204294 [details]
Patch

[SiqinBilige]

What is the meaning of yellow color on mac-wk2 ?

[Ryosuke Niwa]

(In reply to comment #44)
> What is the meaning of yellow color on mac-wk2 ?

It means that it's still processing. Maybe it's stuck?

[SiqinBilige]

(In reply to comment #45)
> (In reply to comment #44)
> > What is the meaning of yellow color on mac-wk2 ?
> 
> It means that it's still processing. Maybe it's stuck?

The last two run is like this :
Unable to pass tests without patch (tree is red?) [results] 1 day, 8 hours ago
Pass 1 day, 9 hours ago
It is seeming to be stuck.

[SiqinBilige]

Created attachment 204687 [details]
Patch

[Enrica Casucci]

Comment on attachment 204687 [details]
Patch

I don't think this is the right approach to the problem.
Mongolian text is rendered with the complex path and the reason why this fails is because the font has vertical glyphs but this information gets lost when we get to the point of preparing the glyphs to draw.
This fix will solve the problem only for Mongolian characters but not for all the other characters for which we should avoid rotation (those included in shouldAvoidRotation) when rendered with a font that has vertical glyphs for those characters.

[SiqinBilige]

(In reply to comment #48)
> (From update of attachment 204687 [details])
> I don't think this is the right approach to the problem.
> Mongolian text is rendered with the complex path and the reason why this fails is because the font has vertical glyphs but this information gets lost when we get to the point of preparing the glyphs to draw.
> This fix will solve the problem only for Mongolian characters but not for all the other characters for which we should avoid rotation (those included in shouldAvoidRotation) when rendered with a font that has vertical glyphs for those characters.

Yes, you are right.
We believe you did work according the UTR#50. and you are did the best.
The problem is that the mongolian definitions in the unicode is not clear or the document about mongolian is not clear.
The fact is that all Existing Mongolian and Phags-Pa OpenType/TrueType/AAT fonts have counter-clockwise rotated glyphs.
We wrote a report about this in http://www.unicode.org/review/pri253/feedback.html.
May be there is mistake about english grammar, because  of my poor english.
The fast is that before this patch the all traditional home page is All OK, but is broken after the patch.

[SiqinBilige]

(In reply to comment #3)
> These screenshots are not self-evident. Please describe what's expected and what WebKit does incorrectly.

この問題ついて、これからどうれすよいでしょうか、教えてください。

[SiqinBilige]

(In reply to comment #10)
> Mongolian and Phags Pa are a bit tricky in UTR#50 and CSS Writing Modes Level 3.
> 
> UTR#50 defines glyph orientation against Unicode code chart. Because Mongolian and Phags Pa are vertical only scripts, the code chart has glyphs for vertical flow, so UTR#50 must define them as U (upright against the code chart.)
> 
> However, some font systems have pre-rotated glyphs for these scripts. This is mentioned in CSS Writing Modes Level 3:
>  # In some systems (e.g. when using OpenType fonts), to correctly orient
>  # a character belonging to the Mongolian or Phags-pa script upright, the
>  # UA must actually typeset it sideways.
> http://dev.w3.org/csswg/css-writing-modes/#vertical-orientations
> 
> So such systems must change the interpretation of UTR#50 U value to R for Mongolian and Phags Pa, specifically:
> U+1800-18AF
> U+A840-A87F


この問題ついて、これからどうれすよいでしょうか、教えてください。

[SiqinBilige]

We download the latest version of webkit source and build it on windows.
We confirmed that There is no such a problem on windows platform.
Including Latest version of Other browsers like Google Chrome, Google Chromium, Opera Next, Safari.

But, There is a problem on Mac OS X and iOS and Android platform.
Including Webkit, Safari 6.1 Safari 7, Safari of iOS7
and latest version of Google Chrome,Opera Next.

The Google Chromium on Android is OK, but on Mac OS X did not.

[SiqinBilige]

Created attachment 204971 [details]
Webkit on windows OK

[Enrica Casucci]

(In reply to comment #53)
> Created an attachment (id=204971) [details]
> Webkit on windows OK

You are correct. The problem does not occur on Windows, because the bug is specific of the Mac complex path for text drawing. The exclusion from rotation based on UTR#50 is correct, but the code in the complex text drawing path fails to handle correctly the case of fonts with vertical glyphs.
I'm pushing back on your fix, simply because it fixes the problem for mongolian characters but leaves it broken for all the fonts that have vertical glyphs for characters that should not be rotated.

[SiqinBilige]

(In reply to comment #54)
> (In reply to comment #53)
> > Created an attachment (id=204971) [details] [details]
> > Webkit on windows OK
> 
> You are correct. The problem does not occur on Windows, because the bug is specific of the Mac complex path for text drawing. The exclusion from rotation based on UTR#50 is correct, but the code in the complex text drawing path fails to handle correctly the case of fonts with vertical glyphs.
Thank you for the thorough explanation.
> I'm pushing back on your fix, simply because it fixes the problem for mongolian characters but leaves it broken for all the fonts that have vertical glyphs for characters that should not be rotated.
> it fixes the problem for mongolian characters
If my patch can fixes the problem for mongolian characters and no minus efect on other conponent, we hope this patch is landed as soon as possible.
Becaurse, The all traditional mongolian sites were broken on latest version of chrome and operanext. We also confirmed that all traditional mongolian sites were broken on prerelease of Safari 6.1, Safari 7 and Safari of iOS7.
When thoese browsers are released, there weill be no browsers can correctly display existing traditional mongolian sites on Mac OS X and iOS.
> but leaves it broken for all the fonts that have vertical glyphs for characters that should not be rotated.
I did not knew about it.
It is true that I unable to read the all source code about drawing text.
But, I tested some characters(not all) which should not rotated.
It seems to no problem on both vertical_rl and vertical_lr.
http://www.mongolfont.com/test/vertical-no-sideways.html
horizontal.png
vertical_rl.png
vertical_lr.png

[SiqinBilige]

Created attachment 205240 [details]
writing-mode : horizontal

[SiqinBilige]

Created attachment 205241 [details]
writing-mode : vertical_rl

[SiqinBilige]

Created attachment 205242 [details]
writing-mode : vertical_lr

[SiqinBilige]

In UTR50 version 11, Mongolian(1800..18AF) and Phags-Pa(A840..A87F) moved from U to R.
http://www.unicode.org/reports/tr50/tr50-11.html

[Alexey Proskuryakov]

We can no longer reproduce this fixed in OS X Yosemite and is iOS 8.1.

[Alexey Proskuryakov]

We can no longer reproduce this in OS X Yosemite and in iOS 8.1. Please verify that this is fixed for you.

[SiqinBilige]

(In reply to comment #61)
> We can no longer reproduce this in OS X Yosemite and in iOS 8.1. Please
> verify that this is fixed for you.

In UTR50 version 11, Mongolian(1800..18AF) and Phags-Pa(A840..A87F) moved from U to R.
http://www.unicode.org/reports/tr50/tr50-11.html

As shown in the screenshot correct.png, it is what it should be expected, but in screenshot incorrect.png, the problem is still existing. Please pay attention to the textarea of vertical-lr(Mongolian) specifically in both screenshots. Where all these screenshots are taken from following page: http://www.mongolfont.com/test/webkit/div.html. 
Also you could test it by yourself using the URL.

[SiqinBilige]

Created attachment 241535 [details]
correct.png

[SiqinBilige]

Created attachment 241536 [details]
incorrect.png

[Koji Ishii]

(In reply to comment #54)
> The exclusion from
> rotation based on UTR#50 is correct, but the code in the complex text
> drawing path fails to handle correctly the case of fonts with vertical
> glyphs.

Sorry for very very late follow-up, but I'm sorry to say that you misunderstood UTR#50. In short, Mogolian and Phags-Pa should be excluded from rotation, even if their fonts have vertical glyphs. I haven't looked at the details of the patch, but the approach is correct.

A long version. Up to rev. 10 of UTR#50, Mongolian and Phags-Pa were U, just because UTR#50 is defined against characters in the Unicode code charts, and these characters in the code charts are pre-rotated, so U means that "render as in the code charts."

The Unicode Technical Committee was aware of that almost every single font in the wild are pre-rotated, and therefore implementers must interpret U in the table to be R for Mongolian and Phags-Pa, but because fonts being pre-rotated is not defined in Unicode, and because code charts were already there, we hoped implementers read it correctly.

After revision 10 was published, we found that we were too optimistic. One implementer, WebKit, implemented Mongolian and Phags-Pa as U. By that time, we had a luck to update the Unicode Standard, so we changed the orientation of these characters in the code charts and updated UTR#50 to R.

Hope this makes sense.