Bug 6148

Summary: WebKit doesn't shape characters (like Arabic) across style changes
Product: WebKit Reporter: Rosyna <webkit-bugs>
Component: Layout and RenderingAssignee: Myles C. Maxfield <mmaxfield>
Status: NEW    
Severity: Normal CC: abuzakham, ah.abuali2, ahmad.moshref, amir.aharoni, ap, bashar.harfoush, behdad, brettw, caddish-figures-03, cedric, daniel, dbates, dermot_rourke, ehsan, eric, fantasai.bugs, farev88999, glenn, hassan_deldar, hosskhalifa, ian, ishida, johnfeirick, kilise_90, krinklemail, mehmetgelisin, mgoszcz2, michellerodri247, mitz, mmaxfield, mostafa.h, mrbencem, munzirtaha, mustafa.0x, mustafamsy, nickshanks, noonon, peter, playmobil, rik, sabouhallawa, seikwon.kim, shagarah, shokeenali, simon.fraser, siqinbilige, webkit-bug-importer, zalan
Priority: P2 Keywords: BrowserCompat, InRadar
Version: 420+   
Hardware: All   
OS: All   
URL: http://blogs.msdn.com/michkap/archive/2005/12/19/505309.aspx
See Also: https://bugs.webkit.org/show_bug.cgi?id=222734
https://bugs.webkit.org/show_bug.cgi?id=229421
https://bugs.webkit.org/show_bug.cgi?id=282444
Bug Depends on:    
Bug Blocks: 229361, 47213    
Attachments:
Description Flags
Arabic Style Shaping test.
none
test from bug 17116
none
Testcase from bug 91975 none

Rosyna
Reported 2005-12-19 04:47:00 PST
See the attached html document. The runs should look the same, but with different colorings as they do in IE for Windows.
Attachments
Arabic Style Shaping test. (515 bytes, text/html)
2005-12-19 04:47 PST, Rosyna 		no flags
Details
test from bug 17116 (1.41 KB, text/html)
2008-01-31 23:21 PST, Alexey Proskuryakov 		no flags
Details
Testcase from bug 91975 (331 bytes, text/html)
2012-07-23 02:50 PDT, Jeremy Moskovich 		no flags
Details
View All Add attachment proposed patch, testcase, etc.
Rosyna
Comment 1 2005-12-19 04:47:44 PST
Created attachment 5149 [details] Arabic Style Shaping test.
Alexey Proskuryakov
Comment 2 2008-01-31 13:11:48 PST
*** Bug 17116 has been marked as a duplicate of this bug. ***
mitz
Comment 3 2008-01-31 17:20:30 PST
<rdar://problem/5718885>
Alexey Proskuryakov
Comment 4 2008-01-31 23:21:37 PST
Created attachment 18841 [details] test from bug 17116 The original tests pass in Firefox 3 beta, but some of these do not.
Jeremy Moskovich
Comment 5 2009-01-07 20:13:11 PST
Filed in Chrome as http://code.google.com/p/chromium/issues/detail?id=6122
Eric Seidel (no email)
Comment 6 2011-03-25 03:19:37 PDT
*** Bug 47213 has been marked as a duplicate of this bug. ***
Glenn Adams
Comment 7 2011-05-30 22:45:12 PDT
If nobody is actively working on this, I'm willing to take a pass at a patch.
Jeremy Moskovich
Comment 8 2011-05-30 23:46:02 PDT
Go for it!
Glenn Adams
Comment 9 2011-06-03 20:57:45 PDT
Initial investigation shows that BidiResolver::createBidiRunsForLine is breaking runs at a display:inline element boundary, e.g., span, even if the eventual embedding levels on both side of the boundary are the same. This causes RenderBlock::createLineBoxesFromBidiRuns to create distinct inline boxes across this boundary, preventing eventual complex text shaping from applying shaping context across the boundary.
Eric Seidel (no email)
Comment 10 2011-06-06 17:35:13 PDT
Yes. LineBoxes have a pointer back to their renderer and do not span renderers.
Alexey Proskuryakov
Comment 11 2011-06-21 10:44:12 PDT
*** Bug 63038 has been marked as a duplicate of this bug. ***
Munzir Taha (منذر طه)
Comment 12 2011-11-26 12:43:52 PST
Hi, I just encountered this bug while I am loading a page using QWebView *view = new QWebView(); view->load(QUrl("test.html")); while the test.html contains <!DOCTYPE HTML> <html> <head> <meta charset=utf8> <style type="text/css"> p:first-letter { color: red; } </style> </head> <body> The two arabic letters should apear like عر but they really show as <p>عر</p> and <div><span>ع</span>ر</div> </body> </html> Any update on this?
Glenn Adams
Comment 13 2011-11-26 13:55:20 PST
(In reply to comment #12) > Any update on this? Not yet.
Dermot Rourke
Comment 14 2012-04-06 08:12:19 PDT
I was wondering if there's been any progress made on this issue? I've come accross a number of Arabic Language Learning websites that rely on the ability to highlight specific letters within a word for teaching purposes. For example: http://transliteration.org/quran/WebSite_CD/HighlightSample/Fram3.htm http://arabiccomplete.com/modules_colloquial_msa/possessive_suffix_1.htm http://www.dalilusa.com/arabic_course/exercise02.asp Thanks.
Glenn Adams
Comment 15 2012-04-06 09:20:06 PDT
(In reply to comment #14) > I was wondering if there's been any progress made on this issue? I'm trying to get this back on my priority list, and there is a good chance I will do so in the next four weeks.
Dermot Rourke
Comment 16 2012-05-09 08:16:44 PDT
Just checking in to see if any progress has been made? A colleague of mine found a temporary work-around that may be useful to some developers in some scenarios - using the zero-width-joiner (&zwj;/&#8205;) will force the letters to join (or, at least, appear joined). Of course, it's not ideal as you'll need to test for the browser and insert them on page load (or something along those lines). Also, it does not work for every situation - in the example in comment #12 above, the css selector fails. The code would look something like: <!DOCTYPE HTML> <html> <head> <meta charset=utf8> <style type="text/css"> p:first-letter { color: red; } p, div { font-family: times new roman; font-size: x-large; } </style> </head> <body> The two arabic letters should appear like عر but they really show as <p>&zwj;عر</p> and <div><span>ع&zwj;</span>ر</div> </body> </html>
Jeremy Moskovich
Comment 17 2012-07-23 02:44:57 PDT
*** Bug 91975 has been marked as a duplicate of this bug. ***
Jeremy Moskovich
Comment 18 2012-07-23 02:50:40 PDT
Created attachment 153763 [details] Testcase from bug 91975 Amir Aharoni's simple testcase from bug 91975
Jeremy Moskovich
Comment 19 2012-07-23 02:52:52 PDT
Also tracked as http://crbug.com/138434 (http://crbug.com/6122 tracks the color issue)
Glenn Adams
Comment 20 2012-09-13 23:09:07 PDT
this is (finally) on the top of my queue, so assigning to myself
Eric Seidel (no email)
Comment 21 2012-10-27 00:27:53 PDT
*** Bug 77790 has been marked as a duplicate of this bug. ***
Eric Seidel (no email)
Comment 22 2012-10-27 00:29:06 PDT
If you're still working on this Glenn, we should chat.
Nasser Al-Wohaibi
Comment 23 2012-12-01 02:50:34 PST
@Dermot Rourke, to fix this entirely, use TWO zero-width-joiners e.g. <p>عرب&#x200d;<span style="color: Red;">&#x200d;ي</span></p> e.g. <!DOCTYPE HTML> <html> <head> <meta charset=utf8> <style type="text/css"> body{font-size:40px;} .test{ color:red; font-weight:bolder; } </style> </head> <body> The two arabic letters should apear like عربي but they really show as the following in webkit(chrome,safari) <p>عرب<span>ي</span></p> solution: <p>عرب&#x200d;<span style="color: Red;">&#x200d;ي</span></p> </body> </html>​ demo: http://jsfiddle.net/noonon/esz4S/2/
Hamzeh
Comment 24 2012-12-07 10:05:32 PST
Hi, any update on this issue ? My case running Version 23.0.1271.95, i tried to work around the issue with the zero-width-joiner or double zero-width-joiner, still the Arabic letter shapes appears broken. Coloring part of Arabic words is a common practice used in Arabic learning sites, currently we recommend our users to switch to other browsers as (Firefox, IE ) in order to render pages correctly.
Nasser Al-Wohaibi
Comment 25 2012-12-07 22:34:28 PST
@Hamzeh if you can paste some code samples? I might be of some help
Glenn Adams
Comment 26 2012-12-07 23:17:23 PST
it is not necessary to provide any more examples; the problem is well understood; however, the solution requires working around certain design limitations that aren't straightforward
Hamzeh
Comment 27 2012-12-08 03:16:57 PST
(In reply to comment #25) > @Hamzeh > if you can paste some code samples? > I might be of some help Nasser, I've followed the demo link provided by you and the same Arabic shape problem existed with my chrome version. trying to color part of the Arabic word fails on chrome regardless of using zero-width-joiner or not. while Firefox and IE rendering engines are working just fine with or without zero-width-joiner. this defect is vital for learning sites, since coloring part of the word is widely used to identify prefixes, suffixes, and certain language characteristics. I hope that someone from webkit to give it priority, it's a very important for languages such as Arabic, Persian, and Urdo.
Glenn Adams
Comment 28 2012-12-10 09:53:00 PST
*** Bug 104530 has been marked as a duplicate of this bug. ***
Bashar
Comment 29 2014-01-31 18:28:18 PST
Are there any updates on this bug?
Myles C. Maxfield
Comment 30 2014-02-03 14:27:24 PST
Not as of yet.
Bashar
Comment 31 2014-02-10 00:51:31 PST
Is there a way we can give this bug a higher priority(possibly Major)? The ability to style individual characters is very important for educational and word-game apps but it's currently broken for all sites that use complex script.
Glenn Adams
Comment 32 2014-02-10 07:06:08 PST
(In reply to comment #31) > Is there a way we can give this bug a higher priority(possibly Major)? The ability to style individual characters is very important for educational and word-game apps but it's currently broken for all sites that use complex script. Raising the priority on the bug won't make it get fixed faster if there is no body willing to take on the work, which is not going to be trivial. The fundamental problem is that the character to glyph shaping process in WK doesn't make use of any context that crosses an element boundary. Fixing this will most likely introduce a performance regression in the slow text path, which is already slow. There are at least two temporary work arounds for this that authors may use. One is document in comment #16. The other is to specifically code for presentation forms (U+FB50-FDFF, FE70-FEFC). This isn't ideal, but it works.
Bashar
Comment 33 2014-02-10 11:46:37 PST
(In reply to comment #32) > Raising the priority on the bug won't make it get fixed faster if there is no body willing to take on the work, which is not going to be trivial. The fundamental problem is that the character to glyph shaping process in WK doesn't make use of any context that crosses an element boundary. Fixing this will most likely introduce a performance regression in the slow text path, which is already slow. > > There are at least two temporary work arounds for this that authors may use. One is document in comment #16. The other is to specifically code for presentation forms (U+FB50-FDFF, FE70-FEFC). This isn't ideal, but it works. The suggested fix in comment #16 does not produce the correct rendering. Observe the difference in rendering the last character in http://jsfiddle.net/noonon/esz4S/2/ between Chrome and Firefox. For your other suggestion, I think shifting the responsibility of producing the correct glyph to user scripts will add an unnecessary complication. IMO, this should be transparent to the web developer. Any web developer who currently wants to style individual complex characters in Webkit is stuck. I was hoping giving the bug a higher priority would make it more visible and more likely to be picked up and fixed.
Behdad Esfahbod
Comment 34 2014-02-10 11:49:21 PST
(In reply to comment #33) > (In reply to comment #32) > > Raising the priority on the bug won't make it get fixed faster if there is no body willing to take on the work, which is not going to be trivial. The fundamental problem is that the character to glyph shaping process in WK doesn't make use of any context that crosses an element boundary. Fixing this will most likely introduce a performance regression in the slow text path, which is already slow. > > > > There are at least two temporary work arounds for this that authors may use. One is document in comment #16. The other is to specifically code for presentation forms (U+FB50-FDFF, FE70-FEFC). This isn't ideal, but it works. > > The suggested fix in comment #16 does not produce the correct rendering. Observe the difference in rendering the last character in http://jsfiddle.net/noonon/esz4S/2/ between Chrome and Firefox. That was fixed very recently: https://code.google.com/p/chromium/issues/detail?id=311372 > For your other suggestion, I think shifting the responsibility of producing the correct glyph to user scripts will add an unnecessary complication. IMO, this should be transparent to the web developer. True. We understand that. > Any web developer who currently wants to style individual complex characters in Webkit is stuck. I was hoping giving the bug a higher priority would make it more visible and more likely to be picked up and fixed.
Bashar
Comment 35 2014-02-10 12:18:43 PST
(In reply to comment #34) > > The suggested fix in comment #16 does not produce the correct rendering. Observe the difference in rendering the last character in http://jsfiddle.net/noonon/esz4S/2/ between Chrome and Firefox. > > That was fixed very recently: Thank you! I'm looking forward to trying it. I hope you guys can still make a comprehensive fix for this bug so that we wouldn't even need to use zero-width joiners to display individually styled characters.
Myles C. Maxfield
Comment 36 2014-07-31 10:06:59 PDT
*** Bug 135416 has been marked as a duplicate of this bug. ***
Ahmad
Comment 37 2015-02-17 09:49:31 PST
How to solve this problem by javasript in rich text editor to prevent arabic characters from sperated when put the cursor on any word to edit this bug is only in webkit browsers, the solution is to add &#x200d; before and after span element ex: م&#x200d;<span></span>&#x200d;ن, please any one have the function javascript to solve it, please advice me.
Myles C. Maxfield
Comment 38 2015-07-13 13:51:01 PDT
*** Bug 146907 has been marked as a duplicate of this bug. ***
Kilise
Comment 39 2015-11-11 06:59:01 PST
Status on this?
Myles C. Maxfield
Comment 40 2015-11-11 09:42:34 PST
(In reply to comment #39) > Status on this? No update.
Myles C. Maxfield
Comment 41 2017-06-02 11:39:01 PDT
*** Bug 172855 has been marked as a duplicate of this bug. ***
krinklemail
Comment 42 2018-04-05 09:25:05 PDT
This issue affects various Wikipedia content services. Downstream: https://phabricator.wikimedia.org/T188127
Ahmad Adel Eldardiry
Comment 44 2019-04-09 04:23:34 PDT
You can take a look at Madinah Arabic various lessons to see the difference between Chrome and FireFox Arabic rendering: https://www.madinaharabic.com/Arabic_Language_Course/Lessons/ FireFox does display Arabic correctly. An example: https://www.madinaharabic.com/Arabic_Language_Course/Lessons/L057_006.html Look at the difference between the two in the first table (Tashkeel like /đammah/) and the last two ones (colored middle letters).
mustafa.0x
Comment 46 2020-03-26 12:45:41 PDT
This issue has been resolved in Chrome (crbug.com/837574) with the new layout engine (LayoutNG). Safari is now an outlier in this behavior (i.e., Firefox, Chrome, and Edge support shaping characters across style changes).
Ahmad Adel Eldardiry
Comment 47 2020-03-27 13:38:13 PDT
I can see now that the issue of multiple colors in the same word has been resolved. Thank you! I can still see however the issue of some letters being shifted up: https://www.madinaharabic.com/arabic-language-course/lessons/L025_001.html Look at الْمُعْرَبُ وَالْمَبْنِيُّ in chrome and firefox. Firefox is still displaying it the correct way (font?). I'm using Mac Book Pro Catalina, latest.
Shokeen
Comment 48 2020-11-30 22:16:27 PST
Comment on attachment 5149 [details] Arabic Style Shaping test. ><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" > ><html> ><head> > <meta http-equiv="Content-Type" content="text/html; charset=utf-8" /> ><title>Arabic Shaping Test</title> ></head> ><body> ><p style="font-size:48px"><FONT color=#ffa500>ا</FONT><FONT color=#ff1493>Ù</FONT><FONT color=#800080>ع</FONT><FONT color=#0000ff>ر</FONT><FONT color=#800080>ب</FONT><FONT color=#008000>Ù</FONT><FONT color=#a52a2a>Ø©</FONT> and اÙعربÙØ© should look the same (other than the colors)</p> ></body> ></html>
Myles C. Maxfield
Comment 49 2021-03-12 18:21:06 PST
*** Bug 222734 has been marked as a duplicate of this bug. ***
Myles C. Maxfield
Comment 50 2021-08-11 00:29:05 PDT
*** Bug 215643 has been marked as a duplicate of this bug. ***
Myles C. Maxfield
Comment 54 2021-10-25 18:27:16 PDT
*** Bug 231668 has been marked as a duplicate of this bug. ***
r12a
Comment 55 2022-07-01 06:33:08 PDT
Open Bug 1777686 Opened 19 minutes ago Inline elements break cursive shaping Product: Core ▾ Component: Layout: Text and Fonts ▾ Version: Firefox 102 Type: defect Priority: Not set Severity: -- Status: UNCONFIRMED Votes: 0 Richard Ishida Reporter Description • 19 minutes ago User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Firefox/102.0 Steps to reproduce: I'd like to take this bug report back a step, and then expand it to encompass a wider set of issues – all related to the original title of the bug report. The backstep has to do with WebKit splitting cursive text (in Arabic, Adlam, N'Ko, Mongolian, and other cursive scripts) any time you simply insert inline markup (such as a span) around some of the characters within a word (with no styling changes). That, of course, makes it fundamentally difficult to apply style changes, too. The expansion has to do with the fact that since this bug was first raised, the CSS WG has looked at what kinds of styling should and shouldn't break cursiveness, extending the scope well beyond simple colour assignments. See https://www.w3.org/TR/css-text-3/#boundary-shaping. With the above in mind, i'm going to paste here the text of one of the language enablement gap reports produced by the W3C, which points to various tests and results. It also points to the relevant CSS discussion. --- This issue is applicable to text in all cursive scripts. When elements surround part of a cursive run of text, and apply styling, the results often break the cursive joins. (See the results of trying to colour individual letters in the illustration below – as expected above, unsuccessful below.) https://user-images.githubusercontent.com/4839211/105509398-5cca3300-5cc5-11eb-93e3-9398a9959a74.png https://user-images.githubusercontent.com/4839211/73611430-b24f9080-45d9-11ea-8b96-8f75648c725e.png SPECS: After some discussion (https://github.com/w3c/csswg-drafts/issues/698), the CSS spec requires the following (see https://www.w3.org/TR/css-text-3/#boundary-shaping): 1. Markup alone around part of a joined up sequence must not disturb the joining behaviour. 2. Styling that doesn't affect the characters, such as text-decoration, must not break the joins. 3. Styling that does affect the shape of the characters should not break the joins, however the result is not well defined for complex glyph arrangements such as ligatures where the markup occurs between characters that make up the ligature. 4. Non-zero margins, padding, and borders, will break the join, as will isolation boundaries. TESTS & RESULTS: Interactive test, A span with a colour change for one letter in an Arabic word doesn't break the joining behaviour https://github.com/w3c/character_phrase_tests/issues/43 - Gecko: ✅ Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Firefox/102.0 - Blink: ✅ Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36 - Webkit: ❌ Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15 I18n test suite, Cursive joining https://w3c.github.io/i18n-tests/results/css-text-shaping - Gecko: ✅❌ Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Firefox/102.0 - Blink: ✅ ❌ Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36 - Webkit: ❌ Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15 Webkit breaks cursive joining as soon as markup appears around a character, and so obviously fails for any type of styling application, too. Gecko and Blink keep joins for styling that doesn't affect the shape of the characters (eg. text-decoration), and keeps it for colour changes, however Firefox fails for changes in font-weight, font-style, and font-size, as well as for markup such as em and b tags. (Gecko and Blink also only pass some of the tests for non-zero margin/padding/border and bdi isolation. Which expect the cursive joins to be broken.) Expected results: Please fix WebKit so that shaping is not broken by inline markup, and then ensure that appropriate style changes don't break the shaping either (which includes, but goes beyond colour changes). At a minimum, please produce the same results as Gecko and Blink engines.
r12a
Comment 56 2022-07-01 06:35:30 PDT
Apologies, some initial text in the previous comment was inadvertently copied from elsewhere. The actual comment should begin with "Steps to reproduce:". (If i missed how to delete a comment or edit it, please let me know.)
nyro
Comment 57 2023-01-16 07:13:19 PST
Any update about this issue regarding ligation on HTML between spans? There is also the same issue on SVG text with tspan and/or textPath. It has been reported here: https://bugs.webkit.org/show_bug.cgi?id=78711
nyro
Comment 58 2023-03-03 00:20:50 PST
I'm adding here a codePen to see the problem simply: https://codepen.io/nyroDev/full/NWLpEXL You can choose the font to see when the problems occurs. On Safari TP 146, I see problem with: - Caveat - Climate crisis - Lobster - Shantell Sans (not sure if it's a ligature problem here) - Roboto - Brush Script MS.
Alexey Proskuryakov
Comment 59 2023-06-20 10:51:08 PDT
*** Bug 258300 has been marked as a duplicate of this bug. ***
Myles C. Maxfield
Comment 60 2023-06-20 12:05:33 PDT
I have some ideas for how to fix this at some point in the future. It’s a fairly significant architectural change, but IFC makes it easier. We should do: - bidi across the whole paragraph (which we already do) - shaping across the whole paragraph - line breaking across the whole paragraph Because bidi is already done this way in IFC we can use that as a model for the other two.
Mustafa
Comment 61 2023-07-30 10:28:27 PDT
@Myles Any plan or timeline where this would be worked on? I have to tell you that the fix for this issue is really essential for our apps to work correctly. I am sure not only me who is affected by this but many developers as well. Please, prioritize it and initiate a plan to get it fix soon. Appreciate your understanding.
Note You need to log in before you can comment on or make changes to this bug.