Bug 6148 - WebKit doesn't shape characters (like Arabic) across style changes
Summary: WebKit doesn't shape characters (like Arabic) across style changes
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Layout and Rendering (show other bugs)
Version: 420+
Hardware: All All
: P2 Normal
Assignee: Myles C. Maxfield
URL: http://blogs.msdn.com/michkap/archive...
Keywords: BrowserCompat, InRadar
: 17116 47213 63038 77790 91975 104530 135416 146907 222734 258300 (view as bug list)
Depends on:
Blocks: 229361 47213
  Show dependency treegraph
 
Reported: 2005-12-19 04:47 PST by Rosyna
Modified: 2023-08-07 11:44 PDT (History)
48 users (show)

See Also:


Attachments
Arabic Style Shaping test. (515 bytes, text/html)
2005-12-19 04:47 PST, Rosyna
no flags Details
test from bug 17116 (1.41 KB, text/html)
2008-01-31 23:21 PST, Alexey Proskuryakov
no flags Details
Testcase from bug 91975 (331 bytes, text/html)
2012-07-23 02:50 PDT, Jeremy Moskovich
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Rosyna 2005-12-19 04:47:00 PST
See the attached html document. The runs should look the same, but with different colorings as they do in 
IE for Windows.
Comment 1 Rosyna 2005-12-19 04:47:44 PST
Created attachment 5149 [details]
Arabic Style Shaping test.
Comment 2 Alexey Proskuryakov 2008-01-31 13:11:48 PST
*** Bug 17116 has been marked as a duplicate of this bug. ***
Comment 3 mitz 2008-01-31 17:20:30 PST
<rdar://problem/5718885>
Comment 4 Alexey Proskuryakov 2008-01-31 23:21:37 PST
Created attachment 18841 [details]
test from bug 17116

The original tests pass in Firefox 3 beta, but some of these do not.
Comment 5 Jeremy Moskovich 2009-01-07 20:13:11 PST
Filed in Chrome as http://code.google.com/p/chromium/issues/detail?id=6122
Comment 6 Eric Seidel (no email) 2011-03-25 03:19:37 PDT
*** Bug 47213 has been marked as a duplicate of this bug. ***
Comment 7 Glenn Adams 2011-05-30 22:45:12 PDT
If nobody is actively working on this, I'm willing to take a pass at a patch.
Comment 8 Jeremy Moskovich 2011-05-30 23:46:02 PDT
Go for it!
Comment 9 Glenn Adams 2011-06-03 20:57:45 PDT
Initial investigation shows that BidiResolver::createBidiRunsForLine is breaking runs at a display:inline element boundary, e.g., span, even if the eventual embedding levels on both side of the boundary are the same. This causes RenderBlock::createLineBoxesFromBidiRuns to create distinct inline boxes across this boundary, preventing eventual complex text shaping from applying shaping context across the boundary.
Comment 10 Eric Seidel (no email) 2011-06-06 17:35:13 PDT
Yes.  LineBoxes have a pointer back to their renderer and do not span renderers.
Comment 11 Alexey Proskuryakov 2011-06-21 10:44:12 PDT
*** Bug 63038 has been marked as a duplicate of this bug. ***
Comment 12 Munzir Taha (منذر طه) 2011-11-26 12:43:52 PST
Hi, I just encountered this bug while I am loading a page using
  QWebView *view = new QWebView();
  view->load(QUrl("test.html"));

while the test.html contains

<!DOCTYPE HTML>
<html>
<head>
<meta charset=utf8>
<style type="text/css">
p:first-letter {
  color: red;
}
</style>
</head>
<body>
The two arabic letters should apear like عر but they really show as
<p>عر</p>
and
<div><span>ع</span>ر</div>

</body>
</html>

Any update on this?
Comment 13 Glenn Adams 2011-11-26 13:55:20 PST
(In reply to comment #12)
> Any update on this?

Not yet.
Comment 14 Dermot Rourke 2012-04-06 08:12:19 PDT
I was wondering if there's been any progress made on this issue?  I've come accross a number of Arabic Language Learning websites that rely on the ability to highlight specific letters within a word for teaching purposes.  For example:

http://transliteration.org/quran/WebSite_CD/HighlightSample/Fram3.htm

http://arabiccomplete.com/modules_colloquial_msa/possessive_suffix_1.htm

http://www.dalilusa.com/arabic_course/exercise02.asp

Thanks.
Comment 15 Glenn Adams 2012-04-06 09:20:06 PDT
(In reply to comment #14)
> I was wondering if there's been any progress made on this issue?

I'm trying to get this back on my priority list, and there is a good chance I will do so in the next four weeks.
Comment 16 Dermot Rourke 2012-05-09 08:16:44 PDT
Just checking in to see if any progress has been made?

A colleague of mine found a temporary work-around that may be useful to some developers in some scenarios - using the zero-width-joiner (&zwj;/&#8205;) will force the letters to join (or, at least, appear joined).  Of course, it's not ideal as you'll need to test for the browser and insert them on page load (or something along those lines).  Also, it does not work for every situation - in the example in comment #12 above, the css selector fails.  The code would look something like:

<!DOCTYPE HTML>
<html>
<head>
<meta charset=utf8>
<style type="text/css">
p:first-letter { color: red; }
p, div { font-family: times new roman; font-size: x-large; }
</style>
</head>
<body>
The two arabic letters should appear like عر but they really show as
<p>&zwj;عر</p>
and
<div><span>ع&zwj;</span>ر</div>
</body>
</html>
Comment 17 Jeremy Moskovich 2012-07-23 02:44:57 PDT
*** Bug 91975 has been marked as a duplicate of this bug. ***
Comment 18 Jeremy Moskovich 2012-07-23 02:50:40 PDT
Created attachment 153763 [details]
Testcase from bug 91975

Amir Aharoni's simple testcase from bug 91975
Comment 19 Jeremy Moskovich 2012-07-23 02:52:52 PDT
Also tracked as http://crbug.com/138434 (http://crbug.com/6122 tracks the color issue)
Comment 20 Glenn Adams 2012-09-13 23:09:07 PDT
this is (finally) on the top of my queue, so assigning to myself
Comment 21 Eric Seidel (no email) 2012-10-27 00:27:53 PDT
*** Bug 77790 has been marked as a duplicate of this bug. ***
Comment 22 Eric Seidel (no email) 2012-10-27 00:29:06 PDT
If you're still working on this Glenn, we should chat.
Comment 23 Nasser Al-Wohaibi 2012-12-01 02:50:34 PST
@Dermot Rourke,

to fix this entirely, use TWO zero-width-joiners 
e.g.
<p>عرب&#x200d;<span style="color: Red;">&#x200d;ي</span></p>

e.g.
<!DOCTYPE HTML>
<html>
<head>
<meta charset=utf8>
<style type="text/css">
body{font-size:40px;}

.test{
color:red;
font-weight:bolder;
}
</style>
</head>
<body>
The two arabic letters should apear like عربي but they really show as the following in webkit(chrome,safari)
    <p>عرب<span>ي</span></p>
solution:

<p>عرب&#x200d;<span style="color: Red;">&#x200d;ي</span></p>
</body>
</html>​

demo: http://jsfiddle.net/noonon/esz4S/2/
Comment 24 Hamzeh 2012-12-07 10:05:32 PST
Hi, any update on this issue ?

My case running Version 23.0.1271.95, i tried to work around the issue with the zero-width-joiner or double zero-width-joiner, still the Arabic letter shapes appears broken.

Coloring part of Arabic words is a common practice used in Arabic learning sites, currently we recommend our users to switch to other browsers as (Firefox, IE ) in order to render pages correctly.
Comment 25 Nasser Al-Wohaibi 2012-12-07 22:34:28 PST
@Hamzeh 
if you can paste some code samples?
I might be of some help
Comment 26 Glenn Adams 2012-12-07 23:17:23 PST
it is not necessary to provide any more examples; the problem is well understood; however, the solution requires working around certain design limitations that aren't straightforward
Comment 27 Hamzeh 2012-12-08 03:16:57 PST
(In reply to comment #25)
> @Hamzeh 
> if you can paste some code samples?
> I might be of some help

Nasser,

I've followed the demo link provided by you and the same Arabic shape problem existed with my chrome version. trying to color part of the Arabic word fails on chrome regardless of using zero-width-joiner or not.

while Firefox and IE rendering engines are working just fine with or without zero-width-joiner. this defect is vital for learning sites, since coloring part of the word is widely used to identify prefixes, suffixes, and certain language characteristics. 

I hope that someone from webkit to give it priority, it's a very important for languages such as Arabic, Persian, and Urdo.
Comment 28 Glenn Adams 2012-12-10 09:53:00 PST
*** Bug 104530 has been marked as a duplicate of this bug. ***
Comment 29 Bashar 2014-01-31 18:28:18 PST
Are there any updates on this bug?
Comment 30 Myles C. Maxfield 2014-02-03 14:27:24 PST
Not as of yet.
Comment 31 Bashar 2014-02-10 00:51:31 PST
Is there a way we can give this bug a higher priority(possibly Major)? The ability to style individual characters is very important for educational and word-game apps but it's currently broken for all sites that use complex script.
Comment 32 Glenn Adams 2014-02-10 07:06:08 PST
(In reply to comment #31)
> Is there a way we can give this bug a higher priority(possibly Major)? The ability to style individual characters is very important for educational and word-game apps but it's currently broken for all sites that use complex script.

Raising the priority on the bug won't make it get fixed faster if there is no body willing to take on the work, which is not going to be trivial. The fundamental problem is that the character to glyph shaping process in WK doesn't make use of any context that crosses an element boundary. Fixing this will most likely introduce a performance regression in the slow text path, which is already slow.

There are at least two temporary work arounds for this that authors may use. One is document in comment #16. The other is to specifically code for presentation forms (U+FB50-FDFF, FE70-FEFC). This isn't ideal, but it works.
Comment 33 Bashar 2014-02-10 11:46:37 PST
(In reply to comment #32)
> Raising the priority on the bug won't make it get fixed faster if there is no body willing to take on the work, which is not going to be trivial. The fundamental problem is that the character to glyph shaping process in WK doesn't make use of any context that crosses an element boundary. Fixing this will most likely introduce a performance regression in the slow text path, which is already slow.
> 
> There are at least two temporary work arounds for this that authors may use. One is document in comment #16. The other is to specifically code for presentation forms (U+FB50-FDFF, FE70-FEFC). This isn't ideal, but it works.

The suggested fix in comment #16 does not produce the correct rendering. Observe the difference in rendering the last character in http://jsfiddle.net/noonon/esz4S/2/ between Chrome and Firefox. 

For your other suggestion, I think shifting the responsibility of producing the correct glyph to user scripts will add an unnecessary complication. IMO, this should be transparent to the web developer.

Any web developer who currently wants to style individual complex characters in Webkit is stuck. I was hoping giving the bug a higher priority would make it more visible and more likely to be picked up and fixed.
Comment 34 Behdad Esfahbod 2014-02-10 11:49:21 PST
(In reply to comment #33)
> (In reply to comment #32)
> > Raising the priority on the bug won't make it get fixed faster if there is no body willing to take on the work, which is not going to be trivial. The fundamental problem is that the character to glyph shaping process in WK doesn't make use of any context that crosses an element boundary. Fixing this will most likely introduce a performance regression in the slow text path, which is already slow.
> > 
> > There are at least two temporary work arounds for this that authors may use. One is document in comment #16. The other is to specifically code for presentation forms (U+FB50-FDFF, FE70-FEFC). This isn't ideal, but it works.
> 
> The suggested fix in comment #16 does not produce the correct rendering. Observe the difference in rendering the last character in http://jsfiddle.net/noonon/esz4S/2/ between Chrome and Firefox. 

That was fixed very recently:

  https://code.google.com/p/chromium/issues/detail?id=311372


> For your other suggestion, I think shifting the responsibility of producing the correct glyph to user scripts will add an unnecessary complication. IMO, this should be transparent to the web developer.

True.  We understand that.

> Any web developer who currently wants to style individual complex characters in Webkit is stuck. I was hoping giving the bug a higher priority would make it more visible and more likely to be picked up and fixed.
Comment 35 Bashar 2014-02-10 12:18:43 PST
(In reply to comment #34)
> > The suggested fix in comment #16 does not produce the correct rendering. Observe the difference in rendering the last character in http://jsfiddle.net/noonon/esz4S/2/ between Chrome and Firefox. 
> 
> That was fixed very recently:

Thank you! I'm looking forward to trying it. I hope you guys can still make a comprehensive fix for this bug so that we wouldn't even need to use zero-width joiners to display individually styled characters.
Comment 36 Myles C. Maxfield 2014-07-31 10:06:59 PDT
*** Bug 135416 has been marked as a duplicate of this bug. ***
Comment 37 Ahmad 2015-02-17 09:49:31 PST
How to solve this problem by javasript in rich text editor to prevent arabic characters from sperated when put the cursor on any word to edit this bug is only in webkit browsers, the solution is to add &#x200d; before and after span element ex: م&#x200d;<span></span>&#x200d;ن, please any one have the function javascript to solve it, please advice me.
Comment 38 Myles C. Maxfield 2015-07-13 13:51:01 PDT
*** Bug 146907 has been marked as a duplicate of this bug. ***
Comment 39 Kilise 2015-11-11 06:59:01 PST
Status on this?
Comment 40 Myles C. Maxfield 2015-11-11 09:42:34 PST
(In reply to comment #39)
> Status on this?

No update.
Comment 41 Myles C. Maxfield 2017-06-02 11:39:01 PDT
*** Bug 172855 has been marked as a duplicate of this bug. ***
Comment 42 krinklemail 2018-04-05 09:25:05 PDT
This issue affects various Wikipedia content services.
Downstream: https://phabricator.wikimedia.org/T188127
Comment 44 Ahmad Adel Eldardiry 2019-04-09 04:23:34 PDT
You can take a look at Madinah Arabic various lessons to see the difference between Chrome and FireFox Arabic rendering:

https://www.madinaharabic.com/Arabic_Language_Course/Lessons/

FireFox does display Arabic correctly.

An example:

https://www.madinaharabic.com/Arabic_Language_Course/Lessons/L057_006.html

Look at the difference between the two in the first table (Tashkeel like /đammah/) and the last two ones (colored middle letters).
Comment 46 mustafa.0x 2020-03-26 12:45:41 PDT
This issue has been resolved in Chrome (crbug.com/837574) with the new layout engine (LayoutNG). Safari is now an outlier in this behavior (i.e., Firefox, Chrome, and Edge support shaping characters across style changes).
Comment 47 Ahmad Adel Eldardiry 2020-03-27 13:38:13 PDT
I can see now that the issue of multiple colors in the same word has been resolved. Thank you!

I can still see however the issue of some letters being shifted up:

https://www.madinaharabic.com/arabic-language-course/lessons/L025_001.html

Look at الْمُعْرَبُ وَالْمَبْنِيُّ in chrome and firefox. Firefox is still displaying it the correct way (font?).

I'm using Mac Book Pro Catalina, latest.
Comment 48 Shokeen 2020-11-30 22:16:27 PST
Comment on attachment 5149 [details]
Arabic Style Shaping test.

><!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" >
><html>
><head>
> <meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
><title>Arabic Shaping Test</title>
></head>
><body>
><p style="font-size:48px"><FONT color=#ffa500>ا</FONT><FONT color=#ff1493>Ù</FONT><FONT color=#800080>ع</FONT><FONT color=#0000ff>ر</FONT><FONT color=#800080>ب</FONT><FONT color=#008000>Ù</FONT><FONT color=#a52a2a>Ø©</FONT> and اÙعربÙØ© should look the same (other than the colors)</p>
></body>
></html>
Comment 49 Myles C. Maxfield 2021-03-12 18:21:06 PST
*** Bug 222734 has been marked as a duplicate of this bug. ***
Comment 50 Myles C. Maxfield 2021-08-11 00:29:05 PDT
*** Bug 215643 has been marked as a duplicate of this bug. ***
Comment 54 Myles C. Maxfield 2021-10-25 18:27:16 PDT
*** Bug 231668 has been marked as a duplicate of this bug. ***
Comment 55 r12a 2022-07-01 06:33:08 PDT
Open Bug 1777686 Opened 19 minutes ago
Inline elements break cursive shaping
Product:
Core ▾
Component:
Layout: Text and Fonts ▾
Version:
Firefox 102
Type:
defect
Priority:
Not set
Severity:
-- 
Status:
UNCONFIRMED
Votes:
0
	Richard Ishida
Reporter 	
Description • 19 minutes ago

User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Firefox/102.0

Steps to reproduce:
I'd like to take this bug report back a step, and then expand it to encompass a wider set of issues – all related to the original title of the bug report.

The backstep has to do with WebKit splitting cursive text (in Arabic, Adlam, N'Ko, Mongolian, and other cursive scripts) any time you simply insert inline markup (such as a span) around some of the characters within a word (with no styling changes).  That, of course, makes it fundamentally difficult to apply style changes, too.

The expansion has to do with the fact that since this bug was first raised, the CSS WG has looked at what kinds of styling should and shouldn't break cursiveness, extending the scope well beyond simple colour assignments. See https://www.w3.org/TR/css-text-3/#boundary-shaping.

With the above in mind, i'm going to paste here the text of one of the language enablement gap reports produced by the W3C, which points to various tests and results. It also points to the relevant CSS discussion.

---



This issue is applicable to text in all cursive scripts.

When elements surround part of a cursive run of text, and apply styling, the results often break the cursive joins. (See the results of trying to colour individual letters in the illustration below – as expected above, unsuccessful below.)

https://user-images.githubusercontent.com/4839211/105509398-5cca3300-5cc5-11eb-93e3-9398a9959a74.png

https://user-images.githubusercontent.com/4839211/73611430-b24f9080-45d9-11ea-8b96-8f75648c725e.png

SPECS:
After some discussion (https://github.com/w3c/csswg-drafts/issues/698), the CSS spec requires the following (see https://www.w3.org/TR/css-text-3/#boundary-shaping):

1.   Markup alone around part of a joined up sequence must not disturb the joining behaviour.
2.    Styling that doesn't affect the characters, such as text-decoration, must not break the joins.
3.    Styling that does affect the shape of the characters should not break the joins, however the result is not well defined for complex glyph arrangements such as ligatures where the markup occurs between characters that make up the ligature.
4.    Non-zero margins, padding, and borders, will break the join, as will isolation boundaries.

TESTS & RESULTS:
Interactive test, A span with a colour change for one letter in an Arabic word doesn't break the joining behaviour
https://github.com/w3c/character_phrase_tests/issues/43

-    Gecko: ✅ Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Firefox/102.0
-    Blink: ✅ Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36
-    Webkit: ❌ Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15

I18n test suite, Cursive joining
https://w3c.github.io/i18n-tests/results/css-text-shaping

-    Gecko: ✅❌ Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Firefox/102.0
-    Blink: ✅ ❌ Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36
-    Webkit: ❌ Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.5 Safari/605.1.15

Webkit breaks cursive joining as soon as markup appears around a character, and so obviously fails for any type of styling application, too.

Gecko and Blink keep joins for styling that doesn't affect the shape of the characters (eg. text-decoration), and keeps it for colour changes, however Firefox fails for changes in font-weight, font-style, and font-size, as well as for markup such as em and b tags.

(Gecko and Blink also only pass some of the tests for non-zero margin/padding/border and bdi isolation. Which expect the cursive joins to be broken.)

Expected results:

Please fix WebKit so that shaping is not broken by inline markup, and then ensure that appropriate style changes don't break the shaping either (which includes, but goes beyond colour changes).  At a minimum, please produce the same results as Gecko and Blink engines.
Comment 56 r12a 2022-07-01 06:35:30 PDT
Apologies, some initial text in the previous comment was inadvertently copied from elsewhere.  The actual comment should begin with "Steps to reproduce:".

(If i missed how to delete a comment or edit it, please let me know.)
Comment 57 nyro 2023-01-16 07:13:19 PST
Any update about this issue regarding ligation on HTML between spans?  

There is also the same issue on SVG text with tspan and/or textPath.  
It has been reported here: https://bugs.webkit.org/show_bug.cgi?id=78711
Comment 58 nyro 2023-03-03 00:20:50 PST
I'm adding here a codePen to see the problem simply:  
https://codepen.io/nyroDev/full/NWLpEXL

You can choose the font to see when the problems occurs.  
On Safari TP 146, I see problem with:  
- Caveat
- Climate crisis
- Lobster
- Shantell Sans (not sure if it's a ligature problem here)
- Roboto
- Brush Script MS.
Comment 59 Alexey Proskuryakov 2023-06-20 10:51:08 PDT
*** Bug 258300 has been marked as a duplicate of this bug. ***
Comment 60 Myles C. Maxfield 2023-06-20 12:05:33 PDT
I have some ideas for how to fix this at some point in the future. It’s a fairly significant architectural change, but IFC makes it easier.

We should do:
- bidi across the whole paragraph (which we already do)
- shaping across the whole paragraph
- line breaking across the whole paragraph

Because bidi is already done this way in IFC we can use that as a model for the other two.
Comment 61 Mustafa 2023-07-30 10:28:27 PDT
@Myles

Any plan or timeline where this would be worked on? 

I have to tell you that the fix for this issue is really essential for our apps to work correctly. I am sure not only me who is affected by this but many developers as well. Please, prioritize it and initiate a plan to get it fix soon. 

Appreciate your understanding.