Bug 215643 – overwriteCodePoint() in createAndFillGlyphPage() is wrong

• Status: RESOLVED
• Resolution: FIXED
• Component: HTML Editing

[Hossameldin]

Created attachment 406834 [details]
not working on safari

UserAgent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.117 Safari/537.36


Steps to reproduce the problem:
Place HTML tags within an Arabic word.

What is the expected behavior?
The letters of the word should be joined.

What went wrong?
The letters of the word are separate; adding ZWJs is necessary to join them. Even adding ZWJs isn't a full resolution, as ligatures aren't used then.

Does it occur on multiple sites: Yes

Is it a problem with a plugin? No 

Did this work before? No 

Does this work in other browsers? Yes

OS Version: OS X 10.13.3

Here's a small discussion on this issue: https://twitter.com/KhaledGhetas/status/989311667682119681

Adding ZWJs per se isn't an an issue, but:
 - it's much easier to not add them
 - adding them doesn't render ligatures properly, like ألا
 - Firefox (and Edge, reportedly) don't need them.

Coloring parts of Arabic words is a often needed feature, like highlighting matches, adding tajweed coloring to the Qur'an (open muqri.com, click play, and click on the ت), and so on.

[Hossameldin]

Created attachment 406836 [details]
working on firefox

[Hossameldin]

Created attachment 406837 [details]
not working on safari

[Hossameldin]

Created attachment 406838 [details]
working on firefox

[Hossameldin]

Created attachment 406839 [details]
not working properly on safari even with joiner

not displaying as a ligature when using a joiner

[Radar WebKit Bug Importer]

<rdar://problem/67430461>

[Myles C. Maxfield]

Can you attach the HTML and CSS content which the screenshots represent?

[Myles C. Maxfield]

Could be https://bugs.webkit.org/show_bug.cgi?id=6148

[Said Abou-Hallawa]

Created attachment 407118 [details]
test case

[Said Abou-Hallawa]

The attached test case resembles the attached screenshots. This test case shows a bug in rendering the diacritic after the Alef (the second logical character in the third and the fourth paragraphs). Chrome and FireFox render the diacritic as expected.

[Myles C. Maxfield]

This is because of

overwriteCodePoint(zeroWidthJoiner, zeroWidthSpace);

in overrideControlCharacters().

[Myles C. Maxfield]

*** This bug has been marked as a duplicate of bug 187166 ***

[Myles C. Maxfield]

Reverse-duping

[Myles C. Maxfield]

*** Bug 187166 has been marked as a duplicate of this bug. ***

[Myles C. Maxfield]

Shaping inside fonts relies on the correct glyphIDs being used. However, we replace a bunch of glyphs with the zero width space glyph, which fonts won't expect.

[Myles C. Maxfield]

*** Bug 172205 has been marked as a duplicate of this bug. ***

[Myles C. Maxfield]

The joiner appears to be working now.

*** This bug has been marked as a duplicate of bug 6148 ***

[Myles C. Maxfield]

Reopening, because I'd like to fix the underlying problem, even if the specific symptoms no longer occur.

[Myles C. Maxfield]

Created attachment 435384 [details]
Test case

This is a test font which has a ligature to map ZWJ + B to the same glyph as p. This ligature works in Chrome and Firefox, but not in WebKit.

[Myles C. Maxfield]

Confirmed that deleting overrideControlCharacters() fixes this.

[Myles C. Maxfield]

Created attachment 435446 [details]
WIP

[Myles C. Maxfield]

Created attachment 435448 [details]
WIP

[Myles C. Maxfield]

Created attachment 435449 [details]
WIP

[Myles C. Maxfield]

Failing tests:

fast/borders/bidi-002.html
fast/borders/bidi-009a.html
fast/dom/52776.html
fast/text/format-control.html
fast/text/international/bdi-dir-default-to-auto.html
fast/text/international/bdi-neutral-wrapped.html
fast/text/international/bidi-neutral-run.html
fast/text/isolate-ignore.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-embed-001.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-embed-002.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-embed-003.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-embed-004.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-embed-005.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-embed-007.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-embed-008.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-embed-009.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-embed-011.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-001.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-002.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-003.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-004.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-005.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-008.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-009.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-011.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-override-001.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-override-002.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-override-003.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-override-004.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-override-005.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-override-008.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-override-009.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-override-011.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-isolate-override-012.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-normal-001.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-normal-002.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-normal-003.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-normal-004.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-normal-005.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-normal-007.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-normal-008.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-normal-009.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-normal-011.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-override-001.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-override-002.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-override-003.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-override-004.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-override-005.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-override-007.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-override-011.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-override-012.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-plaintext-001.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-plaintext-002.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-plaintext-003.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-plaintext-004.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-plaintext-005.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-plaintext-008.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-plaintext-009.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-unset-001.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-unset-002.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-unset-003.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-unset-004.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-unset-005.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-unset-007.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-unset-008.html
imported/blink/imported/csswg-test/css-writing-modes-3/bidi-unset-009.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-embed-001.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-embed-002.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-embed-003.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-override-001.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-override-002.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-override-003.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-override-004.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-override-isolate-001.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-override-isolate-002.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-override-isolate-003.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-override-isolate-004.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-plaintext-001.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-plaintext-002.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-plaintext-003.html
imported/blink/imported/csswg-test/css-writing-modes-3/block-plaintext-004.html
imported/w3c/web-platform-tests/css/css-text/white-space/white-space-collapse-002.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-embed-001.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-embed-002.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-embed-003.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-embed-004.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-embed-005.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-embed-007.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-embed-008.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-embed-009.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-embed-011.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-001.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-002.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-003.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-004.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-005.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-008.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-009.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-011.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-override-001.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-override-002.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-override-003.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-override-004.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-override-005.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-override-008.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-override-009.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-override-011.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-isolate-override-012.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-normal-001.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-normal-002.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-normal-003.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-normal-004.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-normal-005.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-normal-007.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-normal-008.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-normal-009.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-normal-011.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-override-001.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-override-002.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-override-003.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-override-004.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-override-005.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-override-007.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-override-011.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-override-012.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-plaintext-001.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-plaintext-002.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-plaintext-003.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-plaintext-004.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-plaintext-005.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-plaintext-008.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-plaintext-009.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-unset-001.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-unset-002.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-unset-003.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-unset-004.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-unset-005.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-unset-007.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-unset-008.html
imported/w3c/web-platform-tests/css/css-writing-modes/bidi-unset-009.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-embed-001.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-embed-002.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-embed-003.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-override-001.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-override-002.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-override-003.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-override-004.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-override-isolate-001.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-override-isolate-002.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-override-isolate-003.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-override-isolate-004.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-plaintext-001.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-plaintext-002.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-plaintext-003.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-plaintext-004.html
imported/w3c/web-platform-tests/css/css-writing-modes/block-plaintext-006.html
imported/w3c/web-platform-tests/html/dom/elements/global-attributes/dir_auto-isolate.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-001a.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-001b.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-001c.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-002a.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-002b.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-002c.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-003a.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-003b.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-003c.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-004a.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-004b.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-004c.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-005a.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-005b.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-005c.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-006a.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-006b.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-006c.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-007a.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-007b.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-007c.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-008a.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-008b.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-008c.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-009a.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-009b.html
imported/w3c/web-platform-tests/html/dom/elements/requirements-relating-to-bidirectional-algorithm-formatting-characters/dir-isolation-009c.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-auto-dir-default.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-missing-pdf.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-nested.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-number.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-separate.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-to-another-bdi-1.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-to-another-bdi-2.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-to-letter-following-1.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-to-letter-following-2.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-to-letter-preceding-1.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-to-letter-preceding-2.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-to-number-following-1.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-to-number-following-2.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-to-surrounding-run.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-neutral-wrapped.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-bdi-element/bdi-paragraph-level-container.html
imported/w3c/web-platform-tests/html/semantics/text-level-semantics/the-br-element/br-bidi-in-inline-ancestors.html

[Myles C. Maxfield]

Created attachment 435461 [details]
Patch

[Myles C. Maxfield]

Created attachment 435462 [details]
Patch

[Myles C. Maxfield]

Created attachment 435464 [details]
Patch

[Myles C. Maxfield]

Comment on attachment 435464 [details]
Patch

Marking as r?. EWS will show some red, but that's just because this patch depends on https://bugs.webkit.org/show_bug.cgi?id=229064 which hasn't been landed yet.

[zalan]

Comment on attachment 435464 [details]
Patch

:thumbsup:

[Myles C. Maxfield]

Created attachment 435952 [details]
Patch for committing

[Myles C. Maxfield]

Created attachment 435953 [details]
Patch for committing

[Myles C. Maxfield]

Windows failures are because of https://bugs.webkit.org/show_bug.cgi?id=215318

[Myles C. Maxfield]

Created attachment 436080 [details]
Patch for committing

[Myles C. Maxfield]

Created attachment 436081 [details]
Patch for committing

[Myles C. Maxfield]

Created attachment 436082 [details]
Patch for committing

[EWS]

Tools/Scripts/svn-apply failed to apply attachment 436082 [details] to trunk.
Please resolve the conflicts and upload a new patch.

[Myles C. Maxfield]

Committed r281389 (240804@main): <https://commits.webkit.org/240804@main>