Bug 174418

Summary: [HarfBuzz] Decomposed Vietnamese characters are rendered incorrectly
Product: WebKit Reporter: Fujii Hironori <Hironori.Fujii>
Component: PlatformAssignee: Fujii Hironori <Hironori.Fujii>
Status: RESOLVED FIXED    
Severity: Normal CC: annulen, bashi, commit-queue, fred.wang, mcatanzaro, mmaxfield, webkit-bug-importer, yoshiaki.jitsukawa
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on: 175580    
Bug Blocks:    
Attachments:
Description Flags
screenshot
none
test case
none
Patch
none
vietnamese-nfd-actual.png
none
vietnamese-nfd-expected.png
none
Patch none

Description Fujii Hironori 2017-07-12 02:40:56 PDT
Created attachment 315220 [details]
screenshot

[HarfBuzz] Decomposed Vietnamese characters are rendered incorrectly

For example, "แปƒ" has two unicode representation "&#x1EC3;" and "&#x0065;&#x0302;&#x0309;".
https://en.wiktionary.org/wiki/%E1%BB%83
Those characters should be rendered identically.
Comment 1 Fujii Hironori 2017-07-12 02:41:45 PDT
Created attachment 315221 [details]
test case
Comment 2 Fujii Hironori 2017-07-12 02:52:37 PDT
HarfBuzzShaper should normalize the input text before shaping.

Actually, HarfBuzzShaper::setNormalizedBuffer does the task.
https://trac.webkit.org/browser/webkit/trunk/Source/WebCore/platform/graphics/harfbuzz/HarfBuzzShaper.cpp?rev=219385#L233

But, this function hasn't called from anywhere since Bug 108077.
Comment 3 Fujii Hironori 2017-07-12 18:25:36 PDT
Created attachment 315321 [details]
Patch
Comment 4 Fujii Hironori 2017-07-12 18:26:36 PDT
Created attachment 315322 [details]
vietnamese-nfd-actual.png
Comment 5 Fujii Hironori 2017-07-12 18:26:54 PDT
Created attachment 315323 [details]
vietnamese-nfd-expected.png
Comment 6 Fujii Hironori 2017-07-12 19:40:59 PDT
(In reply to Fujii Hironori from comment #2)
> But, this function hasn't called from anywhere since Bug 108077.

Not Bug 108077, but Bug 90951.
Comment 7 Fujii Hironori 2017-07-12 19:49:14 PDT
It'd be great help if you can take a glance at my patch and give me your thoughts.
Comment 8 Kenichi Ishibashi 2017-07-12 20:59:17 PDT
The patch looks good to me. Please ask WebKit reviewers for actual review :)
Comment 9 Fujii Hironori 2017-07-12 22:21:26 PDT
It's a great helpful. Thank you so much to have a time to review my patch.
Comment 10 Michael Catanzaro 2017-07-13 18:59:45 PDT
Oh wow, apparently it's font bug week or something!

I have not the faintest clue if your patch is right or not. I'm encouraged that it fixes the problem and includes a layout test. I'm also encouraged to see that Kenichi likes your patch. I see that he wrote all this code, so that's good I suppose!
Comment 11 Michael Catanzaro 2017-07-13 19:03:22 PDT
Comment on attachment 315321 [details]
Patch

I'm giving r+ even though I don't understand the code, because I'm pretty sure we don't have any Igalia reviewers who might understand it. Myles, if you see any reason not to commit this, please say so.

I have a suspicion that this commit might affect a lot of test results, so we will need to watch the bots carefully after landing it. So we will have to try landing this another day.
Comment 12 Myles C. Maxfield 2017-07-13 19:38:45 PDT
(In reply to Michael Catanzaro from comment #10)
> I have not the faintest clue if your patch is right or not. I'm encouraged
> that it fixes the problem and includes a layout test. I'm also encouraged to
> see that Kenichi likes your patch. I see that he wrote all this code, so
> that's good I suppose!

๐Ÿ˜
Comment 13 Michael Catanzaro 2017-07-13 21:01:11 PDT
Comment on attachment 315321 [details]
Patch

OK, Myles posted a happy emojii, and the tree is back open, so we should be good to cq+ this tomorrow when we're available for watching the bots.
Comment 14 WebKit Commit Bot 2017-07-14 08:54:34 PDT
Comment on attachment 315321 [details]
Patch

Clearing flags on attachment: 315321

Committed r219504: <http://trac.webkit.org/changeset/219504>
Comment 15 WebKit Commit Bot 2017-07-14 08:54:36 PDT
All reviewed patches have been landed.  Closing bug.
Comment 16 Michael Catanzaro 2017-07-14 10:13:03 PDT
Looks like there's no difference in the test results. OK then.
Comment 17 Fujii Hironori 2017-07-14 13:57:56 PDT
Thank you, Michael. I'll keep an eys on it during this weekend. I'll investigate new failures next Tuesday if any. (next Monday is Japan's national holiday)
Comment 18 Michael Catanzaro 2017-07-14 16:44:45 PDT
There were not any new failures!
Comment 19 Michael Catanzaro 2017-08-15 10:12:56 PDT
Oh actually there was a regression, I just missed it. :( It's imported/blink/fast/text/international/text-shaping-arabic-diffs.html:

Expected image: https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20(Tests)/r220734%20(2623)/imported/blink/fast/text/international/text-shaping-arabic-diffs.html

New image after this patch was committed: https://build.webkit.org/results/GTK%20Linux%2064-bit%20Release%20(Tests)/r220734%20(2623)/imported/blink/fast/text/international/text-shaping-arabic-actual.png

Fujii, any ideas?
Comment 20 WebKit Commit Bot 2017-08-15 10:14:11 PDT
Re-opened since this is blocked by bug 175580
Comment 21 Fujii Hironori 2017-08-15 19:57:07 PDT
Thank you for rolling back my change, Michael. I'm sorry for not noticing the regression.

normalizeCharacters was using FontCascade::treatAsZeroWidthSpaceInComplexScript while normalizeSpacesAndMirrorChars is using FontCascade::treatAsZeroWidthSpace.
Comment 22 Fujii Hironori 2017-08-15 21:35:16 PDT
Created attachment 318221 [details]
Patch
Comment 23 Michael Catanzaro 2017-08-15 22:23:11 PDT
Careful, or you'll become a text layout expert!

Is this patch ready for review?
Comment 24 Fujii Hironori 2017-08-16 02:35:26 PDT
Comment on attachment 318221 [details]
Patch

EWS got green. finished testing in my env. It's ready for review now.
Comment 25 WebKit Commit Bot 2017-08-16 09:36:39 PDT
Comment on attachment 318221 [details]
Patch

Clearing flags on attachment: 318221

Committed r220797: <http://trac.webkit.org/changeset/220797>
Comment 26 WebKit Commit Bot 2017-08-16 09:36:41 PDT
All reviewed patches have been landed.  Closing bug.
Comment 27 Radar WebKit Bug Importer 2017-08-16 09:37:16 PDT
<rdar://problem/33920068>