213253 2020-06-16 09:36:57 -0700 Fix the case about NFC normalization in wpt/FileAPI/unicode.html 2020-10-06 05:07:37 -0700 1 1 1 Unclassified WebKit DOM WebKit Nightly Build Unspecified Unspecified RESOLVED DUPLICATE 217327 P2 Normal --- 213254 1 tetsuharu.ohzeki webkit-unassigned ap ysuzuki oldest_to_newest 1663121 0 tetsuharu.ohzeki 2020-06-16 09:36:57 -0700 I'll attach a patch 1663128 1 402016 tetsuharu.ohzeki 2020-06-16 09:50:19 -0700 Created attachment 402016 Patch 1663736 2 402016 ap 2020-06-17 18:12:03 -0700 Comment on attachment 402016 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=402016&action=review > Source/WebCore/fileapi/BlobBuilder.cpp:74 > - auto bytes = UTF8Encoding().encodeWithNormalization(text, UnencodableHandling::Entities); > + auto bytes = UTF8Encoding().encode(text, UnencodableHandling::Entities); While this is correct in abstract, I don't think that we can do this before the normalization is performed in all code paths where decomposed text from macOS can enter WebKit. And once we do that, encode can stop normalizing. 1663746 3 tetsuharu.ohzeki 2020-06-17 18:36:24 -0700 (In reply to Alexey Proskuryakov from comment #2) > Comment on attachment 402016 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=402016&action=review > > > Source/WebCore/fileapi/BlobBuilder.cpp:74 > > - auto bytes = UTF8Encoding().encodeWithNormalization(text, UnencodableHandling::Entities); > > + auto bytes = UTF8Encoding().encode(text, UnencodableHandling::Entities); > > While this is correct in abstract, I don't think that we can do this before > the normalization is performed in all code paths where decomposed text from > macOS can enter WebKit. And once we do that, encode can stop normalizing. This effect only for constructing `Blob` API from string on JavaScript. The exist `TextEncoding::encode()` do normalization in it and it is widely used in evenrywhere in WebKit. https://trac.webkit.org/browser/webkit/trunk/Source/WebCore/platform/text/TextEncoding.cpp#L77 I had planned to add `TextEncoding::encodeWithoutNormalization()` but I also thought from the comment that it might be better to rename the exist implementation once, I filed bug 213254 and I created this patch built on it. 1663890 4 ap 2020-06-18 09:24:17 -0700 Since blobs can be sent over the wire and/or processed by algorithms that were only tested on Windows, their content should not include decomposed strings coming from macOS. It is true that the test is accurate and we need to make it pass, but disabling normalization would be the last step for making it happen. 1663933 5 tetsuharu.ohzeki 2020-06-18 11:41:04 -0700 Umm..... it's a long way. Thank you for your explanation. 1695102 6 tetsuharu.ohzeki 2020-10-06 05:07:37 -0700 *** This bug has been marked as a duplicate of bug 217327 *** 402016 2020-06-16 09:50:19 -0700 2020-06-16 09:50:19 -0700 Patch bug-213253-20200617015018.patch text/plain 4889 tetsuharu.ohzeki U3VidmVyc2lvbiBSZXZpc2lvbjogMjYzMDg1CmRpZmYgLS1naXQgYS9Tb3VyY2UvV2ViQ29yZS9D aGFuZ2VMb2cgYi9Tb3VyY2UvV2ViQ29yZS9DaGFuZ2VMb2cKaW5kZXggOGFlYmQ0NzdmYjJiMTcy ZWQwODI0YjFlMTBhZTBiMTQ2NTg3NGVkNS4uOWRmZDY2MzcwMzdlYTk4ZTZlN2ExYWZlMmMyMDEx YmJjY2FhNmEwMSAxMDA2NDQKLS0tIGEvU291cmNlL1dlYkNvcmUvQ2hhbmdlTG9nCisrKyBiL1Nv dXJjZS9XZWJDb3JlL0NoYW5nZUxvZwpAQCAtMSwzICsxLDE2IEBACisyMDIwLTA2LTE2ICBUZXRz dWhhcnUgT2h6ZWtpICA8dGV0c3VoYXJ1Lm9oemVraUBnbWFpbC5jb20+CisKKyAgICAgICAgRml4 IHRoZSBjYXNlIGFib3V0IE5GQyBub3JtYWxpemF0aW9uIGluIHdwdC9GaWxlQVBJL3VuaWNvZGUu aHRtbAorICAgICAgICBodHRwczovL2J1Z3Mud2Via2l0Lm9yZy9zaG93X2J1Zy5jZ2k/aWQ9MjEz MjUzCisKKyAgICAgICAgUmV2aWV3ZWQgYnkgTk9CT0RZIChPT1BTISkuCisKKyAgICAgICAgKiBm aWxlYXBpL0Jsb2JCdWlsZGVyLmNwcDoKKyAgICAgICAgKFdlYkNvcmU6OkJsb2JCdWlsZGVyOjph cHBlbmQpOgorICAgICAgICAqIHBsYXRmb3JtL3RleHQvVGV4dEVuY29kaW5nLmNwcDoKKyAgICAg ICAgKFdlYkNvcmU6OlRleHRFbmNvZGluZzo6ZW5jb2RlIGNvbnN0KToKKyAgICAgICAgKiBwbGF0 Zm9ybS90ZXh0L1RleHRFbmNvZGluZy5oOgorCiAyMDIwLTA2LTE2ICB5b3Vlbm4gZmFibGV0ICA8 eW91ZW5uQGFwcGxlLmNvbT4KIAogICAgICAgICBNYWtlIHN1cmUgTWVkaWFSZWNvcmRlci5yZXF1 ZXN0RGF0YSByZXR1cm5zIGRhdGEgd2l0aCB0aGUgbmV3IHdyaXRlciBiYWNrZW5kCmRpZmYgLS1n aXQgYS9Tb3VyY2UvV2ViQ29yZS9maWxlYXBpL0Jsb2JCdWlsZGVyLmNwcCBiL1NvdXJjZS9XZWJD b3JlL2ZpbGVhcGkvQmxvYkJ1aWxkZXIuY3BwCmluZGV4IDg4MGYwOTk4ZTg3ZDFlZWUwM2ZlZmMy Nzc2NDE3ZDhlNmQyNzExMGMuLmUyMmU3OTlhYmU3YjdkYjBhMWZmYjAxYTJkYjA2MjdmNjgwZjI5 MTQgMTAwNjQ0Ci0tLSBhL1NvdXJjZS9XZWJDb3JlL2ZpbGVhcGkvQmxvYkJ1aWxkZXIuY3BwCisr KyBiL1NvdXJjZS9XZWJDb3JlL2ZpbGVhcGkvQmxvYkJ1aWxkZXIuY3BwCkBAIC03MSw3ICs3MSw3 IEBAIHZvaWQgQmxvYkJ1aWxkZXI6OmFwcGVuZChSZWZQdHI8QmxvYj4mJiBibG9iKQogCiB2b2lk IEJsb2JCdWlsZGVyOjphcHBlbmQoY29uc3QgU3RyaW5nJiB0ZXh0KQogewotICAgIGF1dG8gYnl0 ZXMgPSBVVEY4RW5jb2RpbmcoKS5lbmNvZGVXaXRoTm9ybWFsaXphdGlvbih0ZXh0LCBVbmVuY29k YWJsZUhhbmRsaW5nOjpFbnRpdGllcyk7CisgICAgYXV0byBieXRlcyA9IFVURjhFbmNvZGluZygp LmVuY29kZSh0ZXh0LCBVbmVuY29kYWJsZUhhbmRsaW5nOjpFbnRpdGllcyk7CiAKICAgICBpZiAo bV9lbmRpbmdzID09IEVuZGluZ1R5cGU6Ok5hdGl2ZSkKICAgICAgICAgYnl0ZXMgPSBub3JtYWxp emVMaW5lRW5kaW5nc1RvTmF0aXZlKFdURk1vdmUoYnl0ZXMpKTsKZGlmZiAtLWdpdCBhL1NvdXJj ZS9XZWJDb3JlL3BsYXRmb3JtL3RleHQvVGV4dEVuY29kaW5nLmNwcCBiL1NvdXJjZS9XZWJDb3Jl L3BsYXRmb3JtL3RleHQvVGV4dEVuY29kaW5nLmNwcAppbmRleCA2MTA2ODk2NTU5MjgyNzMzMWNk ZDMzYTAzZDVhM2UyMjk2ZTY4NjUxLi4wODY5MDNjY2NlYjE2ZmViYTkxYzQ0MDNhZjljMzk0MTk5 OGQxYjQ0IDEwMDY0NAotLS0gYS9Tb3VyY2UvV2ViQ29yZS9wbGF0Zm9ybS90ZXh0L1RleHRFbmNv ZGluZy5jcHAKKysrIGIvU291cmNlL1dlYkNvcmUvcGxhdGZvcm0vdGV4dC9UZXh0RW5jb2Rpbmcu Y3BwCkBAIC02OSw2ICs2OSwxNCBAQCBTdHJpbmcgVGV4dEVuY29kaW5nOjpkZWNvZGUoY29uc3Qg Y2hhciogZGF0YSwgc2l6ZV90IGxlbmd0aCwgYm9vbCBzdG9wT25FcnJvciwgYgogICAgIHJldHVy biBuZXdUZXh0Q29kZWMoKnRoaXMpLT5kZWNvZGUoZGF0YSwgbGVuZ3RoLCB0cnVlLCBzdG9wT25F cnJvciwgc2F3RXJyb3IpOwogfQogCitWZWN0b3I8dWludDhfdD4gVGV4dEVuY29kaW5nOjplbmNv ZGUoU3RyaW5nVmlldyBzdHJpbmcsIFVuZW5jb2RhYmxlSGFuZGxpbmcgaGFuZGxpbmcpIGNvbnN0 Cit7CisgICAgaWYgKCFtX25hbWUgfHwgc3RyaW5nLmlzRW1wdHkoKSkKKyAgICAgICAgcmV0dXJu IHsgfTsKKworICAgIHJldHVybiBuZXdUZXh0Q29kZWMoKnRoaXMpLT5lbmNvZGUobm9ybWFsaXpl ZFN0cmluZy52aWV3LCBoYW5kbGluZyk7Cit9CisKIFZlY3Rvcjx1aW50OF90PiBUZXh0RW5jb2Rp bmc6OmVuY29kZVdpdGhOb3JtYWxpemF0aW9uKFN0cmluZ1ZpZXcgc3RyaW5nLCBVbmVuY29kYWJs ZUhhbmRsaW5nIGhhbmRsaW5nKSBjb25zdAogewogICAgIGlmICghbV9uYW1lIHx8IHN0cmluZy5p c0VtcHR5KCkpCmRpZmYgLS1naXQgYS9Tb3VyY2UvV2ViQ29yZS9wbGF0Zm9ybS90ZXh0L1RleHRF bmNvZGluZy5oIGIvU291cmNlL1dlYkNvcmUvcGxhdGZvcm0vdGV4dC9UZXh0RW5jb2RpbmcuaApp bmRleCBjMDMwMTI3ZmU1N2FhMGQ4Y2Q3NzFmOTgzZTNiNDIwYjA1NmVlMThlLi5hN2FiMmI2ZGJl NjFkY2MxNGNiNTBmM2Q2YWM0ODk2MWFkMTY3MGY2IDEwMDY0NAotLS0gYS9Tb3VyY2UvV2ViQ29y ZS9wbGF0Zm9ybS90ZXh0L1RleHRFbmNvZGluZy5oCisrKyBiL1NvdXJjZS9XZWJDb3JlL3BsYXRm b3JtL3RleHQvVGV4dEVuY29kaW5nLmgKQEAgLTQ4LDYgKzQ4LDcgQEAgcHVibGljOgogCiAgICAg V0VCQ09SRV9FWFBPUlQgU3RyaW5nIGRlY29kZShjb25zdCBjaGFyKiwgc2l6ZV90IGxlbmd0aCwg Ym9vbCBzdG9wT25FcnJvciwgYm9vbCYgc2F3RXJyb3IpIGNvbnN0OwogICAgIFN0cmluZyBkZWNv ZGUoY29uc3QgY2hhciosIHNpemVfdCBsZW5ndGgpIGNvbnN0OworICAgIFZlY3Rvcjx1aW50OF90 PiBlbmNvZGUoU3RyaW5nVmlldywgVW5lbmNvZGFibGVIYW5kbGluZykgY29uc3Q7CiAgICAgV0VC Q09SRV9FWFBPUlQgVmVjdG9yPHVpbnQ4X3Q+IGVuY29kZVdpdGhOb3JtYWxpemF0aW9uKFN0cmlu Z1ZpZXcsIFVuZW5jb2RhYmxlSGFuZGxpbmcpIGNvbnN0OwogICAgIFZlY3Rvcjx1aW50OF90PiBl bmNvZGVGb3JVUkxQYXJzaW5nKFN0cmluZ1ZpZXcgc3RyaW5nKSBjb25zdCBmaW5hbCB7IHJldHVy biBlbmNvZGVXaXRoTm9ybWFsaXphdGlvbihzdHJpbmcsIFVuZW5jb2RhYmxlSGFuZGxpbmc6OlVS TEVuY29kZWRFbnRpdGllcyk7IH0KIApkaWZmIC0tZ2l0IGEvTGF5b3V0VGVzdHMvaW1wb3J0ZWQv dzNjL0NoYW5nZUxvZyBiL0xheW91dFRlc3RzL2ltcG9ydGVkL3czYy9DaGFuZ2VMb2cKaW5kZXgg M2Y0MTBlMjFlNTFjNDFmNWY4NDBhODVlOTgxNjgxMjdhMDY2NmE0MC4uNjBlNjY5YWFiZTFkMWNl NDE3NGFkOWMwYmY2ZjRmMzYwOWJiMmYzMSAxMDA2NDQKLS0tIGEvTGF5b3V0VGVzdHMvaW1wb3J0 ZWQvdzNjL0NoYW5nZUxvZworKysgYi9MYXlvdXRUZXN0cy9pbXBvcnRlZC93M2MvQ2hhbmdlTG9n CkBAIC0xLDMgKzEsMTIgQEAKKzIwMjAtMDYtMTYgIFRldHN1aGFydSBPaHpla2kgIDx0ZXRzdWhh cnUub2h6ZWtpQGdtYWlsLmNvbT4KKworICAgICAgICBGaXggdGhlIGNhc2UgYWJvdXQgTkZDIG5v cm1hbGl6YXRpb24gaW4gd3B0L0ZpbGVBUEkvdW5pY29kZS5odG1sCisgICAgICAgIGh0dHBzOi8v YnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9pZD0yMTMyNTMKKworICAgICAgICBSZXZpZXdl ZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAgICAqIHdlYi1wbGF0Zm9ybS10ZXN0cy9GaWxl QVBJL3VuaWNvZGUtZXhwZWN0ZWQudHh0OgorCiAyMDIwLTA2LTE2ICBEYXJyeWwgUG9ndWUgIDxk YXJyeWxAZHBvZ3VlLmNhPgogCiAgICAgICAgIEluZGV4ZWREQjogU3VwcG9ydCBJREJGYWN0b3J5 IGRhdGFiYXNlcyBtZXRob2QKZGlmZiAtLWdpdCBhL0xheW91dFRlc3RzL2ltcG9ydGVkL3czYy93 ZWItcGxhdGZvcm0tdGVzdHMvRmlsZUFQSS91bmljb2RlLWV4cGVjdGVkLnR4dCBiL0xheW91dFRl c3RzL2ltcG9ydGVkL3czYy93ZWItcGxhdGZvcm0tdGVzdHMvRmlsZUFQSS91bmljb2RlLWV4cGVj dGVkLnR4dAppbmRleCA5NmU4YWI0MjA5MjIzYWVjYjZjZWRlOTYzZjNjMmExMGI5M2NlNTIwLi5m N2UzNTVkYzZiZjJiZjdlNWYxNWQwZjQ4OWY2MGZjZjM2NjgwNmNjIDEwMDY0NAotLS0gYS9MYXlv dXRUZXN0cy9pbXBvcnRlZC93M2Mvd2ViLXBsYXRmb3JtLXRlc3RzL0ZpbGVBUEkvdW5pY29kZS1l eHBlY3RlZC50eHQKKysrIGIvTGF5b3V0VGVzdHMvaW1wb3J0ZWQvdzNjL3dlYi1wbGF0Zm9ybS10 ZXN0cy9GaWxlQVBJL3VuaWNvZGUtZXhwZWN0ZWQudHh0CkBAIC0xLDYgKzEsNiBAQAogCi1GQUlM IFRlc3QgdGhhdCBzdHJpbmdzIGFyZSBub3QgTkZDIG5vcm1hbGl6ZWQgYnkgQmxvYiBjb25zdHJ1 Y3RvciBhc3NlcnRfZXF1YWxzOiBTdHJpbmcgc2hvdWxkIG5vdCBiZSBub3JtYWxpemVkIGV4cGVj dGVkICLhvbkiIGJ1dCBnb3QgIs+MIgotRkFJTCBUZXN0IHRoYXQgc3RyaW5ncyBhcmUgbm90IE5G QyBub3JtYWxpemVkIGJ5IEZpbGUgY29uc3RydWN0b3IgYXNzZXJ0X2VxdWFsczogU3RyaW5nIHNo b3VsZCBub3QgYmUgbm9ybWFsaXplZCBleHBlY3RlZCAi4b25IiBidXQgZ290ICLPjCIKK1BBU1Mg VGVzdCB0aGF0IHN0cmluZ3MgYXJlIG5vdCBORkMgbm9ybWFsaXplZCBieSBCbG9iIGNvbnN0cnVj dG9yIAorUEFTUyBUZXN0IHRoYXQgc3RyaW5ncyBhcmUgbm90IE5GQyBub3JtYWxpemVkIGJ5IEZp bGUgY29uc3RydWN0b3IgCiBQQVNTIFRlc3QgdGhhdCB1bnBhaXJlZCBzdXJyb2dhdGVzIGFyZSBy ZXBsYWNlZCBieSBCbG9iIGNvbnN0cnVjdG9yIAogUEFTUyBUZXN0IHRoYXQgdW5wYWlyZWQgc3Vy cm9nYXRlcyBhcmUgcmVwbGFjZWQgYnkgRmlsZSBjb25zdHJ1Y3RvciAKIAo=