258251 2023-06-17 16:04:59 -0700 REGRESSION(264918@main): GB18030 encoding isn't hooked up correctly 2023-06-29 15:38:19 -0700 1 1 1 Unclassified WebKit Text WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED InRadar P2 Normal --- 1 mmaxfield achristensen mmaxfield webkit-bug-importer oldest_to_newest 1962297 0 mmaxfield 2023-06-17 16:04:59 -0700 We're encoding U+E78D to 0x83 0x36 0xCB 0x32 which seems totally wrong. That is neither 0xA6 0xD9 nor 0x84 0x31 0x82 0x36 (which are the sequences on L2/23-003R[1]). If we round-trip our byte sequence back to a code point, it decodes to U+E82E which is just a totally different PUA character. [1] https://www.unicode.org/L2/L2023/23003r-gb18030-recommendations.pdf 1962298 1 webkit-bug-importer 2023-06-17 16:05:16 -0700 <rdar://problem/110952885> 1962299 2 466741 mmaxfield 2023-06-17 16:06:41 -0700 Created attachment 466741 encoder observation 1962300 3 466742 mmaxfield 2023-06-17 16:06:50 -0700 Created attachment 466742 decoder observation 1964282 4 achristensen 2023-06-29 10:32:28 -0700 Pull request: https://github.com/WebKit/WebKit/pull/15413 1964337 5 ews-feeder 2023-06-29 15:38:17 -0700 Committed 265633@main (0a2e4563bda0): <https://commits.webkit.org/265633@main> Reviewed commits have been landed. Closing PR #15413 and removing active labels. 466741 2023-06-17 16:06:41 -0700 2023-06-17 16:06:41 -0700 encoder observation encoder.html text/html 885 mmaxfield PCFET0NUWVBFIGh0bWw+CjxodG1sPgo8bWV0YSBjaGFyc2V0PSJHQjE4MDMwIj4KPGhlYWQ+Cjxz Y3JpcHQ+CmZ1bmN0aW9uIGVuY29kZShpKSB7CiAgICBzdGF0dXMudGV4dENvbnRlbnQgPSAiVSsi ICsgaS50b1N0cmluZygxNik7CiAgICBjb25zdCBjaGFyYWN0ZXJTdHJpbmcgPSBTdHJpbmcuZnJv bUNvZGVQb2ludChpKTsKICAgIGNvbnN0IGEgPSBkb2N1bWVudC5jcmVhdGVFbGVtZW50KCJhIik7 CiAgICBhLmhyZWYgPSAiaHR0cHM6Ly9leGFtcGxlLmNvbS8/IiArIGNoYXJhY3RlclN0cmluZzsK ICAgIGNvbnN0IHBlcmNlbnRFbmNvZGVkU3RyaW5nID0gYS5zZWFyY2guc3Vic3RyKDEpOwogICAg YWxlcnQocGVyY2VudEVuY29kZWRTdHJpbmcpOwogICAgY29uc3QgcmVzdWx0ID0gW107CiAgICBm b3IgKGxldCBpID0gMDsgaSA8IHBlcmNlbnRFbmNvZGVkU3RyaW5nLmxlbmd0aDspIHsKICAgICAg ICBpZiAocGVyY2VudEVuY29kZWRTdHJpbmdbaV0gPT0gIiUiKSB7CiAgICAgICAgICAgIHJlc3Vs dC5wdXNoKE51bWJlcigiMHgiICsgcGVyY2VudEVuY29kZWRTdHJpbmcuc3Vic3RyaW5nKGkgKyAx LCBpICsgMykpKTsKICAgICAgICAgICAgaSArPSAzOwogICAgICAgIH0gZWxzZSB7CiAgICAgICAg ICAgIHJlc3VsdC5wdXNoKHBlcmNlbnRFbmNvZGVkU3RyaW5nLmNoYXJDb2RlQXQoaSkpOwogICAg ICAgICAgICArK2k7CiAgICAgICAgfQogICAgfQogICAgcmV0dXJuIHJlc3VsdDsKfQphbGVydChl bmNvZGUoMHhGRTEwKS5tYXAobiA9PiAiVSsiICsgbi50b1N0cmluZygxNikpKTsKYWxlcnQoZW5j b2RlKDB4RTc4RCkubWFwKG4gPT4gIlUrIiArIG4udG9TdHJpbmcoMTYpKSk7Cjwvc2NyaXB0Pgo8 L2hlYWQ+Cjxib2R5Pgo8L2JvZHk+CjwvaHRtbD4K 466742 2023-06-17 16:06:50 -0700 2023-06-17 16:06:50 -0700 decoder observation decoder.html text/html 370 mmaxfield PCFET0NUWVBFIGh0bWw+CjxodG1sPgo8bWV0YSBjaGFyc2V0PSJHQjE4MDMwIj4KPGhlYWQ+Cjxz Y3JpcHQ+CmZ1bmN0aW9uIGRlY29kZShhKSB7CiAgICBjb25zdCB0ZXh0RGVjb2RlciA9IG5ldyBU ZXh0RGVjb2RlcigiR0IxODAzMCIsIHtmYXRhbDogdHJ1ZX0pOwogICAgcmV0dXJuIHRleHREZWNv ZGVyLmRlY29kZShuZXcgVWludDhBcnJheShhKSkuY29kZVBvaW50QXQoMCk7Cn0KYWxlcnQoIlUr IiArIGRlY29kZShbMHhBNiwgMHhEOV0pLnRvU3RyaW5nKDE2KSk7CmFsZXJ0KCJVKyIgKyBkZWNv ZGUoWzB4ODQsIDB4MzEsIDB4ODIsIDB4MzZdKS50b1N0cmluZygxNikpOwo8L3NjcmlwdD4KPC9o ZWFkPgo8Ym9keT4KPC9ib2R5Pgo8L2h0bWw+Cg==