| Summary: | Update WPT encoding test suite to the latest version. | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Product: | WebKit | Reporter: | Maciej Stachowiak <mjs> | ||||||||||
| Component: | New Bugs | Assignee: | Maciej Stachowiak <mjs> | ||||||||||
| Status: | RESOLVED FIXED | ||||||||||||
| Severity: | Normal | CC: | aakash_jain, ap, buildbot, cdumez, commit-queue, darin, rniwa, ryanhaddad, webkit-bug-importer, youennf | ||||||||||
| Priority: | P2 | Keywords: | InRadar | ||||||||||
| Version: | WebKit Nightly Build | ||||||||||||
| Hardware: | Unspecified | ||||||||||||
| OS: | Unspecified | ||||||||||||
| Bug Depends on: | |||||||||||||
| Bug Blocks: | 179303 | ||||||||||||
| Attachments: |
|
||||||||||||
|
Description
Maciej Stachowiak
2017-11-05 23:24:22 PST
Created attachment 326106 [details]
Patch
Did you use Tools/Scripts/import-w3c-tests ? It does a lot os work besides copying the tests over to make sure the tests run properly as webkit layout tests. Among other things, it skips the resource files automatically (via a JSON file iirc, not TestExpectations). It also updates some *.txt files used by import-w3c-tests for later re-syncs of the folder. (In reply to Chris Dumez from comment #2) > Did you use Tools/Scripts/import-w3c-tests ? It does a lot os work besides > copying the tests over to make sure the tests run properly as webkit layout > tests. Among other things, it skips the resource files automatically (via a > JSON file iirc, not TestExpectations). > It also updates some *.txt files used by import-w3c-tests for later re-syncs > of the folder. I tried to use it, but it did not appear to update the encoding directory at all, so I copied manually. Would appreciate pointers on how to use it correctly. imported/w3c/resources/import-expectations.json is used to know which tests to import. You can enable 'encoding' test suite import by updating this file directly and running import-w3c-tests directly. Or you can use something like: 'import-w3c-tests web-platform-tests/encoding' This should import the encoding test suite with the revision defined in TestRepositories file. The script should then update import-expectations.json for you so that later wpt revision bump also includes the encoding test suite. Looking at import-expectations.json, the encoding folder is currently skipped. It was probably imported manually before. Ideally, the current web-platform-tests/encoding folder should be removed and then reimported so that obsolete files are not kept in the folder. > Looking at import-expectations.json, the encoding folder is currently skipped. It was probably imported manually before. > Ideally, the current web-platform-tests/encoding folder should be removed and then reimported so that obsolete files are not kept in the folder. Given that status, I'd like to reimport manually one final time, then the remove/reimport step, so I can verify that nothing went wrong during the re-import. (But first I need a patch that applies clean.) It might be good then to use the WPT commit defined in LayoutTests/imported/w3c/resources/TestRepositories, current value is "a1c0107". As of the resource files to skip, the file to update is LayoutTests/imported/w3c/resources/resource-files.json import-w3c-tests should do the edit automatically. My patch matches WPT revision a1c0107 of web-platform-tests/encoding/, so should be good to go if I do a manual update and then merge. bugzilla is slow to me... r=me as long as bots are happy. Please move the test expectations skipping to imported/w3c/resources/resource-files.json so that we do not forget to remove these expectations in the clean-up patch. Created attachment 326136 [details]
Patch
I'll see if this new version succeeds on the bots and if so I'll try doing the skipping via resource-files.json. Created attachment 326175 [details]
Patch
The patch is green on all bots now and addresses Youenn's comment about using resource-files.json to skip non-test files. Can someone give it the official r+? Warning: Review Patch and Details views will be very slow to load because the patch is huge. r=me. Sorry for not being able to mark r+ but bugzilla is slow for this large patch... I do not understand why it is so huge though. Is it because of TestExpectations? As for tests being marked as slow, import-w3c-tests should be able to handle these directly when "meta timeout" is set in the tests which seem to be the case here. In the cleanup patch, you might be able to remove the slow expectations an let the script do its stuff directly r=me. Sorry for not being able to mark r+ but bugzilla is slow for this large patch... I do not understand why it is so huge though. Is it because of -expected.txt files? As for tests being marked as slow, import-w3c-tests should be able to handle these directly when "meta timeout" is set in the tests which seem to be the case here. In the cleanup patch, you might be able to remove the slow expectations an let the script do its stuff directly Some of the -expected.txt files include the full range of certain character sets, I believe that is why the patch is so large. Comment on attachment 326175 [details]
Patch
Obsoleting because I'm about to upload a patch for CQ to land.
Created attachment 326214 [details]
Patch for landing
This landed in https://trac.webkit.org/changeset/224536/webkit Many of these tests are timing out on debug bots: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#tests=imported%2Fw3c%2Fweb-platform-tests%2Fencoding%2Flegacy-mb (In reply to Ryan Haddad from comment #19) > This landed in https://trac.webkit.org/changeset/224536/webkit > > Many of these tests are timing out on debug bots: > https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard. > html#tests=imported%2Fw3c%2Fweb-platform-tests%2Fencoding%2Flegacy-mb These are the ones that have timed out at least once: imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-han.html imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-extBa.html imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-href.html imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form.html imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-x-x-big5.html imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-cn-big5.html imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-href-errors-han.html imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-extBb.html imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-pua.html imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-decode-big5-hkscs.html imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-csbig5.html imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-hangul.html imported/w3c/web-platform-tests/encoding/legacy-mb-japanese/shift_jis/sjis-decode-ms_kanji.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-href-errors-han.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ks_c_5601-1989.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ksc_5601.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-csksc56011987.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ksc5601.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-href.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-windows-949.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-iso-ir-149.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-errors-han.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ks_c_5601-1987.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-cseuckr.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-korean.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ks_c_5601-1987.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ksc5601.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-korean.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-csksc56011987.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ksc_5601.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-iso-ir-149.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ks_c_5601-1989.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-windows-949.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-cseuckr.html imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode.html I updated TestExpectations for these tests in https://trac.webkit.org/r224554 I'm not quite sure how this landed (and apparently as me, not as the CQ), but I guess it's resolved now. (In reply to Maciej Stachowiak from comment #22) > I'm not quite sure how this landed (and apparently as me, not as the CQ), > but I guess it's resolved now. The commit queue lands patches as the patch author if the author has commit privileges. But also the commit queue adds a comment to the bug, and I don’t see that comment! Bugzilla timed out loading the details page when EWS tried to remove the cq+ flag, and so processing got interrupted. This will be eventually fixed as we migrate from mechanize to Bugzilla REST API (bug 176344). Traceback (most recent call last): File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch", line 84, in <module> main() File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch", line 79, in main WebKitPatch(os.path.abspath(__file__)).main() File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 305, in main result = command.check_arguments_and_execute(options, args, self) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 123, in check_arguments_and_execute return self.execute(options, args, tool) or 0 File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/commands/download.py", line 186, in execute self._process_patch(patch, options, args, tool) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/commands/download.py", line 212, in _process_patch self._main_sequence.run_and_handle_errors(tool, options, state) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 73, in run_and_handle_errors self._run(tool, options, state) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 67, in _run step(tool, options).run(state) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/steps/closepatch.py", line 36, in run self._tool.bugs.clear_attachment_flags(state["patch"].id(), comment_text) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/common/net/bugzilla/bugzilla.py", line 762, in clear_attachment_flags self.browser.select_form(nr=1) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_mechanize.py", line 499, in select_form global_form = self._factory.global_form File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_html.py", line 544, in __getattr__ self.forms() File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_html.py", line 557, in forms self._forms_factory.forms()) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_html.py", line 237, in forms _urlunparse=_rfc3986.urlunsplit, File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_form.py", line 844, in ParseResponseEx _urlunparse=_urlunparse, File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_form.py", line 979, in _ParseFileEx data = file.read(CHUNK) File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_response.py", line 195, in read data = self.wrapped.read(to_read) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 384, in read data = self._sock.recv(left) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 588, in read return self._read_chunked(amt) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 648, in _read_chunked value.append(self._safe_read(amt)) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 703, in _safe_read chunk = self.fp.read(min(amt, MAXAMOUNT)) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 384, in read data = self._sock.recv(left) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 734, in recv return self.read(buflen) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 621, in read v = self._sslobj.read(len or 1024) socket.error: [Errno 54] Connection reset by peer |