Bug 179312 – Update WPT encoding test suite to the latest version.

• Component: New Bugs
• Status: RESOLVED FIXED

[Maciej Stachowiak]

Update WPT encoding test suite to the latest version.

[Maciej Stachowiak]

Created attachment 326106 [details]
Patch

[Chris Dumez]

Did you use Tools/Scripts/import-w3c-tests ? It does a lot os work besides copying the tests over to make sure the tests run properly as webkit layout tests. Among other things, it skips the resource files automatically (via a JSON file iirc, not TestExpectations).
It also updates some *.txt files used by import-w3c-tests for later re-syncs of the folder.

[Maciej Stachowiak]

(In reply to Chris Dumez from comment #2)
> Did you use Tools/Scripts/import-w3c-tests ? It does a lot os work besides
> copying the tests over to make sure the tests run properly as webkit layout
> tests. Among other things, it skips the resource files automatically (via a
> JSON file iirc, not TestExpectations).
> It also updates some *.txt files used by import-w3c-tests for later re-syncs
> of the folder.

I tried to use it, but it did not appear to update the encoding directory at all, so I copied manually. Would appreciate pointers on how to use it correctly.

[youenn fablet]

imported/w3c/resources/import-expectations.json is used to know which tests to import.
You can enable 'encoding' test suite import by updating this file directly and running import-w3c-tests directly.

Or you can use something like: 'import-w3c-tests web-platform-tests/encoding'
This should import the encoding test suite with the revision defined in TestRepositories file.
The script should then update import-expectations.json for you so that later wpt revision bump also includes the encoding test suite.

[youenn fablet]

Looking at import-expectations.json, the encoding folder is currently skipped.
It was probably imported manually before.
Ideally, the current web-platform-tests/encoding folder should be removed and then reimported so that obsolete files are not kept in the folder.

[Maciej Stachowiak]

> Looking at import-expectations.json, the encoding folder is currently skipped.
It was probably imported manually before.
> Ideally, the current web-platform-tests/encoding folder should be removed and then reimported so that obsolete files are not kept in the folder.

Given that status, I'd like to reimport manually one final time, then the remove/reimport step, so I can verify that nothing went wrong during the re-import.

(But first I need a patch that applies clean.)

[youenn fablet]

It might be good then to use the WPT commit defined in LayoutTests/imported/w3c/resources/TestRepositories, current value is "a1c0107".

As of the resource files to skip, the file to update is LayoutTests/imported/w3c/resources/resource-files.json
import-w3c-tests should do the edit automatically.

[Maciej Stachowiak]

My patch matches WPT revision a1c0107 of web-platform-tests/encoding/, so should be good to go if I do a manual update and then merge.

[youenn fablet]

bugzilla is slow to me...

r=me as long as bots are happy.

Please move the test expectations skipping to imported/w3c/resources/resource-files.json so that we do not forget to remove these expectations in the clean-up patch.

[Maciej Stachowiak]

Created attachment 326136 [details]
Patch

[Maciej Stachowiak]

I'll see if this new version succeeds on the bots and if so I'll try doing the skipping via resource-files.json.

[Maciej Stachowiak]

Created attachment 326175 [details]
Patch

[Maciej Stachowiak]

The patch is green on all bots now and addresses Youenn's comment about using resource-files.json to skip non-test files. Can someone give it the official r+?

Warning: Review Patch and Details views will be very slow to load because the patch is huge.

[youenn fablet]

r=me.

Sorry for not being able to mark r+ but bugzilla is slow for this large patch...
I do not understand why it is so huge though. Is it because of TestExpectations?

As for tests being marked as slow, import-w3c-tests should be able to handle these directly when "meta timeout" is set in the tests which seem to be the case here.
In the cleanup patch, you might be able to remove the slow expectations an let the script do its stuff directly

[youenn fablet]

r=me.

Sorry for not being able to mark r+ but bugzilla is slow for this large patch...
I do not understand why it is so huge though. Is it because of -expected.txt files?

As for tests being marked as slow, import-w3c-tests should be able to handle these directly when "meta timeout" is set in the tests which seem to be the case here.
In the cleanup patch, you might be able to remove the slow expectations an let the script do its stuff directly

[Maciej Stachowiak]

Some of the -expected.txt files include the full range of certain character sets, I believe that is why the patch is so large.

[Maciej Stachowiak]

Comment on attachment 326175 [details]
Patch

Obsoleting because I'm about to upload a patch for CQ to land.

[Maciej Stachowiak]

Created attachment 326214 [details]
Patch for landing

[Ryan Haddad]

This landed in https://trac.webkit.org/changeset/224536/webkit

Many of these tests are timing out on debug bots:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#tests=imported%2Fw3c%2Fweb-platform-tests%2Fencoding%2Flegacy-mb

[Ryan Haddad]

(In reply to Ryan Haddad from comment #19)
> This landed in https://trac.webkit.org/changeset/224536/webkit
> 
> Many of these tests are timing out on debug bots:
> https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.
> html#tests=imported%2Fw3c%2Fweb-platform-tests%2Fencoding%2Flegacy-mb

These are the ones that have timed out at least once:
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-han.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-extBa.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-href.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-x-x-big5.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-cn-big5.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-href-errors-han.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-extBb.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-pua.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-decode-big5-hkscs.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-csbig5.html
imported/w3c/web-platform-tests/encoding/legacy-mb-tchinese/big5/big5-encode-form-errors-hangul.html
imported/w3c/web-platform-tests/encoding/legacy-mb-japanese/shift_jis/sjis-decode-ms_kanji.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-href-errors-han.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ks_c_5601-1989.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ksc_5601.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-csksc56011987.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ksc5601.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-href.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-windows-949.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-iso-ir-149.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-errors-han.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-ks_c_5601-1987.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-cseuckr.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form-korean.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-encode-form.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ks_c_5601-1987.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ksc5601.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-korean.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-csksc56011987.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ksc_5601.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-iso-ir-149.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-ks_c_5601-1989.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-windows-949.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode-cseuckr.html
imported/w3c/web-platform-tests/encoding/legacy-mb-korean/euc-kr/euckr-decode.html

[Ryan Haddad]

I updated TestExpectations for these tests in https://trac.webkit.org/r224554

[Maciej Stachowiak]

I'm not quite sure how this landed (and apparently as me, not as the CQ), but I guess it's resolved now.

[Darin Adler]

(In reply to Maciej Stachowiak from comment #22)
> I'm not quite sure how this landed (and apparently as me, not as the CQ),
> but I guess it's resolved now.

The commit queue lands patches as the patch author if the author has commit privileges. But also the commit queue adds a comment to the bug, and I don’t see that comment!

[Alexey Proskuryakov]

Bugzilla timed out loading the details page when EWS tried to remove the cq+ flag, and so processing got interrupted. This will be eventually fixed as we migrate from mechanize to Bugzilla REST API (bug 176344).

Traceback (most recent call last):
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch", line 84, in <module>
    main()
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch", line 79, in main
    WebKitPatch(os.path.abspath(__file__)).main()
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 305, in main
    result = command.check_arguments_and_execute(options, args, self)
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 123, in check_arguments_and_execute
    return self.execute(options, args, tool) or 0
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/commands/download.py", line 186, in execute
    self._process_patch(patch, options, args, tool)
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/commands/download.py", line 212, in _process_patch
    self._main_sequence.run_and_handle_errors(tool, options, state)
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 73, in run_and_handle_errors
    self._run(tool, options, state)
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 67, in _run
    step(tool, options).run(state)
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/tool/steps/closepatch.py", line 36, in run
    self._tool.bugs.clear_attachment_flags(state["patch"].id(), comment_text)
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/common/net/bugzilla/bugzilla.py", line 762, in clear_attachment_flags
    self.browser.select_form(nr=1)
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_mechanize.py", line 499, in select_form
    global_form = self._factory.global_form
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_html.py", line 544, in __getattr__
    self.forms()
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_html.py", line 557, in forms
    self._forms_factory.forms())
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_html.py", line 237, in forms
    _urlunparse=_rfc3986.urlunsplit,
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_form.py", line 844, in ParseResponseEx
    _urlunparse=_urlunparse,
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_form.py", line 979, in _ParseFileEx
    data = file.read(CHUNK)
  File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/mechanize/_response.py", line 195, in read
    data = self.wrapped.read(to_read)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 384, in read
    data = self._sock.recv(left)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 588, in read
    return self._read_chunked(amt)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 648, in _read_chunked
    value.append(self._safe_read(amt))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/httplib.py", line 703, in _safe_read
    chunk = self.fp.read(min(amt, MAXAMOUNT))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 384, in read
    data = self._sock.recv(left)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 734, in recv
    return self.read(buflen)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 621, in read
    v = self._sslobj.read(len or 1024)
socket.error: [Errno 54] Connection reset by peer

[Radar WebKit Bug Importer]

<rdar://problem/35621421>