Bug 222188 – [Meta] Execute run-webkit-tests on Windows Python 3

• Status: RESOLVED
• Resolution: DUPLICATE
• Component: Tools / Tests

[Don Olmstead]

Layout tests should run on Windows Python 3 without Cygwin.

[Don Olmstead]

Created attachment 421011 [details]
WIP Patch

This is a very rough set of changes that got run-webkit-tests to begin execution on Windows using Python 3.9.1 without cygwin.

The larger issue that this patch doesn't address is that a config.json was never created within LayoutTests/imported/w3c/web-platform-tests despite the script starting a wpt.py serve with a config.

python C:/webkit/LayoutTests/imported/w3c/web-platform-tests/wpt.py serve --config C:/webkit/LayoutTests/imported/w3c/web-platform-tests/config.json

To get around this I made a copy of config.default.json in that directory with the expected name. This along with the changes in this patch got it so the servers started and execution of the layout tests began.

Our bots only had openssl because it was included with perl and it was on the path. It would probably be better to include openssl in the requirements and use that one rather than rely on perl.

[Don Olmstead]

Comment on attachment 421011 [details]
WIP Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=421011&action=review

Just some notes on the patch

> LayoutTests/imported/w3c/web-platform-tests/tools/pywebsocket/mod_pywebsocket/util.py:133
> -    __orig_popen3 = os.popen3
> +    #__orig_popen3 = os.popen3

Upstream bug opened at https://github.com/GoogleChromeLabs/pywebsocket3/issues/17

Once its fixed another issue will need to be opened to integrate it into the web platform tests repository.

> LayoutTests/imported/w3c/web-platform-tests/tools/wptserve/wptserve/sslutils/openssl.py:327
> +                                   "-in", cert_path).decode().split("=", 1)[1].strip()

Without this change the following occurs. Looks to be an issue with Strings in Python 3 vs 2.

PS C:\webkit> python C:/webkit/LayoutTests/imported/w3c/web-platform-tests/wpt.py serve --config C:\webkit\LayoutTests\imported\w3c\web-platform-tests\config.json
WARNING:web-platform-tests:bind_hostname in config is deprecated; use bind_address instead
WARNING:web-platform-tests:bind_hostname in config is deprecated; use bind_address instead
DEBUG:web-platform-tests:b'notAfter=Mar 20 00:52:35 2021 GMT\n'
Traceback (most recent call last):
  File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\wpt.py", line 7, in <module>
    exec(compile(open("wpt", "r").read(), "wpt", 'exec'))
  File "wpt", line 5, in <module>
    wpt.main()
  File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\wpt\wpt.py", line 173, in main
    rv = script(*args, **kwargs)
  File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\serve\serve.py", line 971, in run
    with build_config(os.path.join(repo_root, "config.json"),
  File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\wptserve\wptserve\config.py", line 272, in __enter__
    data[key] = getattr(self, prefix + key)(data)
  File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\wptserve\wptserve\config.py", line 348, in _get_ssl_config
    key_path, cert_path = self._ssl_env.host_cert_path(data["domains_set"])
  File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\wptserve\wptserve\sslutils\openssl.py", line 382, in host_cert_path
    key_cert = self._load_host_cert(hosts)
  File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\wptserve\wptserve\sslutils\openssl.py", line 399, in _load_host_cert
    if self.check_key_cert(key_path, cert_path, hosts):
  File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\wptserve\wptserve\sslutils\openssl.py", line 324, in check_key_cert
    end_date_str = openssl("x509",
AttributeError: 'bytes' object has no attribute 'encode'

The file has been modified within the last month for Python 3 compatibility, https://github.com/web-platform-tests/wpt/blob/master/tools/wptserve/wptserve/sslutils/openssl.py , but the last time we have taken in changes is from May 2020, https://github.com/WebKit/WebKit/commit/f1e206ac080c4da01987340857c5a80c37437939 so this issue may go away with an update.

> Tools/Scripts/webkitpy/layout_tests/servers/websocket_server.py:124
> -        if sys.version_info > (3, 0):
> -            python_interp = 'python2'
> +        #if sys.version_info > (3, 0):
> +        #    python_interp = 'python2'

Revert of https://github.com/WebKit/WebKit/commit/490485522c87f5f354c3b0c2a6dd13ace195e6d4

> Tools/Scripts/webkitpy/layout_tests/servers/websocket_server.py:160
> +        print('PYTHONPATH {}'.format(self._env['PYTHONPATH']))

This is just me debugging to be able to run things locally.

[Radar WebKit Bug Importer]

<rdar://problem/74801439>

[Don Olmstead]

The problems within wpt/tools running on Python 3 were fixed upstream in https://github.com/web-platform-tests/wpt/commit/5c496d309c2d31c9867ef7a5023a6f7021ba0751

[Don Olmstead]

Created attachment 422170 [details]
config.json

Attaching a config.json to get things running.

The wpt directory has a config.default.json but no config.json. It isn't clear where the config.json comes from or if its just other ports don't need it.

[Sam Sneddon [:gsnedders]]

Don, what's the status of this now?

[Don Olmstead]

(In reply to Sam Sneddon [:gsnedders] from comment #6)
> Don, what's the status of this now?

Hey Sam apologies for the late response I was on PTO. I can potentially take a look at it this week depending on your availability. Just ping me on Slack.

[Sam Sneddon [:gsnedders]]

Since bug 227255, wptserve has always been run on Python 3, and since bug 230319, pywebsocket has always been run on Python 3.

So… resolved duplicate?

*** This bug has been marked as a duplicate of bug 227255 ***