222188 2021-02-19 12:17:52 -0800 [Meta] Execute run-webkit-tests on Windows Python 3 2024-02-07 03:50:57 -0800 1 1 1 Unclassified WebKit Tools / Tests WebKit Nightly Build Unspecified Unspecified RESOLVED DUPLICATE 227255 https://bugs.webkit.org/show_bug.cgi?id=203701 InRadar P2 Normal --- 222703 1 don.olmstead webkit-unassigned gsnedders webkit-bug-importer oldest_to_newest 1731503 0 don.olmstead 2021-02-19 12:17:52 -0800 Layout tests should run on Windows Python 3 without Cygwin. 1731515 1 421011 don.olmstead 2021-02-19 12:32:55 -0800 Created attachment 421011 WIP Patch This is a very rough set of changes that got run-webkit-tests to begin execution on Windows using Python 3.9.1 without cygwin. The larger issue that this patch doesn't address is that a config.json was never created within LayoutTests/imported/w3c/web-platform-tests despite the script starting a wpt.py serve with a config. python C:/webkit/LayoutTests/imported/w3c/web-platform-tests/wpt.py serve --config C:/webkit/LayoutTests/imported/w3c/web-platform-tests/config.json To get around this I made a copy of config.default.json in that directory with the expected name. This along with the changes in this patch got it so the servers started and execution of the layout tests began. Our bots only had openssl because it was included with perl and it was on the path. It would probably be better to include openssl in the requirements and use that one rather than rely on perl. 1731523 2 421011 don.olmstead 2021-02-19 12:41:15 -0800 Comment on attachment 421011 WIP Patch View in context: https://bugs.webkit.org/attachment.cgi?id=421011&action=review Just some notes on the patch > LayoutTests/imported/w3c/web-platform-tests/tools/pywebsocket/mod_pywebsocket/util.py:133 > - __orig_popen3 = os.popen3 > + #__orig_popen3 = os.popen3 Upstream bug opened at https://github.com/GoogleChromeLabs/pywebsocket3/issues/17 Once its fixed another issue will need to be opened to integrate it into the web platform tests repository. > LayoutTests/imported/w3c/web-platform-tests/tools/wptserve/wptserve/sslutils/openssl.py:327 > + "-in", cert_path).decode().split("=", 1)[1].strip() Without this change the following occurs. Looks to be an issue with Strings in Python 3 vs 2. PS C:\webkit> python C:/webkit/LayoutTests/imported/w3c/web-platform-tests/wpt.py serve --config C:\webkit\LayoutTests\imported\w3c\web-platform-tests\config.json WARNING:web-platform-tests:bind_hostname in config is deprecated; use bind_address instead WARNING:web-platform-tests:bind_hostname in config is deprecated; use bind_address instead DEBUG:web-platform-tests:b'notAfter=Mar 20 00:52:35 2021 GMT\n' Traceback (most recent call last): File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\wpt.py", line 7, in <module> exec(compile(open("wpt", "r").read(), "wpt", 'exec')) File "wpt", line 5, in <module> wpt.main() File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\wpt\wpt.py", line 173, in main rv = script(*args, **kwargs) File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\serve\serve.py", line 971, in run with build_config(os.path.join(repo_root, "config.json"), File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\wptserve\wptserve\config.py", line 272, in __enter__ data[key] = getattr(self, prefix + key)(data) File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\wptserve\wptserve\config.py", line 348, in _get_ssl_config key_path, cert_path = self._ssl_env.host_cert_path(data["domains_set"]) File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\wptserve\wptserve\sslutils\openssl.py", line 382, in host_cert_path key_cert = self._load_host_cert(hosts) File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\wptserve\wptserve\sslutils\openssl.py", line 399, in _load_host_cert if self.check_key_cert(key_path, cert_path, hosts): File "C:\webkit\LayoutTests\imported\w3c\web-platform-tests\tools\wptserve\wptserve\sslutils\openssl.py", line 324, in check_key_cert end_date_str = openssl("x509", AttributeError: 'bytes' object has no attribute 'encode' The file has been modified within the last month for Python 3 compatibility, https://github.com/web-platform-tests/wpt/blob/master/tools/wptserve/wptserve/sslutils/openssl.py , but the last time we have taken in changes is from May 2020, https://github.com/WebKit/WebKit/commit/f1e206ac080c4da01987340857c5a80c37437939 so this issue may go away with an update. > Tools/Scripts/webkitpy/layout_tests/servers/websocket_server.py:124 > - if sys.version_info > (3, 0): > - python_interp = 'python2' > + #if sys.version_info > (3, 0): > + # python_interp = 'python2' Revert of https://github.com/WebKit/WebKit/commit/490485522c87f5f354c3b0c2a6dd13ace195e6d4 > Tools/Scripts/webkitpy/layout_tests/servers/websocket_server.py:160 > + print('PYTHONPATH {}'.format(self._env['PYTHONPATH'])) This is just me debugging to be able to run things locally. 1733917 3 webkit-bug-importer 2021-02-26 12:18:12 -0800 <rdar://problem/74801439> 1735639 4 don.olmstead 2021-03-03 19:03:55 -0800 The problems within wpt/tools running on Python 3 were fixed upstream in https://github.com/web-platform-tests/wpt/commit/5c496d309c2d31c9867ef7a5023a6f7021ba0751 1735640 5 422170 don.olmstead 2021-03-03 19:05:28 -0800 Created attachment 422170 config.json Attaching a config.json to get things running. The wpt directory has a config.default.json but no config.json. It isn't clear where the config.json comes from or if its just other ports don't need it. 1783498 6 gsnedders 2021-08-11 08:27:42 -0700 Don, what's the status of this now? 1784618 7 don.olmstead 2021-08-16 12:41:57 -0700 (In reply to Sam Sneddon [:gsnedders] from comment #6) > Don, what's the status of this now? Hey Sam apologies for the late response I was on PTO. I can potentially take a look at it this week depending on your availability. Just ping me on Slack. 2011352 8 gsnedders 2024-02-07 03:50:57 -0800 Since bug 227255, wptserve has always been run on Python 3, and since bug 230319, pywebsocket has always been run on Python 3. So… resolved duplicate? *** This bug has been marked as a duplicate of bug 227255 *** 421011 2021-02-19 12:32:55 -0800 2021-02-19 12:32:55 -0800 WIP Patch bug-222188.diff text/plain 3106 don.olmstead ZGlmZiAtLWdpdCBhL0xheW91dFRlc3RzL2ltcG9ydGVkL3czYy93ZWItcGxhdGZvcm0tdGVzdHMv dG9vbHMvcHl3ZWJzb2NrZXQvbW9kX3B5d2Vic29ja2V0L3V0aWwucHkgYi9MYXlvdXRUZXN0cy9p bXBvcnRlZC93M2Mvd2ViLXBsYXRmb3JtLXRlc3RzL3Rvb2xzL3B5d2Vic29ja2V0L21vZF9weXdl YnNvY2tldC91dGlsLnB5CmluZGV4IDZjODI4NGNkMDc0NC4uZGE0NDE4ZDUwOGQ3IDEwMDY0NAot LS0gYS9MYXlvdXRUZXN0cy9pbXBvcnRlZC93M2Mvd2ViLXBsYXRmb3JtLXRlc3RzL3Rvb2xzL3B5 d2Vic29ja2V0L21vZF9weXdlYnNvY2tldC91dGlsLnB5CisrKyBiL0xheW91dFRlc3RzL2ltcG9y dGVkL3czYy93ZWItcGxhdGZvcm0tdGVzdHMvdG9vbHMvcHl3ZWJzb2NrZXQvbW9kX3B5d2Vic29j a2V0L3V0aWwucHkKQEAgLTEzMCw3ICsxMzAsNyBAQCBkZWYgd3JhcF9wb3BlbjNfZm9yX3dpbihj eWd3aW5fcGF0aCk6CiAgICAgICBjeWd3aW5fcGF0aDogIHBhdGggZm9yIGN5Z3dpbiBiaW5hcnkg aWYgY29tbWFuZCBwYXRoIGlzIG5lZWRlZCB0byBiZQogICAgICAgICAgICAgICAgICAgICB0cmFu c2xhdGVkLiAgTm9uZSBpZiBubyB0cmFuc2xhdGlvbiByZXF1aXJlZC4KICAgICAiIiIKLSAgICBf X29yaWdfcG9wZW4zID0gb3MucG9wZW4zCisgICAgI19fb3JpZ19wb3BlbjMgPSBvcy5wb3BlbjMK IAogICAgIGRlZiBfX3dyYXBfcG9wZW4zKGNtZCwgbW9kZT0ndCcsIGJ1ZnNpemU9LTEpOgogICAg ICAgICBjbWRsaW5lID0gY21kLnNwbGl0KCcgJykKZGlmZiAtLWdpdCBhL0xheW91dFRlc3RzL2lt cG9ydGVkL3czYy93ZWItcGxhdGZvcm0tdGVzdHMvdG9vbHMvd3B0c2VydmUvd3B0c2VydmUvc3Ns dXRpbHMvb3BlbnNzbC5weSBiL0xheW91dFRlc3RzL2ltcG9ydGVkL3czYy93ZWItcGxhdGZvcm0t dGVzdHMvdG9vbHMvd3B0c2VydmUvd3B0c2VydmUvc3NsdXRpbHMvb3BlbnNzbC5weQppbmRleCA2 NGY2ZDVmYjJkYjIuLjUyNTQ5NzA2N2U5MiAxMDA2NDQKLS0tIGEvTGF5b3V0VGVzdHMvaW1wb3J0 ZWQvdzNjL3dlYi1wbGF0Zm9ybS10ZXN0cy90b29scy93cHRzZXJ2ZS93cHRzZXJ2ZS9zc2x1dGls cy9vcGVuc3NsLnB5CisrKyBiL0xheW91dFRlc3RzL2ltcG9ydGVkL3czYy93ZWItcGxhdGZvcm0t dGVzdHMvdG9vbHMvd3B0c2VydmUvd3B0c2VydmUvc3NsdXRpbHMvb3BlbnNzbC5weQpAQCAtMzI0 LDcgKzMyNCw3IEBAIGNsYXNzIE9wZW5TU0xFbnZpcm9ubWVudChvYmplY3QpOgogICAgICAgICAg ICAgZW5kX2RhdGVfc3RyID0gb3BlbnNzbCgieDUwOSIsCiAgICAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICItbm9vdXQiLAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg ICAiLWVuZGRhdGUiLAotICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiLWluIiwg Y2VydF9wYXRoKS5zcGxpdCgiPSIsIDEpWzFdLnN0cmlwKCkKKyAgICAgICAgICAgICAgICAgICAg ICAgICAgICAgICAgICAgIi1pbiIsIGNlcnRfcGF0aCkuZGVjb2RlKCkuc3BsaXQoIj0iLCAxKVsx XS5zdHJpcCgpCiAgICAgICAgICAgICAjIE5vdCBzdXJlIGlmIHRoaXMgd29ya3MgaW4gb3RoZXIg bG9jYWxlcwogICAgICAgICAgICAgZW5kX2RhdGUgPSBkYXRldGltZS5zdHJwdGltZShlbmRfZGF0 ZV9zdHIsICIlYiAlZCAlSDolTTolUyAlWSAlWiIpCiAgICAgICAgICAgICB0aW1lX2J1ZmZlciA9 IHRpbWVkZWx0YSgqKkNFUlRfRVhQSVJZX0JVRkZFUikKZGlmZiAtLWdpdCBhL1Rvb2xzL1Njcmlw dHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3NlcnZlcnMvd2Vic29ja2V0X3NlcnZlci5weSBiL1Rv b2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3NlcnZlcnMvd2Vic29ja2V0X3NlcnZl ci5weQppbmRleCAyYzAxYzhkNzFjNjkuLmJhNjRkNzdhMmRiMyAxMDA2NDQKLS0tIGEvVG9vbHMv U2NyaXB0cy93ZWJraXRweS9sYXlvdXRfdGVzdHMvc2VydmVycy93ZWJzb2NrZXRfc2VydmVyLnB5 CisrKyBiL1Rvb2xzL1NjcmlwdHMvd2Via2l0cHkvbGF5b3V0X3Rlc3RzL3NlcnZlcnMvd2Vic29j a2V0X3NlcnZlci5weQpAQCAtMTIwLDggKzEyMCw4IEBAIGNsYXNzIFB5V2ViU29ja2V0KGh0dHBf c2VydmVyLkxpZ2h0dHBkKToKIAogICAgICAgICAjIEZJWE1FIGh0dHBzOi8vYnVncy53ZWJraXQu b3JnL3Nob3dfYnVnLmNnaT9pZD0yMDY1NDY6IFNob3VsZCBiZSB1c2luZyB0aGUgc2FtZSB2ZXJz aW9uIG9mIFB5dGhvbiBydW4td2Via2l0LXRlc3RzIGlzCiAgICAgICAgIHB5dGhvbl9pbnRlcnAg PSBzeXMuZXhlY3V0YWJsZQotICAgICAgICBpZiBzeXMudmVyc2lvbl9pbmZvID4gKDMsIDApOgot ICAgICAgICAgICAgcHl0aG9uX2ludGVycCA9ICdweXRob24yJworICAgICAgICAjaWYgc3lzLnZl cnNpb25faW5mbyA+ICgzLCAwKToKKyAgICAgICAgIyAgICBweXRob25faW50ZXJwID0gJ3B5dGhv bjInCiAKICAgICAgICAgd3B0X3Rvb2xzX2Jhc2UgPSBzZWxmLl9maWxlc3lzdGVtLmpvaW4oc2Vs Zi5fbGF5b3V0X3Rlc3RzLCAiaW1wb3J0ZWQiLCAidzNjIiwgIndlYi1wbGF0Zm9ybS10ZXN0cyIs ICJ0b29scyIpCiAgICAgICAgIHB5d2Vic29ja2V0X2Jhc2UgPSBzZWxmLl9maWxlc3lzdGVtLmpv aW4od3B0X3Rvb2xzX2Jhc2UsICJweXdlYnNvY2tldCIpCkBAIC0xNTcsNiArMTU3LDcgQEAgY2xh c3MgUHlXZWJTb2NrZXQoaHR0cF9zZXJ2ZXIuTGlnaHR0cGQpOgogICAgICAgICBzZXJ2ZXJfbmFt ZSA9IHNlbGYuX2ZpbGVzeXN0ZW0uYmFzZW5hbWUocHl3ZWJzb2NrZXRfc2NyaXB0KQogICAgICAg ICBzZWxmLl9lbnYgPSBzZWxmLl9wb3J0X29iai5zZXR1cF9lbnZpcm9uX2Zvcl9zZXJ2ZXIoc2Vy dmVyX25hbWUpCiAgICAgICAgIHNlbGYuX2VudlsnUFlUSE9OUEFUSCddID0gb3MucGF0aC5wYXRo c2VwLmpvaW4oW3B5d2Vic29ja2V0X2Jhc2VdICsgcHl3ZWJzb2NrZXRfZGVwcyArIFtzZWxmLl9l bnYuZ2V0KCdQWVRIT05QQVRIJywgJycpXSkKKyAgICAgICAgcHJpbnQoJ1BZVEhPTlBBVEgge30n LmZvcm1hdChzZWxmLl9lbnZbJ1BZVEhPTlBBVEgnXSkpCiAKICAgICBkZWYgX3JlbW92ZV9zdGFs ZV9sb2dzKHNlbGYpOgogICAgICAgICB0cnk6Cg== 422170 2021-03-03 19:05:28 -0800 2021-03-03 19:05:28 -0800 config.json config.json application/json 1400 don.olmstead ew0KICAgICJicm93c2VyX2hvc3QiOiAibG9jYWxob3N0IiwNCiAgICAiYWx0ZXJuYXRlX2hvc3Rz Ijogew0KICAgICAgICAiYWx0IjogIjEyNy4wLjAuMSINCiAgICB9LA0KICAgICJwb3J0cyI6IHsN CiAgICAgICAgImh0dHAiOiBbDQogICAgICAgICAgICA4ODAwLA0KICAgICAgICAgICAgODgwMQ0K ICAgICAgICBdLA0KICAgICAgICAiaHR0cHMiOiBbDQogICAgICAgICAgICA5NDQzLA0KICAgICAg ICAgICAgOTQ0NA0KICAgICAgICBdLA0KICAgICAgICAid3MiOiBbDQogICAgICAgICAgICA0OTAw MQ0KICAgICAgICBdDQogICAgfSwNCiAgICAiYWxpYXNlcyI6IFsNCiAgICAgICAgew0KICAgICAg ICAgICAgInVybC1wYXRoIjogIi9yZXNvdXJjZXMvdGVzdGhhcm5lc3NyZXBvcnQuanMiLA0KICAg ICAgICAgICAgImxvY2FsLWRpciI6ICIuLi8uLi8uLi9yZXNvdXJjZXMvIg0KICAgICAgICB9LA0K ICAgICAgICB7DQogICAgICAgICAgICAidXJsLXBhdGgiOiAiL3Jlc291cmNlcy90ZXN0aGFybmVz cy5jc3MiLA0KICAgICAgICAgICAgImxvY2FsLWRpciI6ICIuLi8uLi8uLi9yZXNvdXJjZXMvIg0K ICAgICAgICB9LA0KICAgICAgICB7DQogICAgICAgICAgICAidXJsLXBhdGgiOiAiL3dlYmtpdC10 ZXN0LXJlc291cmNlcy8iLA0KICAgICAgICAgICAgImxvY2FsLWRpciI6ICIuLi8uLi8uLi9yZXNv dXJjZXMvIg0KICAgICAgICB9LA0KICAgICAgICB7DQogICAgICAgICAgICAidXJsLXBhdGgiOiAi L1dlYktpdC8iLA0KICAgICAgICAgICAgImxvY2FsLWRpciI6ICIuLi8uLi8uLi9odHRwL3dwdC8i DQogICAgICAgIH0NCiAgICBdLA0KICAgICJjaGVja19zdWJkb21haW5zIjogZmFsc2UsDQogICAg ImxvZ19sZXZlbCI6ICJkZWJ1ZyIsDQogICAgImJpbmRfaG9zdG5hbWUiOiB0cnVlLA0KICAgICJz c2wiOiB7DQogICAgICAgICJ0eXBlIjogIm9wZW5zc2wiLA0KICAgICAgICAiZW5jcnlwdF9hZnRl cl9jb25uZWN0IjogZmFsc2UsDQogICAgICAgICJvcGVuc3NsIjogew0KICAgICAgICAgICAgIm9w ZW5zc2xfYmluYXJ5IjogIkM6XFx3ZWJraXRcXFdlYktpdExpYnJhcmllc1xcd2luXFx0b29sc1xc bGlicmVzc2xcXG9wZW5zc2wuZXhlIiwNCiAgICAgICAgICAgICJiYXNlX3BhdGgiOiAiQzpcXHdl YmtpdFxcV2ViS2l0QnVpbGRcXFJlbGVhc2VcXGJpbjY0XFxsYXlvdXQtdGVzdC1yZXN1bHRzXFxf d3B0X2NlcnRzIiwNCiAgICAgICAgICAgICJmb3JjZV9yZWdlbmVyYXRlIjogZmFsc2UNCiAgICAg ICAgfSwNCiAgICAgICAgInByZWdlbmVyYXRlZCI6IHsNCiAgICAgICAgICAgICJob3N0X2tleV9w YXRoIjogbnVsbCwNCiAgICAgICAgICAgICJob3N0X2NlcnRfcGF0aCI6IG51bGwNCiAgICAgICAg fSwNCiAgICAgICAgIm5vbmUiOiB7fQ0KICAgIH0NCn0=