127676 2014-01-27 01:40:13 -0800 WPT tests should run as LayoutTests from multiple non-localhost hostnames 2025-05-28 11:15:07 -0700 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=160504 https://bugs.webkit.org/show_bug.cgi?id=232223 https://bugs.webkit.org/show_bug.cgi?id=236049 InRadar P2 Normal --- 219198 256632 261038 171934 218795 147782 235994 1 youennf youennf ap cdumez darin fran fred.wang gsnedders livid marcosc mcatanzaro pgriffis simon.fraser webkit-bug-importer oldest_to_newest 972363 0 youennf 2014-01-27 01:40:13 -0800 Some W3C tests make request on dynamically computed URLs such as "http://www2."+location.host. See for instance https://github.com/w3c/web-platform-tests/blob/master/XMLHttpRequest/send-redirect-to-cors.htm This breaks the underlying test framework. First, "www2." + location.host cannot be referenced without changing /etc/hosts. Mapping should be set to 127.0.0.1 Second, "www2." + location.host will not be authorized by DumpRenderTrees as it is neither "localhost" nor 127.0.0.1 972364 1 youennf 2014-01-27 01:47:22 -0800 > First, "www2." + location.host cannot be referenced without changing /etc/hosts. Mapping should be set to 127.0.0.1 The simplest approach would be to require proper setting of /etc/hosts, for instance by adding entries for "www.web-platform.test", "www2.web-platform.test"... > Second, "www2." + location.host will not be authorized by DumpRenderTree as it is neither "localhost" nor 127.0.0.1 One solution is to authorize URLS with specific prefixes (.test or .local e.g.) Another solution is to check that the host resolves to 127.0.0.1. 1016670 2 233362 youennf 2014-06-19 08:11:59 -0700 Created attachment 233362 WIP: Enable all *.localhost URLS 1154756 3 ap 2016-01-12 10:35:45 -0800 I don't think that we should be doing this. If I were a new contributor to submit my first patch, I'd certainly refuse to work on a project that insists on modifying my /etc/hosts, or on making any other persistent changes on my machine. 1154833 4 youennf 2016-01-12 12:31:41 -0800 (In reply to comment #3) > I don't think that we should be doing this. > > If I were a new contributor to submit my first patch, I'd certainly refuse > to work on a project that insists on modifying my /etc/hosts, or on making > any other persistent changes on my machine. This is how it is suggested to be done in https://github.com/w3c/web-platform-tests. Bots can be made to work for these tests quite easily, but that does not help contributors much. Do you see any other practical alternative? I am not sure a proxy would be the right solution here. I do not know exactly how much tests require these URLs. The XHR tests are covered by other WebKit tests so this is probably ok for these tests. It would be nice though to have a bot dedicated to WPT conformance, trying to pass all tests and doing reports. At least we would know whether these tests (and other WPT) are passing or not. 1154863 5 ap 2016-01-12 13:15:46 -0800 In WebKit tests, we've been using the distinction between 127.0.0.1 and localhost, as well as http/https and multiple ports. Perhaps W3C tests could be changed to not require local setup? That seems like it would be a win for everyone. > It would be nice though to have a bot dedicated to WPT conformance That's a good idea. Do you know if there is anyone who would track the progress, and/or actively work on increasing the pass rate? 1154894 6 youennf 2016-01-12 14:22:36 -0800 (In reply to comment #5) > In WebKit tests, we've been using the distinction between 127.0.0.1 and > localhost, as well as http/https and multiple ports. Perhaps W3C tests could > be changed to not require local setup? That seems like it would be a win for > everyone. That is probably not straightforward as WPT tests are made to work both locally as well as from w3c-tests.org. > > It would be nice though to have a bot dedicated to WPT conformance > > That's a good idea. Do you know if there is anyone who would track the > progress, and/or actively work on increasing the pass rate? I am interested in that. cdumez did some work to improve the pass rate, for instance on the binding generator. I do not know the context of this activity though. 1155784 7 youennf 2016-01-15 08:55:01 -0800 > I do not know exactly how much tests require these URLs. 27 -expected.txt files exhibit a "Blocked access message". There may be other tests not running properly due to these limitations though. 1268450 8 cdumez 2017-01-20 10:04:41 -0800 (In reply to comment #3) > I don't think that we should be doing this. > > If I were a new contributor to submit my first patch, I'd certainly refuse > to work on a project that insists on modifying my /etc/hosts, or on making > any other persistent changes on my machine. Well, you don't have to as a developer but then you would have some tests failing locally. The fact is that - at the moment - anyone working on web-platform-tests has to modify their hosts file. Sure, it'd be nice if we could work with upstream to try and get them to adopt a configuration that is more convenient for us. We can certainly try this route but I have my doubts it will be successful. I have also been told that out configuration is insufficient to cover all cases the tests are actually covering. If we cannot get upstream tests to change then I think we need to deal with this downstream somehow. Right now, the solution is to skip to tests or have their baseline claiming the test fails even though it would pass if the machine was properly configured. I think we have to do better than this. I personally think having a script called "prepare-machine-for-development" or so (which would add the right entries to /etc/hosts for you) would be a better solution than what we have right now. 1268453 9 youennf 2017-01-20 10:10:13 -0800 > Sure, it'd be nice if we could work with upstream to try and get them to > adopt a configuration that is more convenient for us. We can certainly try > this route but I have my doubts it will be successful. I have also been told > that out configuration is insufficient to cover all cases the tests are > actually covering. Right, some tests require different hostnames, not just different ports. Some other tests require subdomain relationships (www.localhost/localhost for instance). 1268613 10 darin 2017-01-20 15:21:38 -0800 I think should add a feature at a low level of WebKit itself that can let us test in this fashion with development builds. It can be something that the test runners turn on and something that we can turn on with some kind of debugging menu. There must be a relatively small number of bottlenecks for URL and host name processing that we could hook into. I agree that we don’t want to make test running require changing the configuration of the computer unless it is absolutely necessary and I don’t think it is. 1276115 11 ap 2017-02-12 23:32:15 -0800 *** Bug 168220 has been marked as a duplicate of this bug. *** 1276121 12 ap 2017-02-12 23:45:45 -0800 Simon found another related quirk - in Chrome, subdomains of localhost all resolve to 127.0.0.1. This seems to have started with some weird requirements in <https://tools.ietf.org/html/rfc6761#section-6.3>. Chrome chose to follow this RFC, but no one else appears convinced. Some history in <https://bugs.chromium.org/p/chromium/issues/detail?id=489973> and <https://tools.ietf.org/html/draft-west-let-localhost-be-localhost>. 1707906 13 gsnedders 2020-11-16 11:10:07 -0800 See also: https://github.com/web-platform-tests/wpt/issues/3243 https://github.com/web-platform-tests/wpt/pull/3302 https://github.com/web-platform-tests/wpt/issues/15692 1707907 14 webkit-bug-importer 2020-11-16 11:10:18 -0800 <rdar://problem/71449599> 1709432 15 fred.wang 2020-11-20 03:06:20 -0800 I'm making this block bug 171934 ; localhost and loopback IP addresses are considered non-mixed content so being able to access other addresses for testing purpose would be useful (otherwise we need to disable the preferences when running tests). This is also causing bug 218795. 1709434 16 fred.wang 2020-11-20 03:08:36 -0800 (and incidentally supporting .localhost as done in attachment bug 233362 and the ipv6 loopback ip address as suggested in the FIXME would help for testing bug 218627 and bug 218623) 1774849 17 gsnedders 2021-07-05 14:18:28 -0700 (In reply to Darin Adler from comment #10) > I think should add a feature at a low level of WebKit itself that can let us > test in this fashion with development builds. It can be something that the > test runners turn on and something that we can turn on with some kind of > debugging menu. There must be a relatively small number of bottlenecks for > URL and host name processing that we could hook into. > > I agree that we don’t want to make test running require changing the > configuration of the computer unless it is absolutely necessary and I don’t > think it is. Note that Chrome and Firefox both have such functionality in their shipping browsers, which matters if we want to be able to run WPT tests against Safari without changing the system configuration (which is, of course, impossible on iOS on device). This does also mean that Safari is the only browser that actually requires /etc/hosts to be modified to run the tests through wptrunner. At least currently, WebKit doesn't have any way to intercept the DNS request within the network stack, and I don't know how easy that is for the different network stacks currently supported. Moving away from localhost/loopback addresses is almost certainly needed to match the expected behaviour in some tests, given localhost is special-cased in some places in the web platform. Also, there's also nonexistent.[wpt domain] that's meant to fail to resolve, which we probably need to deal with too. (I know some Gecko developers had issues with some ISPs where the DNS provider intercepted any failure to redirect to a search page, hence it _wasn't_ failing to resolve.) 1797636 18 mcatanzaro 2021-09-27 13:59:47 -0700 (In reply to Sam Sneddon [:gsnedders] from comment #17) > At least currently, WebKit doesn't have any way to intercept the DNS request > within the network stack, and I don't know how easy that is for the > different network stacks currently supported. FYI, for the libsoup backend it is quite easy. We just write a new GResolver, have it wrap the default system GResolver, and handle our special hostnames differently. It would need a WebKit internal API to enable use of the test GResolver. 1836390 19 youennf 2022-02-02 02:34:06 -0800 (In reply to Michael Catanzaro from comment #18) > (In reply to Sam Sneddon [:gsnedders] from comment #17) > > At least currently, WebKit doesn't have any way to intercept the DNS request > > within the network stack, and I don't know how easy that is for the > > different network stacks currently supported. > > FYI, for the libsoup backend it is quite easy. We just write a new > GResolver, have it wrap the default system GResolver, and handle our special > hostnames differently. It would need a WebKit internal API to enable use of > the test GResolver. It would be an interesting experiment to add the necessary logic in WebKit to make use of that GResolver. We might need something like: - Add an internal feature to toggle this on/off - Update test runner source code to allow non localhost/127.0.0.1 hosts. Is there any GTK folk interested in that? I'll be happy to help. 1836391 20 fred.wang 2022-02-02 02:39:34 -0800 Not sure that can help, but FWIW I had experimented non-loopback address for cocoa API tests on https://bugs.webkit.org/show_bug.cgi?id=235083 ; the patch seemed to work locally on my mac but not on EWS. 1884561 21 pgriffis 2022-07-16 11:04:51 -0700 Initial work on this done here: https://github.com/WebKit/WebKit/pull/2489 233362 2014-06-19 08:11:59 -0700 2022-02-02 02:32:12 -0800 WIP: Enable all *.localhost URLS bug-127676-20140619171136.patch text/plain 3148 youennf U3VidmVyc2lvbiBSZXZpc2lvbjogMTY5NjQ1CmRpZmYgLS1naXQgYS9Tb3VyY2UvV2ViS2l0Mi9T aGFyZWQvQVBJL2MvV0tTdHJpbmcuY3BwIGIvU291cmNlL1dlYktpdDIvU2hhcmVkL0FQSS9jL1dL U3RyaW5nLmNwcAppbmRleCA1ZjEzYjljYTdlMGMzOTYxNTk3NDIyZTg0M2M4MDJhYmIzZjY3Nzll Li5lMTJhZDNiMmQzOTYzZWM1OTFjYzA1NGUxN2M2MWJhOTA1MTUwOGE4IDEwMDY0NAotLS0gYS9T b3VyY2UvV2ViS2l0Mi9TaGFyZWQvQVBJL2MvV0tTdHJpbmcuY3BwCisrKyBiL1NvdXJjZS9XZWJL aXQyL1NoYXJlZC9BUEkvYy9XS1N0cmluZy5jcHAKQEAgLTgzLDYgKzgzLDExIEBAIGJvb2wgV0tT dHJpbmdJc0VxdWFsVG9VVEY4Q1N0cmluZ0lnbm9yaW5nQ2FzZShXS1N0cmluZ1JlZiBhUmVmLCBj b25zdCBjaGFyKiBiKQogICAgIHJldHVybiB0b0ltcGwoYVJlZiktPmVxdWFsVG9VVEY4U3RyaW5n SWdub3JpbmdDYXNlKGIpOwogfQogCitib29sIFdLU3RyaW5nRW5kc1dpdGgoV0tTdHJpbmdSZWYg YVJlZiwgY29uc3QgY2hhciogYikKK3sKKyAgICByZXR1cm4gdG9JbXBsKGFSZWYpLT5zdHJpbmco KS5lbmRzV2l0aChiKTsKK30KKwogV0tTdHJpbmdSZWYgV0tTdHJpbmdDcmVhdGVXaXRoSlNTdHJp bmcoSlNTdHJpbmdSZWYganNTdHJpbmdSZWYpCiB7CiAgICAgUmVmUHRyPEFQSTo6U3RyaW5nPiBh cGlTdHJpbmcgPSBBUEk6OlN0cmluZzo6Y3JlYXRlKGpzU3RyaW5nUmVmKTsKZGlmZiAtLWdpdCBh L1NvdXJjZS9XZWJLaXQyL1NoYXJlZC9BUEkvYy9XS1N0cmluZy5oIGIvU291cmNlL1dlYktpdDIv U2hhcmVkL0FQSS9jL1dLU3RyaW5nLmgKaW5kZXggZmMxYjcxZDYxODU5MmQ2OTY4Yjc0YTgwN2M3 MGU2NWUxMzNmNWViOC4uZjhjMjYzMzE3Mjk0ODJkOGZlNDE0Yjg3MWM4N2I0MTdlOTU3MTM0YiAx MDA2NDQKLS0tIGEvU291cmNlL1dlYktpdDIvU2hhcmVkL0FQSS9jL1dLU3RyaW5nLmgKKysrIGIv U291cmNlL1dlYktpdDIvU2hhcmVkL0FQSS9jL1dLU3RyaW5nLmgKQEAgLTU5LDYgKzU5LDcgQEAg V0tfRVhQT1JUIHNpemVfdCBXS1N0cmluZ0dldFVURjhDU3RyaW5nKFdLU3RyaW5nUmVmIHN0cmlu ZywgY2hhciogYnVmZmVyLCBzaXplX3QKIFdLX0VYUE9SVCBib29sIFdLU3RyaW5nSXNFcXVhbChX S1N0cmluZ1JlZiBhLCBXS1N0cmluZ1JlZiBiKTsKIFdLX0VYUE9SVCBib29sIFdLU3RyaW5nSXNF cXVhbFRvVVRGOENTdHJpbmcoV0tTdHJpbmdSZWYgYSwgY29uc3QgY2hhciogYik7CiBXS19FWFBP UlQgYm9vbCBXS1N0cmluZ0lzRXF1YWxUb1VURjhDU3RyaW5nSWdub3JpbmdDYXNlKFdLU3RyaW5n UmVmIGEsIGNvbnN0IGNoYXIqIGIpOworV0tfRVhQT1JUIGJvb2wgV0tTdHJpbmdFbmRzV2l0aChX S1N0cmluZ1JlZiBhLCBjb25zdCBjaGFyKiBiKTsKIAogI2lmZGVmIF9fY3BsdXNwbHVzCiB9CmRp ZmYgLS1naXQgYS9Ub29scy9EdW1wUmVuZGVyVHJlZS9tYWMvUmVzb3VyY2VMb2FkRGVsZWdhdGUu bW0gYi9Ub29scy9EdW1wUmVuZGVyVHJlZS9tYWMvUmVzb3VyY2VMb2FkRGVsZWdhdGUubW0KaW5k ZXggMGZkMGZhZTExZDA5ZDUxNTA4YTUzMzQzMjM1MjEwYzI5OWE1OTA2NS4uODVlMDk5Y2I5NzA3 MGM4OTA4MWE3Njg0MzZhOTU1ZDE1MjUwYjM5NyAxMDA2NDQKLS0tIGEvVG9vbHMvRHVtcFJlbmRl clRyZWUvbWFjL1Jlc291cmNlTG9hZERlbGVnYXRlLm1tCisrKyBiL1Rvb2xzL0R1bXBSZW5kZXJU cmVlL21hYy9SZXNvdXJjZUxvYWREZWxlZ2F0ZS5tbQpAQCAtMTI5LDcgKzEyOSw3IEBAIC0gKGlk KXdlYlZpZXc6IChXZWJWaWV3ICopd3YgaWRlbnRpZmllckZvckluaXRpYWxSZXF1ZXN0OiAoTlNV UkxSZXF1ZXN0ICopcmVxdWVzCiBCT09MIGlzTG9jYWxob3N0KE5TU3RyaW5nICpob3N0KQogewog ICAgIC8vIEZJWE1FOiBTdXBwb3J0IElQdjYgbG9vcGJhY2tzLgotICAgIHJldHVybiBOU09yZGVy ZWRTYW1lID09IFtob3N0IGNvbXBhcmU6QCIxMjcuMC4wLjEiXSB8fCBOU09yZGVyZWRTYW1lID09 IFtob3N0IGNhc2VJbnNlbnNpdGl2ZUNvbXBhcmU6QCJsb2NhbGhvc3QiXTsKKyAgICByZXR1cm4g TlNPcmRlcmVkU2FtZSA9PSBbaG9zdCBjb21wYXJlOkAiMTI3LjAuMC4xIl0gfHwgTlNPcmRlcmVk U2FtZSA9PSBbaG9zdCBjYXNlSW5zZW5zaXRpdmVDb21wYXJlOkAibG9jYWxob3N0Il0gfHwgTlNP cmRlcmVkU2FtZSA9PSBbW2hvc3QgbG93ZXJjYXNlU3RyaW5nXSBoYXNTdWZmaXg6QCIubG9jYWxo b3N0Il07CiB9CiAKIEJPT0wgaG9zdElzVXNlZEJ5U29tZVRlc3RzVG9HZW5lcmF0ZUVycm9yKE5T U3RyaW5nICpob3N0KQpkaWZmIC0tZ2l0IGEvVG9vbHMvV2ViS2l0VGVzdFJ1bm5lci9JbmplY3Rl ZEJ1bmRsZS9JbmplY3RlZEJ1bmRsZVBhZ2UuY3BwIGIvVG9vbHMvV2ViS2l0VGVzdFJ1bm5lci9J bmplY3RlZEJ1bmRsZS9JbmplY3RlZEJ1bmRsZVBhZ2UuY3BwCmluZGV4IGRiZmNjMGM1MmQyYjg5 M2JjM2EwN2Q3M2I3NGMyN2YwOWNkOWM5N2MuLjNiMTc0ZDAyZjM3M2QxNzQwYjY2YzQxZTFmM2M2 ZmM3NDMzOWM0MWUgMTAwNjQ0Ci0tLSBhL1Rvb2xzL1dlYktpdFRlc3RSdW5uZXIvSW5qZWN0ZWRC dW5kbGUvSW5qZWN0ZWRCdW5kbGVQYWdlLmNwcAorKysgYi9Ub29scy9XZWJLaXRUZXN0UnVubmVy L0luamVjdGVkQnVuZGxlL0luamVjdGVkQnVuZGxlUGFnZS5jcHAKQEAgLTEwNDAsNyArMTA0MCw3 IEBAIHZvaWQgSW5qZWN0ZWRCdW5kbGVQYWdlOjpkaWRJbml0aWF0ZUxvYWRGb3JSZXNvdXJjZShX S0J1bmRsZVBhZ2VSZWYgcGFnZSwgV0tCdW5kCiAKIHN0YXRpYyBpbmxpbmUgYm9vbCBpc0xvY2Fs SG9zdChXS1N0cmluZ1JlZiBob3N0KQogewotICAgIHJldHVybiBXS1N0cmluZ0lzRXF1YWxUb1VU RjhDU3RyaW5nKGhvc3QsICIxMjcuMC4wLjEiKSB8fCBXS1N0cmluZ0lzRXF1YWxUb1VURjhDU3Ry aW5nKGhvc3QsICJsb2NhbGhvc3QiKTsKKyAgICByZXR1cm4gV0tTdHJpbmdJc0VxdWFsVG9VVEY4 Q1N0cmluZyhob3N0LCAiMTI3LjAuMC4xIikgfHwgV0tTdHJpbmdJc0VxdWFsVG9VVEY4Q1N0cmlu Zyhob3N0LCAibG9jYWxob3N0IikgfHwgV0tTdHJpbmdFbmRzV2l0aChob3N0LCAiLmxvY2FsaG9z dCIpOwogfQogCiBzdGF0aWMgaW5saW5lIGJvb2wgaXNIVFRQT3JIVFRQU1NjaGVtZShXS1N0cmlu Z1JlZiBzY2hlbWUpCg==