161364 2016-08-29 17:17:24 -0700 URLParser should handle relative URLs that start with // 2016-08-31 14:27:17 -0700 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED P2 Normal --- 1 achristensen achristensen annevk oldest_to_newest 1224455 0 achristensen 2016-08-29 17:17:24 -0700 URLParser should handle relative URLs that start with // 1224460 1 287357 achristensen 2016-08-29 17:24:37 -0700 Created attachment 287357 Patch 1224461 2 achristensen 2016-08-29 17:25:32 -0700 An unspecified difference in URL behavior can be seen by executing this in all browsers: alert(new URL("//", "https://example.com/path1/path2/index.html")) 1224464 3 287357 achristensen 2016-08-29 17:27:16 -0700 Comment on attachment 287357 Patch View in context: https://bugs.webkit.org/attachment.cgi?id=287357&action=review > Source/WebCore/platform/URLParser.cpp:300 > + copyURLPartsUntil(base, URLPart::SchemeEnd); This is not in the spec, but it needs to be. Such URLs are quite common. > Source/WebCore/platform/URLParser.cpp:301 > + m_buffer.append("://"); I'm not sure if this is "correct" behavior, or if "correct" behavior exists. 1224503 4 achristensen 2016-08-29 21:08:04 -0700 The web platform tests have a similar test for file URLs: "input": "//", "base": "file:///tmp/mock/path", ... File URLs have special states in the state machine, but there are no such tests for non-file URLs. 1224521 5 annevk 2016-08-30 01:06:54 -0700 Going through the state machine of https://url.spec.whatwg.org/#concept-basic-url-parser I get: * https://url.spec.whatwg.org/#scheme-start-state "/" -> no scheme state * https://url.spec.whatwg.org/#no-scheme-state "/" -> relative state * https://url.spec.whatwg.org/#relative-state "/" consumed -> relative slash state * https://url.spec.whatwg.org/#relative-slash-state "/" consumed -> special authority ignore slashes state * https://url.spec.whatwg.org/#special-authority-ignore-slashes-state EOF -> authority state * https://url.spec.whatwg.org/#authority-state EOF -> host state * https://url.spec.whatwg.org/#host-state -> EOF and /url/ is special -> failure This seems to match Firefox, which throws. 1224659 6 achristensen 2016-08-30 12:26:02 -0700 I think we need to change the spec in this case. Copying the base url's scheme is not in the spec, but it's needed to successfully search anything on google.com. By the way, Anne, I'm collecting a bunch of feedback on the url spec. I'll let you know once I have a parser working well enough to be more sure about my requested changes. Consider this the first. 1224662 7 achristensen 2016-08-30 12:36:57 -0700 I'm going to go ahead and assume the spec will be changed to match my code and land this so I can move ahead with finding other problems. If it is changed differently, I'll change my new URLParser, which isn't used by default yet. http://trac.webkit.org/changeset/205194 1224830 8 achristensen 2016-08-30 18:52:03 -0700 (In reply to comment #7) > I'm going to go ahead and assume the spec will be changed to match my code > and land this so I can move ahead with finding other problems. If it is > changed differently, I'll change my new URLParser, which isn't used by > default yet. This could be interpreted in a way that is antagonizing. This isn't in harmony with the cooperative nature of open web standards, and that was not the intent. My work so far is experimental, and I hope to contribute positively towards tests, standards, and implementations. 1224989 9 annevk 2016-08-31 10:30:12 -0700 (In reply to comment #6) > I think we need to change the spec in this case. Copying the base url's > scheme is not in the spec, but it's needed to successfully search anything > on google.com. That happens first thing in https://url.spec.whatwg.org/#relative-state, no? > By the way, Anne, I'm collecting a bunch of feedback on the url spec. I'll > let you know once I have a parser working well enough to be more sure about > my requested changes. Consider this the first. Sounds great, looking forward to it. Please do file them as issues at https://github.com/whatwg/url/issues/new so they don't get lost. Really pleased to see some work in this area. Note https://bugzilla.mozilla.org/show_bug.cgi?id=1064700#c7 for changes Firefox is considering relative to the specification, that would move Firefox away from the specification & Safari, but closer to Chrome & Edge. The change seems relatively sensible (more normalization) so I'm hoping you all would be on board with that too. 1225087 10 achristensen 2016-08-31 14:27:17 -0700 (In reply to comment #9) > (In reply to comment #6) > > I think we need to change the spec in this case. Copying the base url's > > scheme is not in the spec, but it's needed to successfully search anything > > on google.com. > > That happens first thing in https://url.spec.whatwg.org/#relative-state, no? Yep, I had overlooked that and done this in a different place, which had a similar effect for the few test cases I'm using so far. > > > > By the way, Anne, I'm collecting a bunch of feedback on the url spec. I'll > > let you know once I have a parser working well enough to be more sure about > > my requested changes. Consider this the first. > > Sounds great, looking forward to it. Please do file them as issues at > https://github.com/whatwg/url/issues/new so they don't get lost. Really > pleased to see some work in this area. > > Note https://bugzilla.mozilla.org/show_bug.cgi?id=1064700#c7 for changes > Firefox is considering relative to the specification, that would move > Firefox away from the specification & Safari, but closer to Chrome & Edge. > The change seems relatively sensible (more normalization) so I'm hoping you > all would be on board with that too. I think it's a little too early for me to respond to that Firefox change right now. 287357 2016-08-29 17:24:37 -0700 2016-08-30 12:16:19 -0700 Patch bug-161364-20160829172226.patch text/plain 5519 achristensen SW5kZXg6IFNvdXJjZS9XZWJDb3JlL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBTb3VyY2UvV2Vi Q29yZS9DaGFuZ2VMb2cJKHJldmlzaW9uIDIwNTE2MCkKKysrIFNvdXJjZS9XZWJDb3JlL0NoYW5n ZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwzICsxLDE1IEBACisyMDE2LTA4LTI5ICBBbGV4IENo cmlzdGVuc2VuICA8YWNocmlzdGVuc2VuQHdlYmtpdC5vcmc+CisKKyAgICAgICAgVVJMUGFyc2Vy IHNob3VsZCBoYW5kbGUgcmVsYXRpdmUgVVJMcyB0aGF0IHN0YXJ0IHdpdGggLy8KKyAgICAgICAg aHR0cHM6Ly9idWdzLndlYmtpdC5vcmcvc2hvd19idWcuY2dpP2lkPTE2MTM2NAorCisgICAgICAg IFJldmlld2VkIGJ5IE5PQk9EWSAoT09QUyEpLgorCisgICAgICAgIENvdmVyZWQgYnkgYW4gQVBJ IHRlc3QuCisKKyAgICAgICAgKiBwbGF0Zm9ybS9VUkxQYXJzZXIuY3BwOgorICAgICAgICAoV2Vi Q29yZTo6VVJMUGFyc2VyOjpwYXJzZSk6CisKIDIwMTYtMDgtMjkgIENocmlzIER1bWV6ICA8Y2R1 bWV6QGFwcGxlLmNvbT4KIAogICAgICAgICBSZWdyZXNzaW9uKHIyMDQ5MjMpOiBJdCBzaG91bGQg YmUgcG9zc2libGUgdG8gc2V0ICdMb2NhdGlvbi5ocmVmJyBjcm9zcyBvcmlnaW4KSW5kZXg6IFNv dXJjZS9XZWJDb3JlL3BsYXRmb3JtL1VSTFBhcnNlci5jcHAKPT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gU291cmNl L1dlYkNvcmUvcGxhdGZvcm0vVVJMUGFyc2VyLmNwcAkocmV2aXNpb24gMjA1MTQ3KQorKysgU291 cmNlL1dlYkNvcmUvcGxhdGZvcm0vVVJMUGFyc2VyLmNwcAkod29ya2luZyBjb3B5KQpAQCAtMjk3 LDYgKzI5Nyw4IEBAIFVSTCBVUkxQYXJzZXI6OnBhcnNlKGNvbnN0IFN0cmluZyYgaW5wdXQKICAg ICAgICAgICAgIExPR19TVEFURSgiUmVsYXRpdmVTbGFzaCIpOwogICAgICAgICAgICAgaWYgKCpj ID09ICcvJyB8fCAqYyA9PSAnXFwnKSB7CiAgICAgICAgICAgICAgICAgKytjOworICAgICAgICAg ICAgICAgIGNvcHlVUkxQYXJ0c1VudGlsKGJhc2UsIFVSTFBhcnQ6OlNjaGVtZUVuZCk7CisgICAg ICAgICAgICAgICAgbV9idWZmZXIuYXBwZW5kKCI6Ly8iKTsKICAgICAgICAgICAgICAgICBzdGF0 ZSA9IFN0YXRlOjpTcGVjaWFsQXV0aG9yaXR5SWdub3JlU2xhc2hlczsKICAgICAgICAgICAgIH0g ZWxzZSB7CiAgICAgICAgICAgICAgICAgY29weVVSTFBhcnRzVW50aWwoYmFzZSwgVVJMUGFydDo6 UG9ydEVuZCk7CkluZGV4OiBUb29scy9DaGFuZ2VMb2cKPT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvQ2hh bmdlTG9nCShyZXZpc2lvbiAyMDUxNjEpCisrKyBUb29scy9DaGFuZ2VMb2cJKHdvcmtpbmcgY29w eSkKQEAgLTEsMyArMSwxNCBAQAorMjAxNi0wOC0yOSAgQWxleCBDaHJpc3RlbnNlbiAgPGFjaHJp c3RlbnNlbkB3ZWJraXQub3JnPgorCisgICAgICAgIFVSTFBhcnNlciBzaG91bGQgaGFuZGxlIHJl bGF0aXZlIFVSTHMgdGhhdCBzdGFydCB3aXRoIC8vCisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJr aXQub3JnL3Nob3dfYnVnLmNnaT9pZD0xNjEzNjQKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JP RFkgKE9PUFMhKS4KKworICAgICAgICAqIFRlc3RXZWJLaXRBUEkvVGVzdHMvV2ViQ29yZS9VUkxQ YXJzZXIuY3BwOgorICAgICAgICAoVGVzdFdlYktpdEFQSTo6Y2hlY2tSZWxhdGl2ZVVSTCk6Cisg ICAgICAgIChUZXN0V2ViS2l0QVBJOjpURVNUX0YpOgorCiAyMDE2LTA4LTI5ICBBYWthc2ggSmFp biAgPGFha2FzaF9qYWluQGFwcGxlLmNvbT4KIAogICAgICAgICBFV1MgcGF0Y2ggc3RhdHVzIHBh Z2Ugc2hvdWxkIGluZGljYXRlIGJvdCBjb3JyZXNwb25kaW5nIHRvIGVhY2ggc3RhdHVzIG1lc3Nh Z2UKSW5kZXg6IFRvb2xzL1Rlc3RXZWJLaXRBUEkvVGVzdHMvV2ViQ29yZS9VUkxQYXJzZXIuY3Bw Cj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT0KLS0tIFRvb2xzL1Rlc3RXZWJLaXRBUEkvVGVzdHMvV2ViQ29yZS9VUkxQYXJz ZXIuY3BwCShyZXZpc2lvbiAyMDUxNDcpCisrKyBUb29scy9UZXN0V2ViS2l0QVBJL1Rlc3RzL1dl YkNvcmUvVVJMUGFyc2VyLmNwcAkod29ya2luZyBjb3B5KQpAQCAtMTMyLDYgKzEzMiw3IEBAIFRF U1RfRihVUkxQYXJzZXJUZXN0LCBQYXJzZVJlbGF0aXZlKQogICAgIGNoZWNrUmVsYXRpdmVVUkwo Ii9pbmRleC5odG1sIiwgImh0dHA6Ly93ZWJraXQub3JnL3BhdGgxL3BhdGgyLyIsIHsiaHR0cCIs ICIiLCAiIiwgIndlYmtpdC5vcmciLCAwLCAiL2luZGV4Lmh0bWwiLCAiIiwgIiIsICJodHRwOi8v d2Via2l0Lm9yZy9pbmRleC5odG1sIn0pOwogICAgIGNoZWNrUmVsYXRpdmVVUkwoImh0dHA6Ly93 aGF0d2cub3JnL2luZGV4Lmh0bWwiLCAiaHR0cDovL3dlYmtpdC5vcmcvcGF0aDEvcGF0aDIvIiwg eyJodHRwIiwgIiIsICIiLCAid2hhdHdnLm9yZyIsIDAsICIvaW5kZXguaHRtbCIsICIiLCAiIiwg Imh0dHA6Ly93aGF0d2cub3JnL2luZGV4Lmh0bWwifSk7CiAgICAgY2hlY2tSZWxhdGl2ZVVSTCgi aW5kZXguaHRtbCIsICJodHRwOi8vd2Via2l0Lm9yZy9wYXRoMS9wYXRoMi9wYWdlLmh0bWw/cXVl cnkjZnJhZ21lbnQiLCB7Imh0dHAiLCAiIiwgIiIsICJ3ZWJraXQub3JnIiwgMCwgIi9wYXRoMS9w YXRoMi9pbmRleC5odG1sIiwgIiIsICIiLCAiaHR0cDovL3dlYmtpdC5vcmcvcGF0aDEvcGF0aDIv aW5kZXguaHRtbCJ9KTsKKyAgICBjaGVja1JlbGF0aXZlVVJMKCIvL3doYXR3Zy5vcmcvaW5kZXgu aHRtbCIsICJodHRwczovL3d3dy53ZWJraXQub3JnL3BhdGgiLCB7Imh0dHBzIiwgIiIsICIiLCAi d2hhdHdnLm9yZyIsIDAsICIvaW5kZXguaHRtbCIsICIiLCAiIiwgImh0dHBzOi8vd2hhdHdnLm9y Zy9pbmRleC5odG1sIn0pOwogfQogCiBzdGF0aWMgdm9pZCBjaGVja1VSTERpZmZlcmVuY2VzKGNv bnN0IFN0cmluZyYgdXJsU3RyaW5nLCBjb25zdCBFeHBlY3RlZFBhcnRzJiBwYXJ0c05ldywgY29u c3QgRXhwZWN0ZWRQYXJ0cyYgcGFydHNPbGQpCkBAIC0xNjIsNiArMTYzLDM3IEBAIHN0YXRpYyB2 b2lkIGNoZWNrVVJMRGlmZmVyZW5jZXMoY29uc3QgU3QKICAgICBFWFBFQ1RfRkFMU0UoVVJMUGFy c2VyOjphbGxWYWx1ZXNFcXVhbCh1cmwsIG9sZFVSTCkpOwogfQogCitzdGF0aWMgdm9pZCBjaGVj a1JlbGF0aXZlVVJMRGlmZmVyZW5jZXMoY29uc3QgU3RyaW5nJiB1cmxTdHJpbmcsIGNvbnN0IFN0 cmluZyYgYmFzZVVSTFN0cmluZywgY29uc3QgRXhwZWN0ZWRQYXJ0cyYgcGFydHNOZXcsIGNvbnN0 IEV4cGVjdGVkUGFydHMmIHBhcnRzT2xkKQoreworICAgIFVSTFBhcnNlciBiYXNlUGFyc2VyOwor ICAgIGF1dG8gYmFzZSA9IGJhc2VQYXJzZXIucGFyc2UoYmFzZVVSTFN0cmluZyk7CisgICAgCisg ICAgVVJMUGFyc2VyIHBhcnNlcjsKKyAgICBhdXRvIHVybCA9IHBhcnNlci5wYXJzZSh1cmxTdHJp bmcsIGJhc2UpOworICAgIEVYUEVDVF9UUlVFKGVxKHBhcnRzTmV3LnByb3RvY29sLCB1cmwucHJv dG9jb2woKSkpOworICAgIEVYUEVDVF9UUlVFKGVxKHBhcnRzTmV3LnVzZXIsIHVybC51c2VyKCkp KTsKKyAgICBFWFBFQ1RfVFJVRShlcShwYXJ0c05ldy5wYXNzd29yZCwgdXJsLnBhc3MoKSkpOwor ICAgIEVYUEVDVF9UUlVFKGVxKHBhcnRzTmV3Lmhvc3QsIHVybC5ob3N0KCkpKTsKKyAgICBFWFBF Q1RfRVEocGFydHNOZXcucG9ydCwgdXJsLnBvcnQoKSk7CisgICAgRVhQRUNUX1RSVUUoZXEocGFy dHNOZXcucGF0aCwgdXJsLnBhdGgoKSkpOworICAgIEVYUEVDVF9UUlVFKGVxKHBhcnRzTmV3LnF1 ZXJ5LCB1cmwucXVlcnkoKSkpOworICAgIEVYUEVDVF9UUlVFKGVxKHBhcnRzTmV3LmZyYWdtZW50 LCB1cmwuZnJhZ21lbnRJZGVudGlmaWVyKCkpKTsKKyAgICBFWFBFQ1RfVFJVRShlcShwYXJ0c05l dy5zdHJpbmcsIHVybC5zdHJpbmcoKSkpOworICAgIAorICAgIGF1dG8gb2xkVVJMID0gVVJMKFVS TChVUkwoKSwgYmFzZVVSTFN0cmluZyksIHVybFN0cmluZyk7CisgICAgRVhQRUNUX1RSVUUoZXEo cGFydHNPbGQucHJvdG9jb2wsIG9sZFVSTC5wcm90b2NvbCgpKSk7CisgICAgRVhQRUNUX1RSVUUo ZXEocGFydHNPbGQudXNlciwgb2xkVVJMLnVzZXIoKSkpOworICAgIEVYUEVDVF9UUlVFKGVxKHBh cnRzT2xkLnBhc3N3b3JkLCBvbGRVUkwucGFzcygpKSk7CisgICAgRVhQRUNUX1RSVUUoZXEocGFy dHNPbGQuaG9zdCwgb2xkVVJMLmhvc3QoKSkpOworICAgIEVYUEVDVF9FUShwYXJ0c09sZC5wb3J0 LCBvbGRVUkwucG9ydCgpKTsKKyAgICBFWFBFQ1RfVFJVRShlcShwYXJ0c09sZC5wYXRoLCBvbGRV UkwucGF0aCgpKSk7CisgICAgRVhQRUNUX1RSVUUoZXEocGFydHNPbGQucXVlcnksIG9sZFVSTC5x dWVyeSgpKSk7CisgICAgRVhQRUNUX1RSVUUoZXEocGFydHNPbGQuZnJhZ21lbnQsIG9sZFVSTC5m cmFnbWVudElkZW50aWZpZXIoKSkpOworICAgIEVYUEVDVF9UUlVFKGVxKHBhcnRzT2xkLnN0cmlu Zywgb2xkVVJMLnN0cmluZygpKSk7CisgICAgCisgICAgRVhQRUNUX0ZBTFNFKFVSTFBhcnNlcjo6 YWxsVmFsdWVzRXF1YWwodXJsLCBvbGRVUkwpKTsKK30KKwogVEVTVF9GKFVSTFBhcnNlclRlc3Qs IFBhcnNlckRpZmZlcmVuY2VzKQogewogICAgIGNoZWNrVVJMRGlmZmVyZW5jZXMoImh0dHA6Ly8x MjcuMC4xIiwKQEAgLTE3MCw2ICsyMDIsMTIgQEAgVEVTVF9GKFVSTFBhcnNlclRlc3QsIFBhcnNl ckRpZmZlcmVuY2VzKQogICAgIGNoZWNrVVJMRGlmZmVyZW5jZXMoImh0dHA6Ly8wMTEuMTEuMFgx MS4weDAxMSIsCiAgICAgICAgIHsiaHR0cCIsICIiLCAiIiwgIjkuMTEuMTcuMTciLCAwLCAiLyIs ICIiLCAiIiwgImh0dHA6Ly85LjExLjE3LjE3LyJ9LAogICAgICAgICB7Imh0dHAiLCAiIiwgIiIs ICIwMTEuMTEuMHgxMS4weDAxMSIsIDAsICIvIiwgIiIsICIiLCAiaHR0cDovLzAxMS4xMS4weDEx LjB4MDExLyJ9KTsKKworICAgIC8vIEZJWE1FOiBUaGlzIGJlaGF2aW9yIG91Z2h0IHRvIGJlIHNw ZWNpZmllZCBpbiB0aGUgc3RhbmRhcmQuCisgICAgLy8gV2l0aCB0aGUgZXhpc3RpbmcgVVJMOjpw YXJzZSwgV2ViS2l0IHJldHVybnMgImh0dHBzOi8iLCBGaXJlZm94IHJldHVybnMgImh0dHBzOi8v LyIsIGFuZCBDaHJvbWUgdGhyb3dzIGFuIGVycm9yLgorICAgIGNoZWNrUmVsYXRpdmVVUkxEaWZm ZXJlbmNlcygiLy8iLCAiaHR0cHM6Ly93d3cud2Via2l0Lm9yZy9wYXRoIiwKKyAgICAgICAgeyJo dHRwcyIsICIiLCAiIiwgIiIsIDAsICIiLCAiIiwgIiIsICJodHRwczovLyJ9LAorICAgICAgICB7 Imh0dHBzIiwgIiIsICIiLCAiIiwgMCwgIi8iLCAiIiwgIiIsICJodHRwczovIn0pOwogfQogCiBz dGF0aWMgdm9pZCBzaG91bGRGYWlsKGNvbnN0IFN0cmluZyYgdXJsU3RyaW5nKQo=