Bug 258863 – REGRESSION (iOS 17 beta): "Origin" field is missing in the request when using "Link modulepreload"

• Component: Page Loading
• Status: RESOLVED FIXED

[CassielX]

Created attachment 466928 [details]
This is the error printed out by JavaScript.

In iOS 17 beta version, when using link tag + modulePreload in JavaScript, and Native enables HTTPS interceptor, if there is a cross-domain request, the "Origin" field is missing in the request header of urlSchemeTask.request in the interceptor callback, which causes the resource loading to fail.

I found the following code in the newly merged code on the master branch:
auto linkRequest = [&]() {
    if (params.relAttribute.isLinkModulePreload) {
        options.mode = FetchOptions::Mode::Cors;
        options.credentials = equalLettersIgnoringASCIICase(params.crossOrigin, "use-credentials"_s) ? FetchOptions::Credentials::Include : FetchOptions::Credentials::SameOrigin;
        CachedResourceRequest cachedRequest { ResourceRequest { url }, WTFMove(options) };
        cachedRequest.setOrigin(document.securityOrigin());
        return cachedRequest;
    }
    return createPotentialAccessControlRequest(url, WTFMove(options), document, params.crossOrigin);
}();

The code logic executed under the isLinkModulePreload condition is different from that in createPotentialAccessControlRequest. In createPotentialAccessControlRequest, the setting of options.mode = FetchOptions::Mode::Cors and cachedRequest.resourceRequest().request.setHTTPOrigin are paired, but in the new code, the setting of options.mode = FetchOptions::Mode::Cors and cachedRequest.setOrigin are set at the same time, but cachedRequest.resourceRequest().request.setHTTPOrigin is not set. I don't know if this is the cause of the problem that occurred.

Link：https://github.com/WebKit/WebKit/pull/10505/commits/2581333cdb03631d467c1538d6b7f7f0f8e1b1c8#diff-8e32b2aefbbce8573dd734958027b29d33b30c7e079da9dc9b51dc49330009a0


The issue of missing the "Origin" field in the request header of the interceptor has only been found in the iOS 17 beta version. For the same link and native environment, the "Origin" field is carried in the request of the interceptor in iOS 16.4/16.5/16.6 beta versions, and there is no problem.

[Radar WebKit Bug Importer]

<rdar://problem/111800089>

[Alex Christensen]

Exactly what version of "the iOS 17 beta" are you using?  Do you have any extensions installed?  I fixed an issue just like this in https://bugs.webkit.org/show_bug.cgi?id=256232 and I'm not sure if that fix in the version you're using.

[Alex Christensen]

Also, are you able to point me to any pages that have this error or used to have this error.  You can either post a link here or send me an email by clicking my name while logged into bugs.webkit.org

[CassielX]

(In reply to Alex Christensen from comment #3)
> Also, are you able to point me to any pages that have this error or used to
> have this error.  You can either post a link here or send me an email by
> clicking my name while logged into bugs.webkit.org

Open the test link: https://s1.meituan.net/static-prod01/com.sankuai.knb.fe.template/html/test/demo8.html, which contains a load with "Link" and "modulePreload".

Set a breakpoint in the following method in WebURLSchemeHandlerCocoa.mm file:
void WebURLSchemeHandlerCocoa::platformStartTask(WebPageProxy& page, WebURLSchemeTask& task)

Alternatively, set a breakpoint in the following method in WebLoaderStrategy.cpp file:
void WebLoaderStrategy::scheduleLoad(ResourceLoader& resourceLoader, CachedResource* resource, bool shouldClearReferrerOnHTTPSToHTTPRedirect)

Inspect the request. If the URL in the request is "https://s0.meituan.net/bs/knb/v1.9.4/knb.js", the "Origin" field will be lost.

[Alex Christensen]

I verified the Origin header field is being sent with that demo and with current WebKit.  Given that that is the exact set of circumstances that cause the bug I fixed and that it seems to be fixed, I'm going to resolve this bug as a duplicate.

What is the build number of the OS you are using?  It is found in Settings>General>About>iOS Version.  If the issue is still present in iOS 17.0 beta 3 (21A5277h) then please reopen this bug.

*** This bug has been marked as a duplicate of bug 256232 ***

[CassielX]

(In reply to Alex Christensen from comment #5)
> I verified the Origin header field is being sent with that demo and with
> current WebKit.  Given that that is the exact set of circumstances that
> cause the bug I fixed and that it seems to be fixed, I'm going to resolve
> this bug as a duplicate.
> 
> What is the build number of the OS you are using?  It is found in
> Settings>General>About>iOS Version.  If the issue is still present in iOS
> 17.0 beta 3 (21A5277h) then please reopen this bug.
> 
> *** This bug has been marked as a duplicate of bug 256232 ***


Debugging simulator iOS 17.0 beta 3 (21A5277g) using Xcode 15.0 beta 3 (15A5195k), when urlSchemeTask.request.URL=https://s0.meituan.net/bs/knb/v1.9.4/knb.js, the content of urlSchemeTask.request.allHTTPHeaderFields obtained in the callback of 
- (void)webView:(WKWebView *)webView startURLSchemeTask:(id <WKURLSchemeTask>)urlSchemeTask is as follows:

{type = mutable dict, count = 6,
entries =>
   0 : Sec-Fetch-Dest = script
   1 : User-Agent = <CFString 0x7fb1d55950c0 [0x10c3bc1d0]>{contents = "Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 TitansX/20.0.1.old KNB/1.0 iOS/17.0 meituangroup/com.meituan.titansx.TitansDemo2/1.0.0 meituangroup/1.0.0 App/10111/1.0.0 iPhone/iPhone14ProMax WKWebView"}
   2 : Accept = */*
   4 : Referer = <CFString 0x60000373ca80 [0x10c3bc1d0]>{contents = "https://s1.meituan.net/"}
   5 : Sec-Fetch-Site = same-site
   6 : Sec-Fetch-Mode = cors
}


However, when debugging on iOS 14.8 using Xcode 14.2 (14C18) in the same method, the content of urlSchemeTask.request.allHTTPHeaderFields printed is as follows:
{
   Accept = "*/*";
   Origin = "https://s1.meituan.net";
   Referer = "https://s1.meituan.net/static-prod01/com.sankuai.knb.fe.template/html/test/demo8.html";
   "User-Agent" = "Mozilla/5.0 (iPhone; CPU iPhone OS 14_8 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Mobile/15E148 TitansX/20.0.1.old KNB/1.0 iOS/14.8 meituangroup/com.meituan.titansx.TitansDemo1/1.0.0 meituangroup/1.0.0 App/10111/1.0.0 iPhone/iPhone12ProMax WKWebView";
}

As you can see, the Origin field is still missing in iOS 17 beta 3.

Is it possible that iOS 17 beta 3 has not integrated your fix code? Also, could you please advise on how to check which version of WebKit is integrated in iOS 17 beta 3?

[Alex Christensen]

WKURLSchemeTask does not support https.  How are you seeing this?

[CassielX]

(In reply to Alex Christensen from comment #7)
> WKURLSchemeTask does not support https.  How are you seeing this?

When debugging the WebKit source code, changing the following code can intercept HTTPS links:
bool URLParser::isSpecialScheme(StringView schemeArg){ 
return false; 
// return scheme(schemeArg) != Scheme::NonSpecial;
}
During my debugging process on iOS 17 beta 2, I encountered the JSError issue mentioned before. After comparing it with the official version of iOS 16.4, I found that there was a new Error added. After multiple attempts, I concluded that it is highly likely to be related to Link+ModulePreload+Cross-Origin. I also noticed that there are related commits on the master branch of WebKit. Based on debugging the WebKit source code, I found that in the scenario of Link+ModulePreload+Cross-Origin, after the resourceLoader is built and before sending the network request, the header's Origin is lost, which can be reproduced stably. Currently, I want to try to verify whether this Origin loss is the real cause of the JSError I encountered. In addition, the header information in the above scenario is inconsistent between iOS 17 beta 2, beta 3 and iOS 16.4, and it is unknown whether this issue will trigger unpredictable problems at some point or in specific scenarios.

I'm not sure if the fix you mentioned has been integrated into iOS 17 beta 3, but the problem still exists. Is there a way to see which commit has been integrated into iOS 17 beta 3?

[Alex Christensen]

There is not a good way for you to see which commit has been integrated into iOS 17 beta 3.  You are also describing something that we consciously and intentionally do not support.

However, regular WKURLSchemeHandler using regular custom schemes have the regression you describe.  I'm fixing that with a unit test.

[Alex Christensen]

Pull request: https://github.com/WebKit/WebKit/pull/15829

[CassielX]

(In reply to Alex Christensen from comment #10)
> Pull request: https://github.com/WebKit/WebKit/pull/15829

Thank you for your support. I will pay attention to whether this issue will still exist in future versions of iOS 17, and please follow up on the status of the PR merge.
Thank you very much for your response.

[EWS]

Committed 266054@main (ba2a851809a3): <https://commits.webkit.org/266054@main>

Reviewed commits have been landed. Closing PR #15829 and removing active labels.