Created attachment 364402[details]
Archive of layout-test-results from ews101 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews101 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 364415[details]
Archive of layout-test-results from ews114 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 364419[details]
Archive of layout-test-results from ews206 for win-future
The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews206 Port: win-future Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Created attachment 364704[details]
Archive of layout-test-results from ews102 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 364708[details]
Archive of layout-test-results from ews105 for mac-highsierra-wk2
The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews105 Port: mac-highsierra-wk2 Platform: Mac OS X 10.13.6
Created attachment 364712[details]
Archive of layout-test-results from ews116 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 364725[details]
Archive of layout-test-results from ews201 for win-future
The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews201 Port: win-future Platform: CYGWIN_NT-6.1-2.10.0-0.325-5-3-x86_64-64bit
Created attachment 364761[details]
Archive of layout-test-results from ews122 for ios-simulator-wk2
The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews122 Port: ios-simulator-wk2 Platform: Mac OS X 10.13.6
Created attachment 365023[details]
Archive of layout-test-results from ews102 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 365024[details]
Archive of layout-test-results from ews105 for mac-highsierra-wk2
The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews105 Port: mac-highsierra-wk2 Platform: Mac OS X 10.13.6
Created attachment 365030[details]
Archive of layout-test-results from ews114 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 365045[details]
Archive of layout-test-results from ews101 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews101 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 365058[details]
Archive of layout-test-results from ews114 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 365101[details]
Archive of layout-test-results from ews204 for win-future
The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews204 Port: win-future Platform: CYGWIN_NT-6.1-2.10.0-0.325-5-3-x86_64-64bit
Created attachment 365183[details]
Archive of layout-test-results from ews103 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 365194[details]
Archive of layout-test-results from ews201 for win-future
The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews201 Port: win-future Platform: CYGWIN_NT-6.1-2.10.0-0.325-5-3-x86_64-64bit
Created attachment 365198[details]
Archive of layout-test-results from ews112 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews112 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 365235[details]
Archive of layout-test-results from ews103 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 365248[details]
Archive of layout-test-results from ews104 for mac-highsierra-wk2
The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104 Port: mac-highsierra-wk2 Platform: Mac OS X 10.13.6
Created attachment 365253[details]
Archive of layout-test-results from ews115 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews115 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 365265[details]
Archive of layout-test-results from ews202 for win-future
The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews202 Port: win-future Platform: CYGWIN_NT-6.1-2.10.0-0.325-5-3-x86_64-64bit
Created attachment 365340[details]
Archive of layout-test-results from ews200 for win-future
The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews200 Port: win-future Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Created attachment 365342[details]
Archive of layout-test-results from ews107 for mac-highsierra-wk2
The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107 Port: mac-highsierra-wk2 Platform: Mac OS X 10.13.6
Created attachment 365460[details]
Archive of layout-test-results from ews206 for win-future
The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews206 Port: win-future Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Comment on attachment 365401[details]
Patch
This goes in the right direction.
Some suggestions below.
We probably want to solve properly the case of being prefetched resources that match navigation loads.
View in context: https://bugs.webkit.org/attachment.cgi?id=365401&action=review> Source/WebCore/loader/LinkLoader.cpp:279
> ResourceLoaderOptions options = CachedResourceLoader::defaultCachedResourceOptions();
I think we want the load to not include any credential.
Also, we might want to NOT follow redirections and skip service workers.
Tests might be useful there.
> Source/WebCore/loader/LinkLoader.cpp:281
> + options.certificateInfoPolicy = CertificateInfoPolicy::IncludeCertificateInfo;
Is it needed because this might fail some cache checks in NetworkProcess?
> Source/WebCore/loader/cache/CachedResourceLoader.cpp:808
> + if (frame() && frame()->page() && m_documentLoader && type != CachedResource::Type::LinkPrefetch) {
Why should we skip content extension rules for prefetch loads?
> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:215
> + if (auto entry = PrefetchCache::singleton().take(request.url(), sessionID())) {
The prefetchCache should probably be owned by NetworkProcess instead of being a singleton.
NetworkResourceLoader can access it through its connection
> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:217
> + m_cache->store(request, entry->response(), WTFMove(buffer), nullptr);
Maybe we should have entry->releaseBuffer() instead of entry->buffer() and then WTFMove.
> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:767
> + auto request = originalRequest();
Why do we need this copy?
> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:768
> + if (request.httpHeaderField(HTTPHeaderName::Purpose) == "prefetch" && (!m_parameters.sourceOrigin || !m_parameters.sourceOrigin->canRequest(request.url()))) {
I think there should always be a m_parameters.sourceOrigin for prefetch resources.
> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:770
> + return;
This sounds good as a first approach.
If the navigation load kicks in before the prefetch is finished, the prefetch will not be used.
Can we improve this?
> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:36
> + : m_response(response), m_buffer(WTFMove(buffer))
We could take a ResourceResponse&&
> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:47
> + : m_expirationTimer(*this, &PrefetchCache::expiration)
We could have a better name for expiration.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:96
> + ASSERT(resources->contains(requestUrl));
Is this guaranteed if we take the requestUrl in between?
> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:74
> + typedef Vector<std::tuple<PAL::SessionID, URL, WallTime>> SessionPrefetchExpirationList;
Could be a Deque?
Comment on attachment 365401[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=365401&action=review>> Source/WebCore/loader/LinkLoader.cpp:279
>> ResourceLoaderOptions options = CachedResourceLoader::defaultCachedResourceOptions();
>
> I think we want the load to not include any credential.
> Also, we might want to NOT follow redirections and skip service workers.
> Tests might be useful there.
Ah, I remember now the discussion on the github issue. Should be fixed.
>> Source/WebCore/loader/LinkLoader.cpp:281
>> + options.certificateInfoPolicy = CertificateInfoPolicy::IncludeCertificateInfo;
>
> Is it needed because this might fail some cache checks in NetworkProcess?
Correct, this hits "retrieveCacheEntry: Resource does not have required certificate" in NetworkResourceLoader.cpp.
>> Source/WebCore/loader/cache/CachedResourceLoader.cpp:808
>> + if (frame() && frame()->page() && m_documentLoader && type != CachedResource::Type::LinkPrefetch) {
>
> Why should we skip content extension rules for prefetch loads?
We hit an ASSERT in toResourceType in Debug, even without my patch, on prefetches. My thinking was that content extension would be done on the actual navigate anyway.
>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:215
>> + if (auto entry = PrefetchCache::singleton().take(request.url(), sessionID())) {
>
> The prefetchCache should probably be owned by NetworkProcess instead of being a singleton.
> NetworkResourceLoader can access it through its connection
Done.
>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:217
>> + m_cache->store(request, entry->response(), WTFMove(buffer), nullptr);
>
> Maybe we should have entry->releaseBuffer() instead of entry->buffer() and then WTFMove.
Done.
>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:767
>> + auto request = originalRequest();
>
> Why do we need this copy?
It is was not my intention to make a copy, sorry! Fixed.
>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:768
>> + if (request.httpHeaderField(HTTPHeaderName::Purpose) == "prefetch" && (!m_parameters.sourceOrigin || !m_parameters.sourceOrigin->canRequest(request.url()))) {
>
> I think there should always be a m_parameters.sourceOrigin for prefetch resources.
Ok I removed the check.
>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:770
>> + return;
>
> This sounds good as a first approach.
> If the navigation load kicks in before the prefetch is finished, the prefetch will not be used.
> Can we improve this?
It may be possible if we track prefetches as soon as possible, i.e when the request is sent. Then when the navigation load is finishing up, there could be a check to see if any prefetch is pending and it could be cancelled. I will try tomorrow, but not too sure about actually testing this.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:36
>> + : m_response(response), m_buffer(WTFMove(buffer))
>
> We could take a ResourceResponse&&
Done.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:47
>> + : m_expirationTimer(*this, &PrefetchCache::expiration)
>
> We could have a better name for expiration.
I am still thinking :) I'll probably have a better name tomorrow.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:96
>> + ASSERT(resources->contains(requestUrl));
>
> Is this guaranteed if we take the requestUrl in between?
I think it is okay because of how PrefetchCache::take works. But I am fine with making these if checks, to be on the safe side.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:74
>> + typedef Vector<std::tuple<PAL::SessionID, URL, WallTime>> SessionPrefetchExpirationList;
>
> Could be a Deque?
Done.
Created attachment 365591[details]
Archive of layout-test-results from ews200 for win-future
The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews200 Port: win-future Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Created attachment 365649[details]
Archive of layout-test-results from ews104 for mac-highsierra-wk2
The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104 Port: mac-highsierra-wk2 Platform: Mac OS X 10.13.6
(In reply to Yoav Weiss from comment #65)
> Comment on attachment 365562[details]
> Patch
>
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=365562&action=review
>
> > Source/WebCore/loader/LinkLoader.cpp:284
> > + options.mode = FetchOptions::Mode::Cors;
>
> The current spec PR[1] doesn't automagically set CORS mode, credentials mode
> and referrer policy, but verifies that they are properly set (e.g. via
> crossorigin and referrerpolicy attributes)
Regarding Omit mode, that's not what the spec PR currently says, so I wonder if this is required (it would render some same-origin prefetch use-cases rather useless, e.g. prefetching the next step in a shopping cart funnel).
If it is required, we would need to add a `crossorigin` "omit" value, so that it can be set with an attribute.
>
> [1]
> https://whatpr.org/html/4115/e32a6f8...e4193e2/semantics.html#process-the-
> linked-resource
>
> > Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:55
> > +static const Seconds expirationTimeout { 5_s };
>
> 5 seconds expiration seems rather low
Comment on attachment 365562[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=365562&action=review>>> Source/WebCore/loader/LinkLoader.cpp:284
>>> + options.mode = FetchOptions::Mode::Cors;
>>
>> The current spec PR[1] doesn't automagically set CORS mode, credentials mode and referrer policy, but verifies that they are properly set (e.g. via crossorigin and referrerpolicy attributes)
>>
>> [1] https://whatpr.org/html/4115/e32a6f8...e4193e2/semantics.html#process-the-linked-resource
>
> Regarding Omit mode, that's not what the spec PR currently says, so I wonder if this is required (it would render some same-origin prefetch use-cases rather useless, e.g. prefetching the next step in a shopping cart funnel).
>
> If it is required, we would need to add a `crossorigin` "omit" value, so that it can be set with an attribute.
So in general here we have to set up decent default values that work for prefetches. Not sure if adding a `crossorigin` should be part of this ppatch.
>>> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:55
>>> +static const Seconds expirationTimeout { 5_s };
>>
>> 5 seconds expiration seems rather low
>
>
It is certainly up for discussion :) I had in the back of my mind, I would like to write a test for the timeout, but with a large timeout, it becomes a not so attractive test. But apart from that practical problem, of course we can increase it.
Created attachment 365700[details]
Archive of layout-test-results from ews104 for mac-highsierra-wk2
The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104 Port: mac-highsierra-wk2 Platform: Mac OS X 10.13.6
Created attachment 365705[details]
Archive of layout-test-results from ews202 for win-future
The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews202 Port: win-future Platform: CYGWIN_NT-6.1-2.10.0-0.325-5-3-x86_64-64bit
(In reply to Rob Buis from comment #68)
> Comment on attachment 365562[details]
> Patch
>
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=365562&action=review
>
> >>> Source/WebCore/loader/LinkLoader.cpp:284
> >>> + options.mode = FetchOptions::Mode::Cors;
> >>
> >> The current spec PR[1] doesn't automagically set CORS mode, credentials mode and referrer policy, but verifies that they are properly set (e.g. via crossorigin and referrerpolicy attributes)
> >>
> >> [1] https://whatpr.org/html/4115/e32a6f8...e4193e2/semantics.html#process-the-linked-resource
> >
> > Regarding Omit mode, that's not what the spec PR currently says, so I wonder if this is required (it would render some same-origin prefetch use-cases rather useless, e.g. prefetching the next step in a shopping cart funnel).
> >
> > If it is required, we would need to add a `crossorigin` "omit" value, so that it can be set with an attribute.
>
> So in general here we have to set up decent default values that work for
> prefetches. Not sure if adding a `crossorigin` should be part of this ppatch.
Note that for preload, we have avoided setting "magical" values (e.g. for "font" destinations), and left that as extra work (but more predictable) to the developer. I'm open to discuss that decision though. Mind commenting on the PR[1], saying that you'd prefer setting the values, rather than checking for them.
Youenn - thoughts on the above would be appreciated as well
[1] https://github.com/whatwg/html/pull/4115>
> >>> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:55
> >>> +static const Seconds expirationTimeout { 5_s };
> >>
> >> 5 seconds expiration seems rather low
> >
> >
>
> It is certainly up for discussion :) I had in the back of my mind, I would
> like to write a test for the timeout, but with a large timeout, it becomes a
> not so attractive test. But apart from that practical problem, of course we
> can increase it.
Comment on attachment 365697[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=365697&action=review> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:35
> +PrefetchEntry::PrefetchEntry(WebCore::ResourceResponse&& response, RefPtr<WebCore::SharedBuffer>&& buffer)
We need to clear this cache upon receiving memory pressure.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:87
> + std::tie(sessionID, requestUrl, timestamp) = m_sessionExpirationList.first();
Use .random()
It looks like it's unsound to prefetch cross-site because the content can be read with Spectre class of attacks. We'd have to solve that problem before we can ever enable this feature.
> We need to clear this cache upon receiving memory pressure.
Good point.
(In reply to Ryosuke Niwa from comment #75)
> It looks like it's unsound to prefetch cross-site because the content can be
> read with Spectre class of attacks. We'd have to solve that problem before
> we can ever enable this feature.
The prefetch cache is in NetworkProcess.
The plan is to only use it for now for navigations so only the right process should get the response.
Should we extend prefetch to other loads (I dislike this though), we should apply the regular sanitization including CORS and CORP as done for other resources coming from network or from network cache.
We might also want to limit prefetches for now, maybe with something like:
- third-party iframes cannot trigger prefetches
- Number of prefetches of a given page is limited (to 1 maybe?)
I would add a FIXME in LinkLoader::prefetchIfNeeded about this.
SameOrigin for credentials is probably ok.
CORS mode does not make a lot of sense in that context.
We actually do not want to trigger CORS or CORP checks on the response since the target is navigation loads.
FetchOptions::Mode::Navigate makes some sense but has some implications with regards to our implementation. Probably that is the one making most sense currently since we will treat it as if it is a top level navigation.
I wonder how/whether we want want load/error events.
Should load event be fired when load starts or when load finishes? Can the timing of the load event leak some information if done at load finish?
If load fails, should error event be sent?
If we want to support load/error events when load finishes, we might need to provide an opaque response to the web process. This might be best handled by a new mode, so that NetworkLoadChecker does the necessary sanitization. And we do not want the body to be transmitted in any case.
One safe way might be to fire the load event whenever the load starts and the error event if the load is not even started.
Ideally, we would then directly return didReceiveResponse/didFinishLoading.
But we will then need to handle the case of loads that last longer than what the WebProcess thinks they do.
(In reply to youenn fablet from comment #76)
> > We need to clear this cache upon receiving memory pressure.
>
> Good point.
Agreed, I added code for this in the latest patch. Note that I am not sure how to test it though.
(In reply to youenn fablet from comment #77)
> We might also want to limit prefetches for now, maybe with something like:
> - third-party iframes cannot trigger prefetches
> - Number of prefetches of a given page is limited (to 1 maybe?)
> I would add a FIXME in LinkLoader::prefetchIfNeeded about this.
I added the FIXME.
> SameOrigin for credentials is probably ok.
Using SameOrigin now.
> CORS mode does not make a lot of sense in that context.
> We actually do not want to trigger CORS or CORP checks on the response since
> the target is navigation loads.
> FetchOptions::Mode::Navigate makes some sense but has some implications with
> regards to our implementation. Probably that is the one making most sense
> currently since we will treat it as if it is a top level navigation.
Using FetchOptions::Mode::Navigate now.
(In reply to Rob Buis from comment #80)
> (In reply to youenn fablet from comment #76)
> > > We need to clear this cache upon receiving memory pressure.
> >
> > Good point.
>
> Agreed, I added code for this in the latest patch. Note that I am not sure
> how to test it though.
We might need a testRunner API to trigger memory pressure in Network Process.
This might be done in a follow-up patch though, w/o the memory pressure cleanup.
Created attachment 365880[details]
Archive of layout-test-results from ews206 for win-future
The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews206 Port: win-future Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Comment on attachment 366000[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=366000&action=review> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:213
> + if (m_parameters.options.mode == FetchOptions::Mode::Navigate) {
We probably want to restrict to top level navigation for now.
> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:768
> + }
We should probably add a check on didReceiveResponse/didReceiveBuffer to not send data if it is a prefetch request.
> LayoutTests/http/tests/cache/link-prefetch-main-resource.html:1
> +<!-- webkit-test-runner [ experimental:LinkPrefetch=false ] -->
Should be true?
> LayoutTests/http/tests/cache/link-prefetch-main-resource.html:10
> + internals.settings.setStorageBlockingPolicy('BlockThirdParty')
Why is it needed?
> LayoutTests/http/tests/cache/link-prefetch-main-resource.html:40
> +<link rel="prefetch" href="http://localhost:8000/cache/resources/prefetched-main-resource.php" onload="setTimeout(loadAfterPrefetch, 0);">
We might want to contribute these tests to WPT maybe.
> LayoutTests/http/tests/cache/resources/prefetched-main-resource.php:4
> + header("Access-Control-Allow-Origin: http://127.0.0.1:8000");
Maybe we should have echo "parent.window.postMessage('PASS', '*');";
> LayoutTests/http/tests/cache/resources/prefetched-main-resource.php:12
> +echo "parent.window.postMessage('nonprefetch', '*');";
Maybe we should have echo "parent.window.postMessage('FAIL', '*');";
Created attachment 366057[details]
Archive of layout-test-results from ews124 for ios-simulator-wk2
The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews124 Port: ios-simulator-wk2 Platform: Mac OS X 10.13.6
(In reply to youenn fablet from comment #78)
> I wonder how/whether we want want load/error events.
> Should load event be fired when load starts or when load finishes? Can the
> timing of the load event leak some information if done at load finish?
> If load fails, should error event be sent?
>
> If we want to support load/error events when load finishes, we might need to
> provide an opaque response to the web process. This might be best handled by
> a new mode, so that NetworkLoadChecker does the necessary sanitization. And
> we do not want the body to be transmitted in any case.
>
> One safe way might be to fire the load event whenever the load starts and
> the error event if the load is not even started.
Firing at load start would diverge from typical `load` event semantics. The opaque response approach seems best to me.
>
> Ideally, we would then directly return didReceiveResponse/didFinishLoading.
> But we will then need to handle the case of loads that last longer than what
> the WebProcess thinks they do.
Comment on attachment 366000[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=366000&action=review>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:213
>> + if (m_parameters.options.mode == FetchOptions::Mode::Navigate) {
>
> We probably want to restrict to top level navigation for now.
Done.
>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:768
>> + }
>
> We should probably add a check on didReceiveResponse/didReceiveBuffer to not send data if it is a prefetch request.
Done.
>> LayoutTests/http/tests/cache/link-prefetch-main-resource.html:1
>> +<!-- webkit-test-runner [ experimental:LinkPrefetch=false ] -->
>
> Should be true?
I think it has no effect, probably since link prefetch is an experimental flag and will get enabled on test runs by default, so I removed the line.
>> LayoutTests/http/tests/cache/link-prefetch-main-resource.html:10
>> + internals.settings.setStorageBlockingPolicy('BlockThirdParty')
>
> Why is it needed?
I think this was needed at some point but it does not have an effect anymore, I removed it.
>> LayoutTests/http/tests/cache/link-prefetch-main-resource.html:40
>> +<link rel="prefetch" href="http://localhost:8000/cache/resources/prefetched-main-resource.php" onload="setTimeout(loadAfterPrefetch, 0);">
>
> We might want to contribute these tests to WPT maybe.
I hope they are good enough :)
>> LayoutTests/http/tests/cache/resources/prefetched-main-resource.php:4
>> + header("Access-Control-Allow-Origin: http://127.0.0.1:8000");
>
> Maybe we should have echo "parent.window.postMessage('PASS', '*');";
Done.
>> LayoutTests/http/tests/cache/resources/prefetched-main-resource.php:12
>> +echo "parent.window.postMessage('nonprefetch', '*');";
>
> Maybe we should have echo "parent.window.postMessage('FAIL', '*');";
Done.
Comment on attachment 366069[details]
Patch
This seems almost ready.
Some comments and suggestions.
View in context: https://bugs.webkit.org/attachment.cgi?id=366069&action=review> Source/WebCore/loader/LinkLoader.cpp:279
> + // FIXME further prefetch limitations:
s/FIXME further/FIXME: Add further prefetch restrictions/
> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:470
> + send(Messages::WebResourceLoader::DidReceiveResponse { response, willWaitForContinueDidReceiveResponse });
So, the idea would be to keep sending the prefetch to the memory cache if same origin.
That makes some sense for optimizations.
The fetch PR for defining is not really aligned but we could reconcile both progressively.
My question is what happens for cross origin prefetch in the web process?
I guess they do not go in the memory cache.
Do we have cases of loading where we go directly from send request to didFinishLoading without going through didReceiveResponse?
It seems we could do:
if (isCrossOriginPrefetch(originalRequest()) {
send(Messages::WebResourceLoader::DidReceiveResponse { ResourceResponse { }, false });
return completionHandler(PolicyAction::Use);
}
> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:67
> +static const Seconds expirationTimeout { 5_s };
We might want to update this in the future, 5 seconds might not be long enough.
I wonder whether it should be something like 5_s after the page doing the prefetch navigated away.
This might add some complexity to the implementation though.
Not for this patch.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:89
> + return *map;
We use m_sessionPrefetches.ensure nowadays.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:59
> + void clear();
We might need to clear part of the prefetch cache in NetworkProcess::destroySession.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:61
> + std::unique_ptr<PrefetchEntry> take(URL, PAL::SessionID);
const URL&.
I believe we usually pass SessionID parameter first.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:62
> + void store(URL, WebCore::ResourceResponse&&, RefPtr<WebCore::SharedBuffer>&&, PAL::SessionID);
URL&&
> LayoutTests/platform/mac-wk1/TestExpectations:699
> +webkit.org/b/195623 http/tests/cache/link-prefetch-main-resource.html [ Failure ]
Let's skip it as well since we miss the implementation.
> LayoutTests/platform/win/TestExpectations:4308
> +webkit.org/b/195623 http/tests/cache/link-prefetch-main-resource.html [ Timeout ]
Let's skip it then
Rethinking about it, we do not want cross origin prefetch to end in the memory cache with an empty response. Maybe that was your original idea?
I would anyway make it clear that compeltionHandler is called with use in any prefetch
Attachment 366266[details] did not pass style-queue:
ERROR: LayoutTests/platform/win/TestExpectations:4306: Path does not exist. [test/expectations] [5]
ERROR: /Volumes/Data/StyleQueue/WebKit/LayoutTests/platform/mac-wk1/TestExpectations:699: Path does not exist. [test/expectations] [5]
ERROR: /Volumes/Data/StyleQueue/WebKit/LayoutTests/platform/win/TestExpectations:4306: Path does not exist. [test/expectations] [5]
Total errors found: 3 in 17 files
If any of these errors are false positives, please file a bug against check-webkit-style.
Created attachment 366278[details]
Archive of layout-test-results from ews103 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103 Port: mac-highsierra Platform: Mac OS X 10.13.6
Comment on attachment 366069[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=366069&action=review>> Source/WebCore/loader/LinkLoader.cpp:279
>> + // FIXME further prefetch limitations:
>
> s/FIXME further/FIXME: Add further prefetch restrictions/
Done.
>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:470
>> + send(Messages::WebResourceLoader::DidReceiveResponse { response, willWaitForContinueDidReceiveResponse });
>
> So, the idea would be to keep sending the prefetch to the memory cache if same origin.
> That makes some sense for optimizations.
> The fetch PR for defining is not really aligned but we could reconcile both progressively.
>
> My question is what happens for cross origin prefetch in the web process?
> I guess they do not go in the memory cache.
>
> Do we have cases of loading where we go directly from send request to didFinishLoading without going through didReceiveResponse?
> It seems we could do:
> if (isCrossOriginPrefetch(originalRequest()) {
> send(Messages::WebResourceLoader::DidReceiveResponse { ResourceResponse { }, false });
> return completionHandler(PolicyAction::Use);
> }
I don't think we have cases of didFinishLoading without going through didReceiveResponse.
Sending an empty ResourceResponse actually hits an ASSERT on WebCore side.
I don't think it is too important if the prefetch ends up in the memory cache as it will be a cache miss anyway (partitioning). But I'll think on this further.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:67
>> +static const Seconds expirationTimeout { 5_s };
>
> We might want to update this in the future, 5 seconds might not be long enough.
> I wonder whether it should be something like 5_s after the page doing the prefetch navigated away.
> This might add some complexity to the implementation though.
> Not for this patch.
Ok.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:89
>> + return *map;
>
> We use m_sessionPrefetches.ensure nowadays.
Oops, forgot about this, will try to fix it soon.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:59
>> + void clear();
>
> We might need to clear part of the prefetch cache in NetworkProcess::destroySession.
When would that be useful?
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:61
>> + std::unique_ptr<PrefetchEntry> take(URL, PAL::SessionID);
>
> const URL&.
> I believe we usually pass SessionID parameter first.
Done.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:62
>> + void store(URL, WebCore::ResourceResponse&&, RefPtr<WebCore::SharedBuffer>&&, PAL::SessionID);
>
> URL&&
I think that makes things more complex so I left it off for now.
>> LayoutTests/platform/mac-wk1/TestExpectations:699
>> +webkit.org/b/195623 http/tests/cache/link-prefetch-main-resource.html [ Failure ]
>
> Let's skip it as well since we miss the implementation.
Done.
>> LayoutTests/platform/win/TestExpectations:4308
>> +webkit.org/b/195623 http/tests/cache/link-prefetch-main-resource.html [ Timeout ]
>
> Let's skip it then
Done.
(In reply to youenn fablet from comment #95)
> Btw, the test is doing an iframe load.
> Should we do a top level navigation prefetch load instead?
Yes that sounds useful. I added such a test and renamed the iframe test to make it more clear it is using an iframe.
(In reply to youenn fablet from comment #96)
> Rethinking about it, we do not want cross origin prefetch to end in the
> memory cache with an empty response. Maybe that was your original idea?
> I would anyway make it clear that compeltionHandler is called with use in
> any prefetch
My first patches were concentrated on the memory cache which turned out to be the wrong approach. Since then I have not been too concerned with it because of the cache miss. I assume at this point making cached result empty is kind of efficient, compared to storing the full result. Do we even really need an entry for prefetches in the memory cache? Again I need to think about it, but let me know if you get convinced of the best way to go here.
Created attachment 366280[details]
Archive of layout-test-results from ews204 for win-future
The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews204 Port: win-future Platform: CYGWIN_NT-6.1-2.10.0-0.325-5-3-x86_64-64bit
Created attachment 366281[details]
Archive of layout-test-results from ews114 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 366479[details]
Archive of layout-test-results from ews100 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 366481[details]
Archive of layout-test-results from ews117 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews117 Port: mac-highsierra Platform: Mac OS X 10.13.6
> My first patches were concentrated on the memory cache which turned out to
> be the wrong approach. Since then I have not been too concerned with it
> because of the cache miss. I assume at this point making cached result empty
> is kind of efficient, compared to storing the full result. Do we even really
> need an entry for prefetches in the memory cache? Again I need to think
> about it, but let me know if you get convinced of the best way to go here.
We certainly do not want cross-origin prefetches to go in the memory cache.
We could be ok with same-origin prefetches to go in the memory cache but I feel like it might not be worth the added complexity.
Comment on attachment 366488[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=366488&action=review> Source/WebCore/ChangeLog:14
> + http/tests/cache/link-prefetch-main-resource.html
Double line.
> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:213
> + if (isMainResource() && m_parameters.options.mode == FetchOptions::Mode::Navigate) {
Let's restrict it to isMainFrameLoad().
I would think iframes do not need to opt out of partitioning and preload could be used instead for these.
> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:467
> + send(Messages::WebResourceLoader::DidReceiveResponse { response, false });
I think we should do ResourceResponse { } instead of passing response.
> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:770
> + const auto& request = originalRequest();
request is only used once, maybe we can remove it.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:64
> + return resources->take(url);
Do we need the call to resources->contains().
If not there, I would believe take will return nullptr.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:92
> +void PrefetchCache::expiration()
How about clearExpiredEntries?
> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:36
> +#include <wtf/text/WTFString.h>
CompletionHandler.h/Seconds.h do not seem needed.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:50
> + mutable RefPtr<WebCore::SharedBuffer> m_buffer;
We can probably simplify this by removing mutable and making releaseBuffer() not const.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:54
> + WTF_MAKE_NONCOPYABLE(PrefetchCache); WTF_MAKE_FAST_ALLOCATED;
Add a \n
Comment on attachment 366488[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=366488&action=review>> Source/WebCore/ChangeLog:14
>> + http/tests/cache/link-prefetch-main-resource.html
>
> Double line.
Fixed.
>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:213
>> + if (isMainResource() && m_parameters.options.mode == FetchOptions::Mode::Navigate) {
>
> Let's restrict it to isMainFrameLoad().
> I would think iframes do not need to opt out of partitioning and preload could be used instead for these.
Done.
>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:467
>> + send(Messages::WebResourceLoader::DidReceiveResponse { response, false });
>
> I think we should do ResourceResponse { } instead of passing response.
Done.
>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:770
>> + const auto& request = originalRequest();
>
> request is only used once, maybe we can remove it.
Done.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:64
>> + return resources->take(url);
>
> Do we need the call to resources->contains().
> If not there, I would believe take will return nullptr.
Right, not needed.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:92
>> +void PrefetchCache::expiration()
>
> How about clearExpiredEntries?
Sorry, I forgot about this issue! I like the name, done.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:36
>> +#include <wtf/text/WTFString.h>
>
> CompletionHandler.h/Seconds.h do not seem needed.
Done.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:50
>> + mutable RefPtr<WebCore::SharedBuffer> m_buffer;
>
> We can probably simplify this by removing mutable and making releaseBuffer() not const.
Done.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:54
>> + WTF_MAKE_NONCOPYABLE(PrefetchCache); WTF_MAKE_FAST_ALLOCATED;
>
> Add a \n
Done.
(In reply to youenn fablet from comment #115)
> > My first patches were concentrated on the memory cache which turned out to
> > be the wrong approach. Since then I have not been too concerned with it
> > because of the cache miss. I assume at this point making cached result empty
> > is kind of efficient, compared to storing the full result. Do we even really
> > need an entry for prefetches in the memory cache? Again I need to think
> > about it, but let me know if you get convinced of the best way to go here.
>
> We certainly do not want cross-origin prefetches to go in the memory cache.
> We could be ok with same-origin prefetches to go in the memory cache but I
> feel like it might not be worth the added complexity.
Okay, thanks for clarifying. In the latest patch I now use CachingPolicy::DisallowCaching to not use the memory cache for prefetches at all.
Comment on attachment 366602[details]
Patch
r=me
LGTM.
Some other things we might need:
- Web Inspector integration?
- Handling of still loading prefetched responses
View in context: https://bugs.webkit.org/attachment.cgi?id=366602&action=review> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:34
> +#include <wtf/text/WTFString.h>
probably not needed.
Comment on attachment 366602[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=366602&action=review> Source/WebCore/loader/ResourceLoadInfo.h:47
> + Prefetch = 0x0400,
> };
> const uint16_t ResourceTypeMask = 0x03FF;
Seems dangerous that there’s a new type that’s not in ResourceTypeMask. Shouldn’t this have been changed to 0x7FF?
Comment on attachment 366602[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=366602&action=review> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:46
> + clear();
This seems unnecessary. I don’t see any work done in clear() that has any value if the objects are being destroyed immediately afterward. Seems like this will just make the destructor slower with no benefit.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:86
> + auto& map = m_sessionPrefetches.add(sessionID, nullptr).iterator->value;
> + if (!map)
> + map = std::make_unique<PrefetchEntriesMap>();
> + return *map;
Could use HashMap::ensure instead of add to avoid the if statement.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:49
> +class PrefetchEntry {
> + WTF_MAKE_FAST_ALLOCATED;
> +public:
> + PrefetchEntry(WebCore::ResourceResponse&&, RefPtr<WebCore::SharedBuffer>&&);
> +
> + const WebCore::ResourceResponse& response() const { return m_response; }
> + RefPtr<WebCore::SharedBuffer> releaseBuffer() { return m_buffer.releaseNonNull(); }
> +
> +private:
> + WebCore::ResourceResponse m_response;
> + RefPtr<WebCore::SharedBuffer> m_buffer;
> +};
Why a class instead of a struct for this? I think it would be better with a struct. I also think this should be a member instead of a separate struct. PrefetchCache::Entry.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:66
> + typedef HashMap<URL, std::unique_ptr<PrefetchEntry>> PrefetchEntriesMap;
New code should us "using" instead of typedef.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:70
> + typedef HashMap<PAL::SessionID, std::unique_ptr<PrefetchEntriesMap>> SessionPrefetchResourceMap;
Ditto.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:73
> + typedef Deque<std::tuple<PAL::SessionID, URL, WallTime>> SessionPrefetchExpirationList;
Ditto.
Comment on attachment 367385[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=367385&action=review> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:55
> + struct Entry {
> + Entry(WebCore::ResourceResponse&&, RefPtr<WebCore::SharedBuffer>&&);
> +
> + const WebCore::ResourceResponse& response() const { return m_response; }
> + RefPtr<WebCore::SharedBuffer> releaseBuffer() { return m_buffer.releaseNonNull(); }
> +
> + WebCore::ResourceResponse m_response;
> + RefPtr<WebCore::SharedBuffer> m_buffer;
> + };
More about what I meant when I suggested this be a struct: This struct can simply have "response" and "buffer" public data members. No need for the "m_" prefix for public struct data members. Also likely don’t need to define a constructor since aggregate construction should work. Could keep the releaseBuffer function if you like, or callers can just write buffer.releaseNonNull() directly. So ideally the struct just has two lines in its body, keeping it simpler.
Hi,
(In reply to Ryan Haddad from comment #133)
> The two tests added with this change are consistently failing an assertion
> on macOS and iOS Debug WK2 bots:
So mac-debug and mac-wk2 are not covering this? If so I have no real way of detecting this (except an expensive local WK2 Debug build).
I think I know what is going on and can try to fix it tomorrow.
Comment on attachment 367534[details]
Patch
Ready for review. Another option is to convert the ASSERT to an if check with early return for null response. Both options seem like no-ops to me, so it may be best to just not call it for our case.
Comment on attachment 367804[details]
Patch
Hi Darin,
The only change is that PrefetchCache::take now removes entries from both sessionPrefetchMap and m_sessionExpirationList. Before only the sessionPrefetchMap entry was removed, so clearExpiredEntries still would process the m_sessionExpirationList and hit the resources->contains(requestUrl) ASSERT.
Comment on attachment 367804[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=367804&action=review> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:61
> + return std::get<0>(tuple) == sessionID && std::get<1>(tuple) == url;
I think it’s more elegant to write it this way:
return tuple == std::tie(sessionID, url);
Unless that doesn’t work for some reason?
Comment on attachment 367804[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=367804&action=review> Source/WebCore/loader/ResourceLoadInfo.h:45
> + Prefetch = 0x0400,
This does not need to be added. If I am correct, these "Prefetch" loads are just Documents, right?
> Source/WebCore/loader/ResourceLoader.cpp:345
> + ASSERT(m_resourceType != ResourceType::Invalid || m_resourceType == ResourceType::Prefetch);
This is redundant. If the resource type is equal to Prefetch, then it is also not equal to Invalid. Why did you add this?
> Source/WebCore/loader/ResourceLoader.cpp:356
> + if (!redirectResponse.isNull() && frameLoader() && m_resourceType != ResourceType::Prefetch) {
This adds a way to bypass WKContentRuleLists entirely with prefetch. We do not want to do this.
Comment on attachment 367804[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=367804&action=review>> Source/WebCore/loader/ResourceLoadInfo.h:45
>> + Prefetch = 0x0400,
>
> This does not need to be added. If I am correct, these "Prefetch" loads are just Documents, right?
More or less. I am not sure if you can navigate to a (document) image, I guess you can, but I think the only useful thing to navigate to is a Document, so I removed the Prefetch value.
>> Source/WebCore/loader/ResourceLoader.cpp:345
>> + ASSERT(m_resourceType != ResourceType::Invalid || m_resourceType == ResourceType::Prefetch);
>
> This is redundant. If the resource type is equal to Prefetch, then it is also not equal to Invalid. Why did you add this?
I did hit this at some point, but now removed since Prefetch value is gone.
>> Source/WebCore/loader/ResourceLoader.cpp:356
>> + if (!redirectResponse.isNull() && frameLoader() && m_resourceType != ResourceType::Prefetch) {
>
> This adds a way to bypass WKContentRuleLists entirely with prefetch. We do not want to do this.
Right, removed.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:30
>> +#include <wtf/NeverDestroyed.h>
>
> Really surprised this include is needed.
You are right, not needed. I think this stems from the initial singleton design.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.cpp:61
>> + return std::get<0>(tuple) == sessionID && std::get<1>(tuple) == url;
>
> I think it’s more elegant to write it this way:
>
> return tuple == std::tie(sessionID, url);
>
> Unless that doesn’t work for some reason?
This did not seem to work for me, I left as-is. I did not check the comparison operator implementation here but I think it has problems with the third tuple item.
Comment on attachment 367932[details]
Patch
Please include a test that blocks the prefetch load with a rule that has resource-type to make sure we are not introducing a way to bypass content rule lists. You can find similar tests in http/tests/contentextensions
(In reply to Alex Christensen from comment #155)
> Comment on attachment 367932[details]
> Patch
>
> Please include a test that blocks the prefetch load with a rule that has
> resource-type to make sure we are not introducing a way to bypass content
> rule lists. You can find similar tests in http/tests/contentextensions
Done.
Comment on attachment 368016[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=368016&action=review> Source/WebKit/NetworkProcess/NetworkProcess.h:560
> + PrefetchCache m_prefetchCache;
This should probably live on the NetworkSession. We don't want private browsing to share state with non-private browsing.
> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:50
> + RefPtr<WebCore::SharedBuffer> releaseBuffer() { return buffer.releaseNonNull(); }
If we're really releasing non null, then this could be a Ref.
> LayoutTests/http/tests/contentextensions/prefetch-blocked.html.json:7
> + "url-filter": "should-not-load"
This trigger should have "resource-type": "document"
Created attachment 368129[details]
Archive of layout-test-results from ews102 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 368134[details]
Archive of layout-test-results from ews113 for mac-highsierra
The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113 Port: mac-highsierra Platform: Mac OS X 10.13.6
Created attachment 368136[details]
Archive of layout-test-results from ews123 for ios-simulator-wk2
The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews123 Port: ios-simulator-wk2 Platform: Mac OS X 10.13.6
Comment on attachment 368016[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=368016&action=review>> Source/WebKit/NetworkProcess/NetworkProcess.h:560
>> + PrefetchCache m_prefetchCache;
>
> This should probably live on the NetworkSession. We don't want private browsing to share state with non-private browsing.
Done.
>> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:50
>> + RefPtr<WebCore::SharedBuffer> releaseBuffer() { return buffer.releaseNonNull(); }
>
> If we're really releasing non null, then this could be a Ref.
Done.
>> LayoutTests/http/tests/contentextensions/prefetch-blocked.html.json:7
>> + "url-filter": "should-not-load"
>
> This trigger should have "resource-type": "document"
Done.
Comment on attachment 368137[details]
Patch
Some additional comments.
View in context: https://bugs.webkit.org/attachment.cgi?id=368137&action=review> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:214
> + if (isMainFrameLoad() && m_parameters.options.mode == FetchOptions::Mode::Navigate) {
We could simplify to isMainFrameLoad() here.
If need be, we could also ASSERT(m_parameters.options.mode == FetchOptions::Mode::Navigate).
> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:1127
> +bool NetworkResourceLoader::isCrossOriginPrefetch(const ResourceRequest& request) const
We could simplify by removing request parameter and use originalRequest() here since prefetch -> redirect mode is manual.
Do we have tests for prefetch redirections?
We should be able to store in the prefetch cache these redirections.
> LayoutTests/http/tests/contentextensions/prefetch-blocked.html:3
> +<link rel="prefetch" href="resources/should-not-load.html">
Can we call testRunner.dumpAsText()?
Created attachment 368313[details]
Archive of layout-test-results from ews123 for ios-simulator-wk2
The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews123 Port: ios-simulator-wk2 Platform: Mac OS X 10.13.6
Comment on attachment 368137[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=368137&action=review>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:214
>> + if (isMainFrameLoad() && m_parameters.options.mode == FetchOptions::Mode::Navigate) {
>
> We could simplify to isMainFrameLoad() here.
> If need be, we could also ASSERT(m_parameters.options.mode == FetchOptions::Mode::Navigate).
Done.
>> Source/WebKit/NetworkProcess/NetworkResourceLoader.cpp:1127
>> +bool NetworkResourceLoader::isCrossOriginPrefetch(const ResourceRequest& request) const
>
> We could simplify by removing request parameter and use originalRequest() here since prefetch -> redirect mode is manual.
> Do we have tests for prefetch redirections?
> We should be able to store in the prefetch cache these redirections.
Done. Except we do not have tests for prefetch redirections yet. Do you have suggestions how to test prefetch redirections?
>> LayoutTests/http/tests/contentextensions/prefetch-blocked.html:3
>> +<link rel="prefetch" href="resources/should-not-load.html">
>
> Can we call testRunner.dumpAsText()?
Done.
> > We could simplify by removing request parameter and use originalRequest() here since prefetch -> redirect mode is manual.
> > Do we have tests for prefetch redirections?
> > We should be able to store in the prefetch cache these redirections.
>
> Done. Except we do not have tests for prefetch redirections yet. Do you have
> suggestions how to test prefetch redirections?
To be clear, it is fine to leave the support of storing prefetch redirections to a follow-up patch.This can be tested as follows:
- Write a script that triggers a redirection for the first load and returns "FAIL" for any subsequent load. Script state might be stored using WPT/stash and cleared on demand.
- Write a test prefetching the new script. Redirection will be stored.
- Navigate to the script. redirection should be followed if prefetch is hit. Otherwise, the frame will show "FAIL"
Another test that might be interesting to have is prefetched resources that are not cacheable or have a very short expiracy time.
(In reply to youenn fablet from comment #176)
> > > We could simplify by removing request parameter and use originalRequest() here since prefetch -> redirect mode is manual.
> > > Do we have tests for prefetch redirections?
> > > We should be able to store in the prefetch cache these redirections.
> >
> > Done. Except we do not have tests for prefetch redirections yet. Do you have
> > suggestions how to test prefetch redirections?
>
> To be clear, it is fine to leave the support of storing prefetch
> redirections to a follow-up patch.This can be tested as follows:
> - Write a script that triggers a redirection for the first load and returns
> "FAIL" for any subsequent load. Script state might be stored using WPT/stash
> and cleared on demand.
> - Write a test prefetching the new script. Redirection will be stored.
> - Navigate to the script. redirection should be followed if prefetch is hit.
> Otherwise, the frame will show "FAIL"
>
> Another test that might be interesting to have is prefetched resources that
> are not cacheable or have a very short expiracy time.
Thanks for the decriptions, I have opened https://bugs.webkit.org/show_bug.cgi?id=197371 for it.
(In reply to Rob Buis from comment #177)
> > To be clear, it is fine to leave the support of storing prefetch
> > redirections to a follow-up patch.This can be tested as follows:
> > - Write a script that triggers a redirection for the first load and returns
> > "FAIL" for any subsequent load. Script state might be stored using WPT/stash
> > and cleared on demand.
> > - Write a test prefetching the new script. Redirection will be stored.
> > - Navigate to the script. redirection should be followed if prefetch is hit.
> > Otherwise, the frame will show "FAIL"
> >
> > Another test that might be interesting to have is prefetched resources that
> > are not cacheable or have a very short expiracy time.
>
> Thanks for the decriptions, I have opened
> https://bugs.webkit.org/show_bug.cgi?id=197371 for it.
To be clear, I meant a test for the redirect, not the short expiracy test. So the mentioned bug now has tat test, and I have a local fix to make it pass, but it is easier if this patch goes in first, to make sure the dependent 197371 patch builds.
Comment on attachment 368343[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=368343&action=review> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:50
> + Ref<WebCore::SharedBuffer> releaseBuffer() { return buffer.releaseNonNull(); }
I do not think we can guarantee buffer will always be null.
We might just need to WTFMove(buffer) when storing it in network cache.
Comment on attachment 368343[details]
Patch
View in context: https://bugs.webkit.org/attachment.cgi?id=368343&action=review> Source/WebKit/NetworkProcess/cache/PrefetchCache.h:50
> + Ref<WebCore::SharedBuffer> releaseBuffer() { return buffer.releaseNonNull(); }
I do not think we can guarantee buffer will always be null.
We might just need to WTFMove(buffer) when storing it in network cache.
2019-03-12 09:32 PDT, Rob Buis
2019-03-12 10:38 PDT, EWS Watchlist
2019-03-12 11:20 PDT, EWS Watchlist
2019-03-12 11:46 PDT, EWS Watchlist
2019-03-14 14:31 PDT, Rob Buis
2019-03-14 15:28 PDT, EWS Watchlist
2019-03-14 15:43 PDT, EWS Watchlist
2019-03-14 16:21 PDT, EWS Watchlist
2019-03-14 17:20 PDT, EWS Watchlist
2019-03-14 19:49 PDT, EWS Watchlist
2019-03-18 08:57 PDT, Rob Buis
2019-03-18 10:07 PDT, EWS Watchlist
2019-03-18 10:17 PDT, EWS Watchlist
2019-03-18 10:56 PDT, EWS Watchlist
2019-03-18 11:45 PDT, Rob Buis
2019-03-18 12:59 PDT, EWS Watchlist
2019-03-18 14:07 PDT, EWS Watchlist
2019-03-18 18:48 PDT, EWS Watchlist
2019-03-19 09:40 PDT, Rob Buis
2019-03-19 10:42 PDT, EWS Watchlist
2019-03-19 11:51 PDT, EWS Watchlist
2019-03-19 11:54 PDT, EWS Watchlist
2019-03-19 13:49 PDT, Rob Buis
2019-03-19 15:02 PDT, EWS Watchlist
2019-03-19 15:46 PDT, EWS Watchlist
2019-03-19 16:21 PDT, EWS Watchlist
2019-03-19 16:55 PDT, EWS Watchlist
2019-03-20 04:05 PDT, Rob Buis
2019-03-20 05:47 PDT, EWS Watchlist
2019-03-20 06:19 PDT, EWS Watchlist
2019-03-20 14:30 PDT, Rob Buis
2019-03-20 18:38 PDT, EWS Watchlist
2019-03-21 08:40 PDT, Rob Buis
2019-03-21 10:06 PDT, Rob Buis
2019-03-21 12:36 PDT, EWS Watchlist
2019-03-21 17:13 PDT, EWS Watchlist
2019-03-22 02:07 PDT, Rob Buis
2019-03-22 03:27 PDT, EWS Watchlist
2019-03-22 04:05 PDT, EWS Watchlist
2019-03-25 09:44 PDT, Rob Buis
2019-03-25 12:15 PDT, EWS Watchlist
2019-03-25 12:33 PDT, Rob Buis
2019-03-26 12:38 PDT, Rob Buis
2019-03-26 14:42 PDT, Rob Buis
2019-03-27 00:01 PDT, EWS Watchlist
2019-03-27 07:39 PDT, Rob Buis
2019-03-29 04:56 PDT, Rob Buis
2019-03-29 08:37 PDT, Rob Buis
2019-03-29 09:45 PDT, EWS Watchlist
2019-03-29 10:25 PDT, EWS Watchlist
2019-03-29 10:35 PDT, EWS Watchlist
2019-04-02 01:44 PDT, Rob Buis
2019-04-02 02:53 PDT, EWS Watchlist
2019-04-02 03:35 PDT, EWS Watchlist
2019-04-02 05:11 PDT, Rob Buis
2019-04-03 01:39 PDT, Rob Buis
2019-04-03 01:59 PDT, Rob Buis
2019-04-03 07:38 PDT, Rob Buis
2019-04-13 05:07 PDT, Rob Buis
2019-04-13 08:17 PDT, Rob Buis
2019-04-14 12:55 PDT, Rob Buis
2019-04-16 06:49 PDT, Rob Buis
2019-04-19 09:46 PDT, Rob Buis
2019-04-20 10:12 PDT, Rob Buis
2019-04-22 02:59 PDT, Rob Buis
2019-04-22 23:48 PDT, Rob Buis
2019-04-24 08:00 PDT, Rob Buis
2019-04-24 09:10 PDT, EWS Watchlist
2019-04-24 09:52 PDT, EWS Watchlist
2019-04-24 10:00 PDT, EWS Watchlist
2019-04-24 10:09 PDT, Rob Buis
2019-04-26 01:02 PDT, Rob Buis
2019-04-26 02:47 PDT, EWS Watchlist
2019-04-26 08:14 PDT, Rob Buis
2019-04-26 09:56 PDT, Rob Buis
2019-04-26 12:33 PDT, Rob Buis
2019-05-08 04:10 PDT, Rob Buis