Bug 187435 – "serviceworker.js" is fetched several times in a row

• Component: Page Loading
• Status: RESOLVED FIXED

[N]

I enabled an Apache module that will block IPs that send several requests to my website in a very short time.

I noticed that a number of IPs were blocked straight away, and all of those users use Safari on iOS.

After a quick investigation, I saw that those users were requesting "serviceworker.js" several times. I believe this is a bug and this file should only be fetched once.



Example access logs below:

07/Jul/2018 17:02:51	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:51	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:51	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:51	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:51	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:52	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:52	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:52	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:52	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:52	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:52	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:52	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:52	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:52	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:52	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:53	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:53	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:53	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:53	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:53	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:53	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:53	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:53	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:53	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:53	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:53	GET	HTTP/1.1	403	815	/serviceworker.js?v=29
07/Jul/2018 17:02:54	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:54	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29
07/Jul/2018 17:02:54	GET	HTTP/1.1	200	1718	/serviceworker.js?v=29



Response Headers:

HTTP/1.1 200 OK
Date: Sat, 07 Jul 2018 17:22:36 GMT
Server: Apache
Last-Modified: Sun, 01 Jul 2018 18:48:58 GMT
ETag: "284a38-1328-56ff488a4561e-gzip"
Accept-Ranges: bytes
Cache-Control: max-age=290304000, public
Expires: Tue, 02 Jul 2019 17:22:36 GMT
Vary: Accept-Encoding
Content-Encoding: gzip
Access-Control-Allow-Origin: *
Content-Length: 1718
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Content-Type: application/javascript


Happens on both 11.3 and 11.4.

[N]

Please note that the access logs example above are just for 1 IP.

All IPs blocked so far followed the same pattern.

[Radar WebKit Bug Importer]

<rdar://problem/41940569>

[youenn fablet]

As per service worker spec, every time a navigation load happens (a page or an iframe is loaded), the service worker script is fetched, so this repetitive loads could actually happen.
I would expect 304 though, not 200, and it seems suspicious that only Safari/iOS would behave this way.
Nuno, is there a web site where we could repro this case?

[Chris Dumez]

(In reply to youenn fablet from comment #3)
> As per service worker spec, every time a navigation load happens (a page or
> an iframe is loaded), the service worker script is fetched, so this
> repetitive loads could actually happen.
> I would expect 304 though, not 200, and it seems suspicious that only
> Safari/iOS would behave this way.
> Nuno, is there a web site where we could repro this case?

Note that by default, service workers script fetches bypass the HTTP cache, as per the specification. We actually reported a bug against the specification about this but this behavior was apparently intentional.

As a Web developer, you can opt into using the HTTP cache by using {updateViaCache: "all" } as options when registering your service worker.

[Chris Dumez]

(In reply to Chris Dumez from comment #4)
> (In reply to youenn fablet from comment #3)
> > As per service worker spec, every time a navigation load happens (a page or
> > an iframe is loaded), the service worker script is fetched, so this
> > repetitive loads could actually happen.
> > I would expect 304 though, not 200, and it seems suspicious that only
> > Safari/iOS would behave this way.
> > Nuno, is there a web site where we could repro this case?
> 
> Note that by default, service workers script fetches bypass the HTTP cache,
> as per the specification. We actually reported a bug against the
> specification about this but this behavior was apparently intentional.
> 
> As a Web developer, you can opt into using the HTTP cache by using
> {updateViaCache: "all" } as options when registering your service worker.

Spec says:
"""
If request is a non-subresource request, or request is a subresource request and the time difference in seconds calculated by the current time minus registration’s last update check time is greater than 86400, invoke Soft Update algorithm with registration.
"""

And Soft update is defined at https://w3c.github.io/ServiceWorker/#soft-update.

The way I understand the specification, the "last update check time is greater than 86400" check only applies to sub resource loads, not main resource requests. Although it could be me misunderstanding the spec.

I personally, really do not like this behavior though.

[youenn fablet]

> The way I understand the specification, the "last update check time is
> greater than 86400" check only applies to sub resource loads, not main
> resource requests. Although it could be me misunderstanding the spec.

Right, I believe we currently fetch the worker script for every main resource load, as per spec.
I believe we should still send conditional GET requests whenever possible though, which might not be the case here.

Checking https://mdn.github.io/sw-test/, we are actually sending conditional GET requests.

[Chris Dumez]

Created attachment 344612 [details]
Patch

[Chris Dumez]

Comment on attachment 344612 [details]
Patch

Causes WPT test failures.

[Chris Dumez]

Created attachment 344621 [details]
Patch

[youenn fablet]

Comment on attachment 344621 [details]
Patch

As long as the bots are happy

[N]

> > As a Web developer, you can opt into using the HTTP cache by using {updateViaCache: "all" } as options when registering your service worker.


Couldn't really find much information about this, but this option seems to be used for when updating inside a registration?

However, that's not what I'm doing. I'm simply registering the service worker like this:


	navigator.serviceWorker.register(serviceWorkerUrl).then(function(swReg) {
		serviceWorkerRegistration = swReg;
		
		(PushManager stuff here)
	}).catch(function(error) {
		console.error(error);
	});



> > As per service worker spec, every time a navigation load happens (a page or
> > an iframe is loaded), the service worker script is fetched, so this
> > repetitive loads could actually happen.


Are you saying that for every single image/js/css/etc, the serviceworker.js is retrieved from the server?!
That doesn't make any sense.

And only Safari (iPhone and MacOS) is doing that... all other browsers are working fine.

Only with Safari I'm having issues so far. The previous one I had is that users can't use my website on Private mode since 11.4.
https://bugs.webkit.org/show_bug.cgi?id=186617

Also, Safari is the only browser I know that supports service workers and doesn't support Push....

[N]

Note that my website uses ajax heavily, so there should really be only one serviceworker call. Once the user loaded the page, any further navigations shouldn't invoke serviceworker, since the calls are done through ajax.

So, unless serviceworker is fetched for every image, css, js, etc,
the only other thing I can think of is that when Safari opens and there are multiple tabs with my website open, Safari will fetch serviceworker for every tab. I have no idea what really happens.

[Chris Dumez]

(In reply to Nuno from comment #12)
> Note that my website uses ajax heavily, so there should really be only one
> serviceworker call. Once the user loaded the page, any further navigations
> shouldn't invoke serviceworker, since the calls are done through ajax.
> 
> So, unless serviceworker is fetched for every image, css, js, etc,
> the only other thing I can think of is that when Safari opens and there are
> multiple tabs with my website open, Safari will fetch serviceworker for
> every tab. I have no idea what really happens.

For sub resources (js, css, img, ...), we would only fetch the service worker at most once a day. It is only for main resources (main document and subframes) that we would fetch the service worker script.

Yes, if you have multiple tab to your site open, each tab load would be intercepted by the service worker and trigger a serviceworker.js load.

In any case, the patch that is about to land should align our behavior with Chrome and you should still only one serviceworker.js load per page load.

As I mentioned earlier, you can opt into using the disk cache so that your server will be hit only when the serviceworker.js script in the HTTP cache has expired. To do so:

navigator.serviceWorker.register(serviceWorkerUrl, { updateViaCache: "all" });

This is on registration, unlike what you stated earlier.

[WebKit Commit Bot]

Comment on attachment 344621 [details]
Patch

Clearing flags on attachment: 344621

Committed r233719: <https://trac.webkit.org/changeset/233719>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.