156115 2016-04-01 13:54:32 -0700 Change frequency limit on replaceState to fail without throwing an exception 2016-06-10 11:21:59 -0700 1 1 1 Unclassified WebKit History Safari 9 All All NEW http://bl.ocks.org/jameskeane/raw/e0b145e6287b3f12f5e47e6b1d313308/ P2 Major --- 1 james.keane webkit-unassigned beidson james.keane nekr.fabula oldest_to_newest 1179926 0 275427 james.keane 2016-04-01 13:54:32 -0700 Created attachment 275427 Chat example - must be served over http for security restrictions An arbitrary 100 maximum calls to 'pushState' and 'replaceState' was recently added to WebKit, which I have been informed was recently softened to 100 calls per 30 second interval (https://trac.webkit.org/changeset/198687) in #155901. I can find no reasonable justification for diverging from the spec (https://html.spec.whatwg.org/multipage/browsers.html#the-history-interface). I assume it is related to "crashsafari.com", but I am "not authorized" to view the ticket (#153435) which introduced this change. I was asked on twitter by @beidson, how 3 updates per second is not enough; my use case is updating scroll state at 60fps, but this example of a chat app: http://bl.ocks.org/jameskeane/raw/e0b145e6287b3f12f5e47e6b1d313308/ also fails and is less easily coalesced. 1180076 1 beidson 2016-04-02 12:26:56 -0700 (In reply to comment #0) > I was asked on twitter by @beidson, how 3 updates per second is not enough; > my use case is updating scroll state at 60fps, Here's our thinking on this: The use case for replaceState is to make sure that the current history item has the most recent state possible before the current history item changes. The current history item really only needs to be updated exactly once - Right before the current history item moves to a new history item. What is the difference between "updating the current history item 60 times during a 1 second scroll operation" versus "updating the current history item once at the end of a 1 second scroll operation"? If the user is scrolling at 60fps for 1 second, for example, there's no need to update the state of the current history item 60 times. Only once at the end of the scrolling... or even better, only once right before the user "navigates" In theory, I can't actually think of a reason you'd ever actually have to call "replaceState" other than "right before you ever call pushState, or once inside of your pagehide handler" > but this example of a chat app: > http://bl.ocks.org/jameskeane/raw/e0b145e6287b3f12f5e47e6b1d313308/ > also fails and is less easily coalesced. Again, I'm confused why this is true - replaceState literally only needs to be called once before pushState, or once during pagehide. I don't see what makes this chat app any different. > I can find no reasonable justification for diverging from the spec > (https://html.spec.whatwg.org/multipage/browsers.html#the-history-interface). WebKit's current behavior is definitely inline with the HTML spec. The WebApp section of the spec gives browsers leeway to prevent abuse of system resources by throwing the QuotaExceededError exception, which is precisely what we're doing here. (https://html.spec.whatwg.org/multipage/webappapis.html#killing-scripts) Also, the the History spec is clear about the flexibility user agents can take with regard to the state APIs in section 7.7.4. (https://html.spec.whatwg.org/multipage/browsers.html#history-notes) While that section specifically calls out pushState, I assure you the intention was to call out replaceState also. I know this because I was directly involved in that conversation years ago when the APIs were new. I've filed https://github.com/whatwg/html/issues/982 to update this. 1180394 2 james.keane 2016-04-04 09:58:12 -0700 (In reply to comment #1) > The current history item really only needs to be updated exactly once - Right before the current history item moves to a new history item. While that may be the case, it goes against the spirit of the API i.e. of allowing the state or URL to be replaced when convenient, otherwise why provide a method at all? What you are describing may in fact be a better way to fix this issue in WebKit itself while hiding platform specific memory limitations. Not being familiar with WebKit myself, how is this: > for(var i = 0; i < 1000; i++) document.location.hash = String(i); limited, protected, or otherwise made 'safe' in a similar manner? As for Section 7.7.4 it states (emphasis mine): > In addition, a user agent could *ignore calls* to pushState() that are invoked on a timer, or from event listeners that are not triggered in response to a clear user action, or that are invoked in rapid succession. Which is *not* WebKit's behaviour in this case, as it is not merely ignoring those calls. Also in Section 7.7.4, the paragraph you reference, labelled 'Security', is discussing history hijacking, in which a nefarious page prevents a user from using the back button to leave a page; which does not appear relevant to this discussion. 1180691 3 beidson 2016-04-04 21:49:16 -0700 (In reply to comment #2) > (In reply to comment #1) > > > The current history item really only needs to be updated exactly once - Right before the current history item moves to a new history item. > > While that may be the case, it goes against the spirit of the API i.e. of > allowing the state or URL to be replaced when convenient, otherwise why > provide a method at all? I'm suggesting that replaceState only needs to be called once once. You're suggesting it's necessary to call it 60 fps. "When convenient" is somewhere in between and would likely be allowed by the current policy. > What you are describing may in fact be a better way to fix this issue in > WebKit itself while hiding platform specific memory limitations. WebKit cannot possibly know when the page is about to "fake navigate" via a pushState. Analyzing a program to try to figure that out is subject to the halting problem. Only the web author can know this. > Not being familiar with WebKit myself, how is this: > > for(var i = 0; i < 1000; i++) document.location.hash = String(i); > limited, protected, or otherwise made 'safe' in a similar manner? A browser engine can't stop JS programmers from doing *all* unreasonably inefficient things. It can step in to stop particularly nefarious ones, though. > > As for Section 7.7.4 it states (emphasis mine): > > In addition, a user agent could *ignore calls* to pushState() that are invoked on a timer, or from event listeners that are not triggered in response to a clear user action, or that are invoked in rapid succession. > > Which is *not* WebKit's behaviour in this case, as it is not merely ignoring > those calls. So you're suggesting it's the exception being thrown that is the problem, and not the policy itself? If so, we should probably close this bug and restart that conversation with a clean slate, because that's a different conversation. > Also in Section 7.7.4, the paragraph you reference, labelled 'Security', is > discussing history hijacking, in which a nefarious page prevents a user from > using the back button to leave a page; which does not appear relevant to > this discussion. Apologies for suggesting that I was only referencing that paragraph - I'm references the entire section. If we're citing sections of the spec with emphasis, though, I'll emphasize 2.2.1 (https://html.spec.whatwg.org/multipage/infrastructure.html#conformance-classes): > User agents *may impose implementation-specific limits on otherwise unconstrained inputs*, e.g. to prevent denial of service attacks, to guard against running out of memory, or to work around platform-specific limitations. And 8.1.3.5 (https://html.spec.whatwg.org/multipage/webappapis.html#killing-scripts): > ...it's sometimes necessary to abort a running script. > ... > User agents may impose resource limitations on scripts, for example CPU quotas, memory limits, total execution time limits, or bandwidth limitations. When a script exceeds a limit, the user agent may either throw a QuotaExceededError exception QuotaExceededError when a script exceeds a limit is what we're doing. Again, if the objection is to the exception as opposed to the limit itself, we should discuss in a clean slate bug. 1180817 4 james.keane 2016-04-05 08:50:49 -0700 (In reply to comment #3) You've convinced me, that the spec is flexible enough to allow user agents to impose limits on unreasonable usage. I will concede that it is on your authority to define what is and what isn't 'unreasonable' based on, as Section 2.2.1 calls it, 'implementation-specific limits'. Brady, I would also like to personally apologize, if my initial tweets offended you; I was frustrated at what, at the time, I believed was a spec violating change causing hundreds of thousands of users to see a completely broken site. I will leave it to you to decide whether throwing a QuotaExceededError, or as the spec _suggests_ ignoring calls is the preferred behaviour; my opinion is the latter. Either way, I think we can both agree that finding a solution which is not subject to these limits is preferable. > WebKit cannot possibly know when the page is about to "fake navigate" via a pushState. It can, and it is definitely *not* a halting problem; it knows when the page is about to fake navigate, as it knows when pushState is called. Again, not knowing the specifics, how could 'replaceState' create an OOM situation? Is it not deallocating the previous state object? 1181026 5 beidson 2016-04-05 16:14:20 -0700 Re-ordering my reply a bit to make a better narrative: (In reply to comment #4) > (In reply to comment #3) > Brady, I would also like to personally apologize, if my initial tweets > offended you; I was frustrated at what, at the time, I believed was a spec > violating change causing hundreds of thousands of users to see a completely > broken site. Apology noted, but not necessary; I wasn't offended, I just wanted to probe the use case and get to the bottom of the problem. > I will leave it to you to decide whether throwing a QuotaExceededError, or > as the spec _suggests_ ignoring calls is the preferred behaviour; my opinion > is the latter. We will definitely consider ignoring the call, but logging to the console anyways. > Again, not knowing the specifics, how could 'replaceState' create an OOM > situation? Is it not deallocating the previous state object? Yes, replaceState deallocates the previous state objects, but this policy is not about OOM. Calling replaceState at a high frequency, even with tiny payloads, does a surprising amount of work. It burns through a surprising amount of CPU, which means a surprising amount of battery. And if the frequency is high enough it actually quickly turns into a DOS attack. Not only on the current web page but on the entire browser, as the browser itself is necessarily involved with session history management. > > WebKit cannot possibly know when the page is about to "fake navigate" via a pushState. > It can, and it is definitely *not* a halting problem; it knows when the page > is about to fake navigate, as it knows when pushState is called. That's absolutely true. Let me rephrase: WebKit can't do analysis about each replaceState to see if it is the "last replaceState that is needed before a pushState" I believe what you're considering an appropriate fix in WebKit would be remember the most recent "replaceState" in a lightweight manner, then only do the heavyweight work when pushState is called or a navigation happens. We considered that at first, but there are significant problems with it, both technical and architectural. 1181029 6 beidson 2016-04-05 16:16:10 -0700 Retitling to: Change frequency limit on replaceState to fail without throwing an exception The goal being: - Both the size and frequency limits on pushState will remain, with exception. - The size limit on replaceState will remain, with exception. - The frequency limit on replaceState will cause replaceState to silently fail (with a one-time log to the console) 275427 2016-04-01 13:54:32 -0700 2016-04-01 13:54:32 -0700 Chat example - must be served over http for security restrictions chat.html text/html 3869 james.keane PCFET0NUWVBFIGh0bWw+CjxodG1sPgo8aGVhZD4KICA8dGl0bGU+Q2hhdCBhcHAgZXhhbXBsZTwv dGl0bGU+CiAgPHN0eWxlPgogIGxpIHsKICAgIGxpc3Qtc3R5bGUtdHlwZTogbm9uZTsgCiAgICB0 cmFuc2l0aW9uOiBiYWNrZ3JvdW5kIDVzIGVhc2Utb3V0OwogICAgYmFja2dyb3VuZDogI2ZmZmZm ZjsKICB9CiAgc3BhbiB7IHBhZGRpbmc6IDEwcHg7IH0KICAudW5yZWFkIHsKICAgIGJhY2tncm91 bmQ6ICNGRkUwRTA7CiAgfQogICNjb3ZlciB7CiAgICBkaXNwbGF5OiBub25lOwogICAgcG9zaXRp b246IGFic29sdXRlOyB0b3A6IDA7IGxlZnQ6IDA7IGJvdHRvbTogMDsgcmlnaHQ6IDA7CiAgICBi YWNrZ3JvdW5kOiByZ2JhKDAsIDAsIDAsIDAuNSk7CiAgICB0ZXh0LWFsaWduOiBjZW50ZXI7CiAg ICBmb250LXNpemU6IDRlbTsKICAgIGNvbG9yOiB3aGl0ZTsKICB9CiAgPC9zdHlsZT4KPC9oZWFk Pgo8Ym9keT4KICA8dWwgaWQ9ImNoYXRzIj4KICAgIDxsaT48c3BhbiBjbGFzcz0idXNlciI+am9l eTwvc3Bhbj48c3BhbiBjbGFzcz0iY2hhdCI+RGlkIHlvdSBndXlzIHNlZSB0aGUgbmV3IHRlc2xh PyE8L3NwYW4+PC9saT4KICA8L3VsPgogIDxkaXYgaWQ9ImNvdmVyIj5Mb3N0IGZvY3VzPGRpdj4K ICA8c2NyaXB0PgogICAgdmFyIGNoYXRzID0gZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ2NoYXRz Jyk7CiAgICB2YXIgc3RhdGUgPSBoaXN0b3J5LnN0YXRlIHx8IHsKICAgICAgbWVzc2FnZXM6IFtd LAogICAgICB1bnJlYWQ6IFtdCiAgICB9OwoKICAgIGZ1bmN0aW9uIGluc2VydE1lc3NhZ2UobWVz c2FnZSwgdW5yZWFkKSB7CiAgICAgIHZhciBlbCA9IGRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoJ2xp Jyk7CiAgICAgIGVsLmlkID0gbWVzc2FnZS5pZDsKICAgICAgZWwuaW5uZXJIVE1MID0gJzxzcGFu IGNsYXNzPSJ1c2VyIj4nICsgbWVzc2FnZS51c2VyICsgJzwvc3Bhbj4nICsKICAgICAgICAgICAg ICAgICAgICAgJzxzcGFuIGNsYXNzPSJjaGF0Ij4nICsgbWVzc2FnZS5jaGF0ICsgJzwvc3Bhbj4n OwogICAgICBpZiAodW5yZWFkKSB7CiAgICAgICAgZWwuc2V0QXR0cmlidXRlKCdjbGFzcycsICd1 bnJlYWQnKTsKICAgICAgfQogICAgICBjaGF0cy5hcHBlbmRDaGlsZChlbCk7CiAgICB9CgogICAg Ly8gUmVjaWV2ZSBhIGNoYXQgbWVzc2FnZSBpbiAncmVhbCB0aW1lJwogICAgdmFyIGhhc0ZvY3Vz ID0gdHJ1ZTsKICAgIHZhciBuZXh0SWQgPSAwOwogICAgZnVuY3Rpb24gcmVjaWV2ZUNoYXQobWVz c2FnZSkgewogICAgICBtZXNzYWdlLmlkID0gJ20tJyArIG5leHRJZCsrOwogICAgICBzdGF0ZS5t ZXNzYWdlcy5wdXNoKG1lc3NhZ2UpOwoKICAgICAgaWYgKCFoYXNGb2N1cykgewogICAgICAgIHN0 YXRlLnVucmVhZC5wdXNoKG1lc3NhZ2UuaWQpOwogICAgICAgIGRvY3VtZW50LnRpdGxlID0gJygn ICsgc3RhdGUudW5yZWFkLmxlbmd0aCArICcpIFVucmVhZCAtIENoYXQgYXBwIGV4YW1wbGUnOwog ICAgICB9CgogICAgICBoaXN0b3J5LnJlcGxhY2VTdGF0ZShzdGF0ZSwgJycsIHdpbmRvdy5sb2Nh dGlvbi5ocmVmKTsKICAgICAgaW5zZXJ0TWVzc2FnZShtZXNzYWdlLCAhaGFzRm9jdXMpOwogICAg fQoKICAgIGlmICghaGlzdG9yeS5zdGF0ZSkgewogICAgICAvLyBzaW11bGF0ZSBhIGNoYXQKICAg ICAgdmFyIG1lc3NhZ2VzID0gWwogICAgICAgIHsgdXNlcjogJ21pa2UnLCBjaGF0OiAnd2hvYScg fSwKICAgICAgICB7IHVzZXI6ICdmb29iYXInLCBjaGF0OiAnbG9va3Mgd2VpcmQnIH0sCiAgICAg ICAgeyB1c2VyOiAnZW5yb24nLCBjaGF0OiAnb2lsIGlzIHRoZSBmdXR1cmUnIH0sCiAgICAgICAg eyB1c2VyOiAnZ3JlZW5wZWFjZScsIGNoYXQ6ICdAZW5yb24sIGppZyBpcyB1cCcgfSwKICAgICAg ICB7IHVzZXI6ICdjaHJpcycsIGNoYXQ6ICdzbyBhd2Vzb21lIGNhbnQgd2FpdCcgfSwKICAgICAg ICB7IHVzZXI6ICdmYWJpbycsIGNoYXQ6ICdtZWgnIH0sCiAgICAgICAgeyB1c2VyOiAnZ2F0ZXMn LCBjaGF0OiAnQGVsb24sIGJ1dCBkb2VzIGl0IGN1cmUgbWFsYXJpYT8nIH0sCiAgICAgICAgeyB1 c2VyOiAnd2lsbGl0YmxlbmRndXknLCBjaGF0OiAnYnV0IHdpbGwgaXQgYmxlbmQnIH0sCiAgICAg ICAgeyB1c2VyOiAnam9obicsIGNoYXQ6ICcjZWxlY3RyaWNmdXR1cmUnIH0sCiAgICAgICAgeyB1 c2VyOiAnZWxvbicsIGNoYXQ6ICdhcHJpbCBmb29scywgYm96b3MnIH0sCiAgICAgICAgeyB1c2Vy OiAnamVyc2gnLCBjaGF0OiAnR09UIEVNJyB9LAogICAgICAgIHsgdXNlcjogJ2JpbGwnLCBjaGF0 OiAnI25lZWRzbW9yZXRydWNrJyB9LAogICAgICAgIHsgdXNlcjogJ2RhdmUnLCBjaGF0OiAnaG93 IGZhc3QgaXMgaXQ/JyB9LAogICAgICAgIHsgdXNlcjogJ25wbScsIGNoYXQ6ICdob3BlIGl0IGRv ZXNudCBkZXBlbmQgb24gbGVmdC1wYWQnIH0sCiAgICAgICAgeyB1c2VyOiAncnN0YWxsJywgY2hh dDogJ0BlbG9uLCB5b3UgcmVsZWFzZWQgdGhlIHNvdXJjZXMgeWV0PycgfSwKICAgICAgICB7IHVz ZXI6ICdzZXJnZXknLCBjaGF0OiAnYnV0IGRvZXMgaXQgZHJpdmUgaXRzZWxmPycgfQogICAgICBd OwoKICAgICAgbWVzc2FnZXMuZm9yRWFjaChmdW5jdGlvbihtZXNzYWdlKSB7CiAgICAgICAgc2V0 VGltZW91dChyZWNpZXZlQ2hhdC5iaW5kKG51bGwsIG1lc3NhZ2UpLCBNYXRoLnJhbmRvbSgpICog MjUwMCk7CiAgICAgIH0pOwoKICAgICAgc2V0VGltZW91dChzaW11bGF0ZUxvc3RGb2N1cywgMTAw MCk7CiAgICAgIHNldFRpbWVvdXQoZnVuY3Rpb24oKSB7CiAgICAgICAgZG9jdW1lbnQuZ2V0RWxl bWVudEJ5SWQoJ2NvdmVyJykuaW5uZXJIVE1MID0KICAgICAgICAgICAgJ3JlZnJlc2ggcGFnZSwg b3IgZ28gc29tZXdoZXJlIDxhIGhyZWY9Imh0dHBzOi8vdHJhYy53ZWJraXQub3JnL2NoYW5nZXNl dC8xOTg2ODciPmVsc2U8L2E+IGFuZCBjb21lIGJhY2sgZm9yIHNhdmVkIHN0YXRlLicKICAgICAg fSwgMzAwMCkKCiAgICAgIGZ1bmN0aW9uIHNpbXVsYXRlTG9zdEZvY3VzKCkgewogICAgICAgIGhh c0ZvY3VzID0gZmFsc2U7CiAgICAgICAgZG9jdW1lbnQuZ2V0RWxlbWVudEJ5SWQoJ2NvdmVyJyku c3R5bGUuZGlzcGxheSA9ICdibG9jayc7CiAgICAgIH0KICAgIH0gZWxzZSB7CiAgICAgIC8vIHJl bG9hZCBzdGF0ZQogICAgICBpZiAoc3RhdGUudW5yZWFkLmxlbmd0aCA+IDApIHsKICAgICAgICBk b2N1bWVudC50aXRsZSA9ICcoJyArIHN0YXRlLnVucmVhZC5sZW5ndGggKyAnKSBVbnJlYWQgLSBD aGF0IGFwcCBleGFtcGxlJzsKICAgICAgfQoKICAgICAgc3RhdGUubWVzc2FnZXMuZm9yRWFjaChm dW5jdGlvbihtKSB7IGluc2VydE1lc3NhZ2UobSk7IH0pOwogICAgICBzdGF0ZS51bnJlYWQuZm9y RWFjaChmdW5jdGlvbihpZCkgewogICAgICAgIHZhciBlbCA9IGRvY3VtZW50LmdldEVsZW1lbnRC eUlkKGlkKTsKICAgICAgICBlbC5zZXRBdHRyaWJ1dGUoJ2NsYXNzJywgJ3VucmVhZCcpOwogICAg ICAgIHNldFRpbWVvdXQoZnVuY3Rpb24oKSB7IGVsLnNldEF0dHJpYnV0ZSgnY2xhc3MnLCAnJyk7 IH0sIDApOwogICAgICB9KTsKICAgICAgc2V0VGltZW91dChmdW5jdGlvbigpIHsKICAgICAgICAv LyBpbiByZWFsIGFwcCwgbWVzc2FnZXMgd291bGQgc3RpbGwgYmUgY29taW5nIGluIGJ1dCBpdCBk b2Vzbid0IG1hdHRlcgogICAgICAgIC8vIGZvciB0aGUgZXhhbXBsZQogICAgICAgIHN0YXRlLnVu cmVhZCA9IFtdOwogICAgICAgIGRvY3VtZW50LnRpdGxlID0gJ0NoYXQgZXhhbXBsZSc7CiAgICAg ICAgaGlzdG9yeS5yZXBsYWNlU3RhdGUoc3RhdGUsICcnLCB3aW5kb3cubG9jYXRpb24uaHJlZik7 CiAgICAgIH0sIDMwMDApOwogICAgfQogIDwvc2NyaXB0Pgo8L2JvZHk+CjwvaHRtbD4=