Bug 168631 – Feature Request: Make partitioned localStorage persistent

• Status: NEW
• Component: WebKit Misc.

[Malte Ubl]

Safari behaves differently for localStorage accessed in third party context compared to other browsers in 2 fundamental ways:

1. Storage is scoped to the entire chain of iframes.
2. Storage is non-persistent. Data is deleted when browsers exit.

#1 seems great and working as intended, but while talking with John Wilander on Twitter we were wondering if #2 might be a bug. (https://twitter.com/johnwilander/status/833462485592125441)

Here is a simple test case http://output.jsbin.com/siwulo/1/quiet

- Press +1 a few times
- Hit reload. Observe that count is kept.
- Restart Safari (Desktop or iOS)
- Load page again. Observe that count is 0 again.

Is this behavior intended? On Desktop where Safari may be running for many months and on mobile where it is very unpredictable when it restarts, this seems unpredictable with little actual privacy benefit.

[Radar WebKit Bug Importer]

<rdar://problem/30624580>

[John Wilander]

Thanks for filing, Malte!

As can be seen in StorageNamespaceProvider::localStorageArea(Document& document), partitioned localStorage uses a transient storage space. I.e. it is never persisted and behaves like sessionStorage.

This is not a bug but also not correct behavior, rather "not yet implemented." I assume you'd like to change your bug into an enhancement/feature request?

[Malte Ubl]

Yes, it would be great to get this implemented.

We currently use a very complicated system in which the 1st party does the storage (leading to the exact same semantics as desired here) and would love to replace it with a simpler model.

[Brady Eidson]

(In reply to comment #3)
> Yes, it would be great to get this implemented.
> 
> We currently use a very complicated system in which the 1st party does the
> storage (leading to the exact same semantics as desired here) and would love
> to replace it with a simpler model.

Note that some other options (such as IndexedDB) work fine in 3rd party contexts today.

[Malte Ubl]

It appears that IDB is completely blocked in 3P context. See https://output.jsbin.com/lijagic/quiet

The error message is "open — LyNOje:43SecurityError (DOM Exception 18): IDBFactory.open() called in an invalid security context"

Are we missing something?

[Brady Eidson]

(In reply to Malte Ubl from comment #5)
> It appears that IDB is completely blocked in 3P context. See
> https://output.jsbin.com/lijagic/quiet
> 
> The error message is "open — LyNOje:43SecurityError (DOM Exception 18):
> IDBFactory.open() called in an invalid security context"
> 
> Are we missing something?

I can't quickly look at this test case and make heads or tails of what you're doing.

Can you provide the link to the source of a truly reduced example?

[Hongfei Ding]

Hi Brady, I've reduced the code (from Malte's test case).

See the following repro page, which contains only a xorigin iframe.
We expect to see "John" printed in the iframe (as in Chrome).

Test page: http://output.jsbin.com/lijagic
The iframe resource: https://codepen.io/lannka/pen/LyNOje

[Brady Eidson]

(In reply to Hongfei Ding from comment #7)
> Hi Brady, I've reduced the code (from Malte's test case).
> 
> See the following repro page, which contains only a xorigin iframe.
> We expect to see "John" printed in the iframe (as in Chrome).
> 
> Test page: http://output.jsbin.com/lijagic
> The iframe resource: https://codepen.io/lannka/pen/LyNOje

It works fine for me when Safari's settings are to "Always allow" cookies and website data.

With any other setting that blocks 3rd party storage, I agree - the 3rd party storage is blocked.

[Brady Eidson]

Let me stop the momentum of this bug in its tracks.

This bug was originally a request to make partitioned localStorage persistent, and had nothing to do with IndexedDB.

If somebody wants to file a different bug about some IndexedDB behavior, please file a different one.

In the meantime, back to the original purpose of this bug, I can state pretty authoritatively that partitioned localStorage is not persistent *by design*.

It is not a bug.

This bugzilla is asking for a change in behavior, and we need to actually think it through on its merits.

[Malte Ubl]

Not sure which momentum you mean. We asked about localStorage and *you* suggested to use IndexedDB instead waiting for persistent partitioned localStorage which John described as an unimplemented feature. Browser features that require non-default user settings are irrelevant as you likely understand.

We don't care about the type of storage. localStorage is fine. IndexedDB is fine. All we care about is to get a non-transient partitioned data store.

So, yeah, lets get back to the original request. It is a privacy invariant feature request since the result can already be achieved by storing the data on the 1p instead (which for example ads often have access to). We (for a non-ads use case) would like to not store data on the 1p, because we'd like to avoid leaking data from the 3p to the 1p to protect user privacy.

[Brady Eidson]

(In reply to Malte Ubl from comment #10)
> All we care about is to get a non-transient partitioned data store.

It's not obviously clear that having this be a default is in the user's best interest.

e.g. For IndexedDB that does support this, it's only supported when the user has explicitly allowed it.

Also, if we did implement persistent storage of partitioning for localStorage, it would almost certainly fall under the same umbrella of "only if the user explicitly allows it"

> It is a privacy invariant
> feature request since the result can already be achieved by storing the data
> on the 1p instead (which for example ads often have access to). We (for a
> non-ads use case) would like to not store data on the 1p, because we'd like
> to avoid leaking data from the 3p to the 1p to protect user privacy.

Can you better describe the mechanism by which 3rd parties store persistent data on the first party?

Knowing details will better help analyzing the tradeoffs between the current state of things and implementing the proposal.

[Malte Ubl]

In the current state, the 1p can store data on behalf on the 3p. Storage-wise this achieves the same semantics as partitioned storage.

Something like this on the 1p:

onmessage = e => {
  if (e.data.type == 'set-localstorage')
    localStorage.setItem(e.origin + ':' + e.data.key, e.data.value);
  else if (e.data.type == 'get-localstorage')
    localStorage.setItem(e.origin + ':' + e.data.key);
}

Of course, this requires cooperation of 1p and 3p, but especially in the ads case, 3ps often get to run script on the 1p, so they can set up the above without **real** cooperation.

If you'd like to maintain the current state of requiring 1p cooperation for storage in 3p context, this could be achieved by adding an opt-in attribute similar to allowfullscreen (e.g. allowstorage). That would maintain the exact same user and storage controls as today, but would not incentivize ads to run code in 1p context (which I think everyone would agree would be a great change).

Our primary motivation is that we'd like to avoid leaking data from the 3p to the 1p for increased privacy of our users.

[Brady Eidson]

(In reply to Malte Ubl from comment #12)
> 
> Our primary motivation is that we'd like to avoid leaking data from the 3p
> to the 1p for increased privacy of our users.

If I understand you correctly, you are primarily the 3rd party, and you are concerned about leaking your data to the 1st party?

[Brady Eidson]

(In reply to Malte Ubl from comment #12)
> If you'd like to maintain the current state of requiring 1p cooperation for
> storage in 3p context, this could be achieved by adding an opt-in attribute
> similar to allowfullscreen (e.g. allowstorage).

If I understand your suggestion, you are suggesting we add a way for 1st parties to override the user's wish that 3rd party storage be blocked?

[Malte Ubl]

Please do not put words in my mouth. All I did, was file a bug because John Wilander suggested that there might be a bug. I'd appreciate the bug being fixed, because it'd allow me to delete a lot of code, and increase user privacy.

What I am suggesting it to add a feature that provides additional privacy control by avoiding data leakage from the 3p to the 1p. This does not add any capabilities that aren't already possible. It is a strict privacy win for users.

In our case we are both the 1p AND the (technical) 3p (the iframe is on a different origin from the 1p window for security reasons), but we want to avoid that the technical 3p having to supply data to the 1p to maintain strict separation of data.

[Brady Eidson]

(In reply to Malte Ubl from comment #15)
> Please do not put words in my mouth. 

I don't believe I did - I asked two clarifying questions to make sure I understood precisely your meaning. This is why my comments were in the form questions.

> All I did, was file a bug because John Wilander suggested that there might be a bug. 

I understand that. While he suspected there might be a bug, we've established at this point that there's no bug. WebKit is behaving as designed.

> I'd appreciate the bug being fixed, because it'd allow me to delete a lot of code, and increase user
> privacy.

Again, we haven't reached an understanding on how making 3rd party storage persistent when the user has blocked 3rd party storage increases their privacy.

I'm trying to understand how this helps the user, but if the description of how it helps them is somewhere in this discussion I missed it.

> What I am suggesting it to add a feature that provides additional privacy
> control by avoiding data leakage from the 3p to the 1p. 
> This does not add any capabilities that aren't already possible. 

But... you are directly asking for something that is currently impossible; For persistent 3rd party storage when the user has blocked 3rd party storage.

> It is a strict privacy win for users.

As mentioned above, I don't understand how making 3rd party storage persistent when the user has blocked 3rd party storage is a strict privacy win for users.

> In our case we are both the 1p AND the (technical) 3p (the iframe is on a
> different origin from the 1p window for security reasons), but we want to
> avoid that the technical 3p having to supply data to the 1p to maintain
> strict separation of data.

I understand this is what you want.

[Malte Ubl]

It seems a valid interpretation of user intent that 1p partionioned 3p storage is effectively 1p storage for the reasons I outlined above. Users are helped because sites can implement strong security boundaries along origin boundaries that have separate storage.


In the end this is up to you, of course. I believe I presented a proposal that improves privacy for users and developer experience while respecting their intent.

If you interpret the user intent as literal, then maybe you cannot grant the privacy and developer experience benefit. That would be sad, but it sometimes happens.

[John Wilander]

(In reply to Malte Ubl from comment #15)
> Please do not put words in my mouth. All I did, was file a bug because John
> Wilander suggested that there might be a bug. I'd appreciate the bug being
> fixed, because it'd allow me to delete a lot of code, and increase user
> privacy.
> 
> What I am suggesting it to add a feature that provides additional privacy
> control by avoiding data leakage from the 3p to the 1p. This does not add
> any capabilities that aren't already possible. It is a strict privacy win
> for users.
> 
> In our case we are both the 1p AND the (technical) 3p (the iframe is on a
> different origin from the 1p window for security reasons), but we want to
> avoid that the technical 3p having to supply data to the 1p to maintain
> strict separation of data.

Hi Malte!

First, for clarification, this is how I interpret your 1st-party opt-in part:

1) Safari's default policy is to partition and not persist 3rd-party storage such as localStorage and IndexedDB.
2) The 1st-party can offer 3rd-parties to store persistently on their behalf through postMessage.
3) Since many 1st-parties run 3rd-party scripts, those 3rd-parties can set up postMessage-to-store themselves. And allowing 3rd-parties to run script in the 1st-party context can be interpreted as an opt-in to such storage.
4) If 1st-parties "opt in" by running 3rd-party scripts we might as well allow 3rd-parties to persist their partitioned storage under that 1st-party. This could be done through an opt-in attribute named allowStorage or similar.

Am I right so far?

If so, I believe our position is that 3rd-party storage is a user setting, not a 1st-party site setting. The postMessage-to-store practice is an unfortunate trick 3rd-parties can pull to track users but it at least requires them to convince the 1st-party to run their script. I understand the user win in less 3rd-party scripts running in the 1st-party context but I honestly doubt that any trackers are running scripts exclusively for this purpose. Maybe I'm wrong. Do you have experience and/or telemetry there?

Then there's your specific case where you are the 1st-party and the 3rd-party for security reasons. That's your choice, right? You have decided to serve AMP cached content under google.com which is an origin many users authenticate to and an origin that controls those users' email, documents, YouTube data, blogs et cetera. I understand that this creates a security challenge for you but you have full control over this. You can serve the AMP cache from a different origin with significantly less security risk to users, right?

As for the the original request, I interpreted it as you wanting partitioned localStorage to work as partitioned IndexedDB persistence-wise. But you also want a change in Safari's default storage policy now that we've established how things work. Am I right? If so, the default settings part should probably be handled in a separate bug/request.

[Malte Ubl]

This sounds right. I believe #1 is not quite correct, as under default setting IndexedDB throws while localStorage uses transient storage.

The AMP case you describe is a bit different: We serve content from its own origin. Google.com is only a container, so this is a standard application of the primary security primitive of the web (the origin model). We'd like to avoid exposing user data to google.com where we can. Persistent partitioned storage would help with that. Imagine you'd build a web browser (like Safari) and you could avoid that the browser (as opposed to the rendering engine) could ever read user data. Would you take that chance of extra security? Browsers other than Safari currently do provide web developers with a device to implement such a security model.

I think a good summary is:

We'd like to get persistent 1p-partitioned storage. Since the data is partitioned by "websites I visit" this seems to be in line with the default storage setting in Safari.  

If your interpretation is that under this setting all storage, including partitioned storage should be blocked, I'd suggest to close this ticket as working and intended and to

- document the behavior
- emit a warning on storage access for localStorage (which doesn't throw; I assume for webcompat reasons), so that developers are informed that Safari is violating the spec on purpose.
- consider to file a ticket under https://github.com/WICG/interventions

This ticket was already a big step forward for web developers as there appears to be at least one non-anecdotal source of information now for what the intended behavior of Safari is.

[Brady Eidson]

(In reply to Malte Ubl from comment #19)
> If your interpretation is that under this setting all storage, including
> partitioned storage should be blocked, I'd suggest to close this ticket as
> working and intended and to
> 
> - document the behavior

The WebKit project has documented it here, and in source code.

If you mean for Apple to document the behavior as seen in products Apple ships, you'll need to file a bug at developer.apple.com

> - emit a warning on storage access for localStorage (which doesn't throw; I
> assume for webcompat reasons), so that developers are informed that Safari
> is violating the spec on purpose.

I'm actually not clear on what part of the spec we're violating. Could you clarify?

https://html.spec.whatwg.org/multipage/webstorage.html#the-localstorage-attribute states:

> User agents should expire data from the local storage areas only for security reasons or when requested to do so by the user. User agents should always avoid deleting data while a script that could access that data is running.

We "expire data from the local storage area," both "for security reasons" and as "requested to do so by the user" once any script that could access the data can no longer access it.

[Brady Eidson]

(In reply to Brady Eidson from comment #20)
> 
> https://html.spec.whatwg.org/multipage/webstorage.html#the-localstorage-
> attribute states:
> 
> > User agents should expire data from the local storage areas only for security reasons or when requested to do so by the user. User agents should always avoid deleting data while a script that could access that data is running.

I thought there was more explicit language but didn't see it while writing the above comment.

Found it right after:
https://html.spec.whatwg.org/multipage/webstorage.html#user-tracking

"Expiring stored data"

> For example, a user agent could be configured to treat third-party local storage areas as session-only storage, deleting the data once the user had closed all the browsing contexts that could access it.

[Eli Grey (:sephr)]

Would/could this be eventually enabled by gating it through First Party Sets (or similar)?

[Jesper van den Ende]

We have created several games and host each on a dedicated domain, for instance https://raccoonretail.com/

These games are often embedded on game portals such as https://www.crazygames.com/game/raccoon-retail or https://www.silvergames.com/en/raccoon-retail
A large portion of our players come from portals such as these and over the years we've received several complaints that progress isn't saved.
I always thought these complaints were coming from users with obscure browsers or simply had their 'clear cookies when I close my browser' setting enabled, but I only found out today that WebKit does this for ALL of our games across all game portals.

What would be the best approach to solve this issue? Requesting every game portal to implement an api using `postMessage()` is not feasible (although some portals have done this).
The only thing I can think of is to use `document.requestStorageAccess()` but this shows a rather extreme pop up suggesting that we might start tracking the users activity. Which is the last thing we want to do, we just want the user to keep their progress :(

Is there really no other way?

[Jesper van den Ende]

Hmm I didn't realize `requestStorageAccess()` doesn't work unless the page has been opened in a first party context at least once. So I suppose this is not a viable solution.
At this point I'm worried the only thing I can do is to ask the user to disable the 'Prevent Cross-Site Tracking' setting :/