Bug 209563 – Support longterm persistent storage (re: default ITP 7 rolling days of browser use expiry)

• Status: NEW
• Component: WebKit Misc.

At 1Password we are concerned about the sudden announcement (https://webkit.org/blog/10218/full-third-party-cookie-blocking-and-more/) that LocalStorage will expire after 7 days, and want to provide our use case of browser storage and how this will cause harm to our users (including irreversible data loss to some).

We have a full featured web app at 1Password.com that allows users to signup for an account, access their vaults, and perform admin functions for their business.

For context, 1Password is end-to-end encrypted with keys we do not hold. For users to decrypt their 1Password vaults, they need their Master Password and Secret Key: https://support.1password.com/secret-key-security/

The Secret Key is a randomly generated string generated locally during signup and stored on devices users have previously used. This key is required for users to decrypt their vaults. Given that most signups occur in a web browser, that is the first place we store this critical piece of data. While we do try to encourage users to install our native applications, for some (unknown) number of users out there LocalStorage in their browser will be the only place they have this key saved, and in 7 days may now become irreversibly locked out of their 1Password vaults.

Again because 1Password is end-to-end encrypted, unlike most sites, we cannot use authentication cookies to remember a user. We must use LocalStorage since our servers can never know the user's Secret Key. Additionally, a device UUID and 2FA token, are kept in LocalStorage. This means that returning users will often be met with an empty login screen, requiring them to again enter their email, locate and enter their 34 character Secret Key, and complete 2FA again. I think it's unreasonable to expect every user to visit our app at least once every 7 days, especially for business account administrators who may visit sporadically depending on which administrative actions happen to need performing.

We also store an additional 20+ user settings and other state information that will often get erased. It will be unexpected behaviour for users who may have to constantly reconfigure their preferred settings, or see notices they've previously dismissed.

Until now the only way for us to lose all this data stored in the browser is if the user explicitly chose to erase it, which is a pretty intentional action. To unexpectedly hear all this user data will be suddenly erased in less than 7 days from user's devices is very concerning, and will cause confusion and harm to our 1Password customers.

[John Wilander]

Thanks for filing! We appreciate it.

Just to explore the space a little bit.

I loaded your website https://support.1password.com/secret-key-security/ and had a look at scripts running. It seems you have a third-party script running on the site which allows that third party to read, write, and exfiltrate all data stored in your first party space. It can also monitor the user's behavior and read all data the user enters into form fields and such.

Third-party scripts using the first party storage space is why we've made this change, as you probably saw in our blog post.

The page's content security policy opens up for requests to a series of third parties which could all be endpoints for data exfiltration. Maybe it's a site-wide policy?

I'm trying to see if there's a way to allow websites to store things long term while still protecting the user from cross-site tracking. Third-party scripts and requests to third-party domains are a concern in that analysis.

What are your thoughts?

Thanks for the quick reply!

I totally agree with you regarding the dangers of third party scripts. In our web app itself which you can find at https://my.1password.com/ we do not allow any third-party scripts for the reasons you mentioned. (And we do plan to expand this policy to even our support site in the future.)

[Radar WebKit Bug Importer]

<rdar://problem/60893511>

[Maciej Stachowiak]

If the user signed up in the browser on one device, how do they access that account from another device? Do they need access to the original browser?

Also, does that mean clearing history could end up causing a lockout?

(Trying to understand a bit more of how this works.)

> If the user signed up in the browser on one device, how do they access that account from another device? Do they need access to the original browser?

That's right. To sign-in from a new device you need your Secret Key, and the primary method of finding that would be getting it from one of your existing browsers or native apps.

Again, we try to push users into the native apps where we have some better options for storage (like the native keychains). But there will be some users out there who have opted to use 1Password primarily with the web app.

> Also, does that mean clearing history could end up causing a lockout?

Yes, if a user explicitly chooses to clear their browser storage it could potentially cause lockout.

[[e-mail]]

Thanks for the hard work increasing privacy and security around the web.

I'd like to join in on this ticket and share how the sudden expiration of local storage will impact our customers in Simplenote.

At Automattic we build and maintain Simplenote across mobile, desktop, and web platforms so that people can take notes from anywhere with any network connection status and know that their edits will appear when their separate devices eventually synchronize.

In order to support this peace-of-mind for users of our web app we store pending edits in indexedDB and those remain queued up for whenever network connectivity returns, if it fails at the time of making the edits.

It's normal for our customers to make changes while offline and then come back more than a week later to synchronize them. While this may sound rare or outlandish it does happen quite regularly.

Clearing the storage will cause our customers to lose their edits and there will be no explanation or audit trail to find out why. Because the only time where persisting in storage is crucial is when network connectivity is lost there are no other ways for us to indicate that this loss occurred - as far as I understand the webkit blog post there is no possible way for us to hold on to those edits or indicate that they are lost due to the loss of storage.

If someone clears their browser cache they will experience the same impact but the nature of the clearing is categorically different: someone who triggers a reset will have made the intentional act to clear local state and hopefully will have that action in their memory when they question why they lost their data. Someone who happens to login and find that their changes are lost without any intervention will be confused.

A week is a sharp deadline. I think that even a month would go pretty far to alleviate the impact of this change on our customers, as one or two week vacations are pretty normal, even while month-long vacations are rare but legitimate.

We'd love to know if there are any ways to avoid this interruption to our customers or if there could be some way to trust a site so that the storage could persist.

Dashlane is also interested at this conversation.

We are admirative of Apple work to increase privacy and security. By the way, clearing LocalStorage after 7 days of inactivity has an impact similar to what our colleagues at 1Password mentioned. 

We have a fully working web app that user can use to create and use their account in autonomy, at app.dashlane.com. We also have a second web app, used by B2B administrators to monitor and configure their B2B accounts, monitoring adoption and security of their company users, at console.dashlane.com. 

User's data are encrypted and stored locally in IndexedDB from the first login on. If users paid for a premium account, their encrypted data are also stored on our servers, and they can access their data from any other device. Our free users don't have the synchronization across devices, and even if we do backup their encrypted data on our servers, clearing the cache after 7 days means that they will lose their data: they can recover it, but they need to contact our user support. And this is the worst scenario our users can experience: data loss.

Moreover, clearing up the cache impacts all of our users (free, premium and B2B) by adding unintentional friction to their experience: every time users access their account from a new device, we ask them to authorize it through a second factor authentication system. Again, clearing up their cache would mean asking our users to re-authorize the device at next login after cache clearing.

[Ramon]

At Learnbeat we experience the same problem. (https://login.learnbeat.com)

We are an online learning environment served as a web application that gives teachers and students the ability to use digital tools in the classroom. 

Our core feature is the ability to save the student's answers to their questions. Through this we can give students feedback on their work and serve our teachers dashboards displaying the results and progress of their students. 

We use localStorage to temporarily save the student's answers before they are synced to our DB. It is vital for us that a student can continue working with any network state, whether it is because WIFI systems on schools are notoriously poor or because we have an internal problem. 

Over the past months we have received numerous messages from students whose answers went missing and all of them used Safari. 
For us it's paramount that this is fixed. You don't want to give high school students an excuse to not make their homework and blame it on the system.. And above all, you just don't want to delete the hard work of these students. 

Increasing the cap for a longer amount would not really solve the problem, as also mentioned by other posters. As in any web application there are light users and intensive users. 

A better solution would be: making it possible to enable/disable the automatic deletion of the various storage types per domain, with all domains disabled by default. 

I really hope you can give this the attention it deserves. 
At Learnbeat we admire your work on privacy and we think it's not our role to advice users to move to other modern browsers. 
However, if we can't find a solution for this legitimate use of browser storage we don't see any other way. 

All the best, 

Ramon

[Sam Sneddon [:gsnedders]]

*** Bug 209501 has been marked as a duplicate of this bug. ***

[John Wilander]

You may have seen that home screen web apps on iOS and iPadOS have a carveout since last year. See "Home Screen Web Application Domain Exempt From ITP" here: https://webkit.org/blog/11338/cname-cloaking-and-bounce-tracking-defense/

[Sam Sneddon [:gsnedders]]

Changing the summary of the issue to generalise to what this issue is actually being used as.

Another example is Element (the Matrix client) which uses IndexedDB to store the encryption keys used by the session. While Element provides ways to export the keys (to the filesystem), I don't know how common this actually is for users to do. Losing access to prior DMs and/or other encrypted channels seems definitely less than ideal.

StorageManager.persist is one possible option, along with an explicit permission, but whether or not we think we can meaningfully communicate to the user the risks is a more involved UI question.

[Andy]

Wow, big names on this thread. It's hard to understand how there has been no movement on this.

Let me throw Sqrte Tasks onto the growing list. It is a local-first task management application.  It treats local storage as the primary data store and uses end-to-end encryption when syncing across distributed devices.

When I found out about ITP, I basically had to give up on Safari because the risk of data loss for my users was too great. I now have to direct them to other browsers. I spent many hours trying to find a way to make it work, but could not. I had some hope when I saw that Safari supposedly supported the Storage Manager Persisted API:
https://caniuse.com/?search=persisted
https://developer.mozilla.org/en-US/docs/Web/API/StorageManager/persisted
https://web.dev/articles/persistent-storage

This API works well in other browsers and effectively lets the user choose to keep data permanently for a given site. However, I quickly found through testing that, even though Safari seems to grant persistence through this API, ITP still blows away the site's data after 7 days.  Is this a bug?  Why does Safari implement this API but does not actually respect it? That seems really dangerous.

Yes, there's a workaround for iOS users (ie: provide a poor experience by requiring they install to the home screen), but as far as I can tell there is no solution for Desktop Safari!

I'm glad Apple is focusing on privacy, but surely there is a better way to achieve privacy than indiscriminately deleting users' data! Ironically, many of the apps reporting issues here (including Sqrte Tasks) are privacy minded.  Keeping data local is a great way to keep data private and ITP is effectively preventing a whole breed of Local First privacy minded applications from being used on Safari: https://www.inkandswitch.com/local-first/

I would love to support Safari. It would be AWESOME if the Persisted API actually worked.  FireFox has the best workflow I've seen for this API. When a site requests persistence, FF simply asks the user directly if they wish to have the site store data permanently. Amazing!  Is there any chance Safari could support a workflow like this?

Additionally, I'm really curious if anyone on this thread has found additional workarounds.

[Andy]

Would it be best if I open a new bug to convey that Safari's currently implemented StorageManager.persist api is broken due to ITP?

[John Wilander]

> but as far as I can tell there is no solution for Desktop Safari!

Web apps in the Dock on macOS have the same exemption from ITP website data deletion as Home Screen web apps on iOS.

[Andy]

> Web apps in the Dock on macOS have the same exemption from ITP

That is very welcome news! PWAs did not seem to be supported like this the last time I tested desktop Safari. Adding a web app to the dock appears to have been added with Safari 17 which released about a month ago.

Is there any official documentation around how ITP interacts with web apps that are added to the dock?  I'm not seeing anything when searching around. I would want to know if there are any potential gotchas before relying on this and the web app would need to be able to detect if it's running in this mode so that it can properly advise the user what to do otherwise.