Bug 277615 – IndexedDB failing with "UnknownError: Connection to Indexed Database server lost."

• Component: Website Storage
• Status: NEW

[Nisala]

Created attachment 472056 [details]
Example of exception in Sentry, showing iOS 17.6

My Capacitor (WebView in iOS app framework) app's connection to IndexedDB will spontaneously close and fail to reopen, throwing this error: "UnknownError: Connection to Indexed Database server lost. Refresh the page to try again".

Although this bug was closed last month saying it was fixed in iOS 17.6 (https://bugs.webkit.org/show_bug.cgi?id=273827), I am still getting Sentry reports for iOS 17.6 with this error. I've attached a screenshot of the exception from Sentry, and am happy to provide any more information that I can. Based on my Sentry reports, the bug still seems to happen when the user uses the app, puts it into the background for a few hours, and then brings it back to the foreground to use it again. This is a critical bug -- it can't be fixed without the user force-closing and relaunching the app, and as a result, I'm losing users (as I'm sure are many others).

[Radar WebKit Bug Importer]

<rdar://problem/133219094>

[Ahmad Saleem]

@Nisala - is it possible to attach small reproduction in order to help us understand the problem. All relevant engineers are in CC.

[Nisala]

I'll try my best to create a reproduction, but I still don't have a good way to reproduce the error consistently. It seems very much up to the whims of iOS. I was hoping that y'all would have a reproduction already from the earlier bug reports of this issue.

From what I've seen on Sentry, the bug happens when an app that uses IndexedDB via a Webview (app.getbaseline.baseline, in my case) is put into the background and then re-opened many hours later. When the app is reopened, the database is closed and can't be reopened, and throws the UnknownError.

I can confirm this issue as well (user reports, and firsthand) on iOS 17.6.

It is seemingly sporadic and I'm unable to reproduce on demand.

[Zarko]

Created attachment 472314 [details]
zarko - example exception log

See attached example exception log.
It seems like it might also have to with an event where there is no internet connection when the app comes back to foreground.

The exception is close to impossible to reproduce myself. For the time being it's clogging our error reporting system (blowing up our quota) and it's not clear what kind of runtime effect it has for users.

Given that the error message says "Connection to Indexed Database server lost. Refresh the page to try again", is it even worth trying to reload the (web) app? Would that recover the IndexedDB database/connection?

[Zarko]

If it may help, I can inform that we're using https://www.npmjs.com/package/idb-keyval, which works well until things go south with the IndexedDb and read/write operations starts to fail silently. I made an attempt there with a pull request ("Try to reopen database in case of unexpected closure" - https://github.com/jakearchibald/idb-keyval/pull/175) but the library project doesn't seem to well maintained...

I believe Dexie.js and Firebase have similar recovery logic in place, though it is probably only a partial solution - not enough if the db cannot be re-opened at all within the session.

[David Fahlander]

I can confirm that Dexie.js is not able to workaround this issue. Dexie.js has logic in place to re-execute transactions when certain errors happen. However, we've also tried whether reopening the database helps in this case but unfortunately it does not, making it impossible to work around this issue in Dexie.js or in IndexedDB-reliant apps. Doing a window.location.reload() reportedly helped resolving the issue for a customer of mine but it's quite harsh to do that without user confirmation.

Related Dexie issue: https://github.com/dexie/Dexie.js/issues/2008

Similar Ionic issue: https://github.com/ionic-team/ionic-storage/issues/317

[Lincoln Baxter, III]

Confirmed we are also still seeing this. Restarting the iPhone/iPad seems to be the only resolution that consistently brings the database back, for our affected user -- though I am not 100% on that detail. I haven't been able to reproduce this to confirm that force closing and re-opening the app fixes the issue or not.  It apparently hasn't worked for some.

Asking a user to restart their entire device is not a great solution for obvious reasons.

[Louis Spieckerhoff]

I can also confirm that our users get this error (capacitor). A workaround would be to reload the web-app, but that is against the intention of saving an app state when the app goes into background and load it when it goes into foreground (as native apps do).

A user expects to see what they last had seen when moving the app into background. Otherwise they will be confused and probably lose their intention why they opened the app in first place.

We have also difficulties to reproduce this error. Sometimes it happens like every 24hrs, sometimes it just happens once in a whole week.

[Nisala]

@Ahmad I have created a working minimal reproduction of this bug. I found that in order to reproduce the bug, I needed to not only have a WebView that used IndexedDB, but also enable and use Background Modes on the app (I used a background task for the reproduction, but I believe any background mode will cause this issue). 

In order to reproduce the bug, I simply opened and interacted with the app every ~six hours. This is unfortunately not a deterministic reproduction, but I can guarantee with fairly high certainty that within a few days, the bug will appear -- you will see an alert that says "Database forcibly closed", and then the button to add new items to the IndexedDB will stop working because the connection to the database has broken. Force-closing and relaunching the app fixes it.

I also took a sysdiagnose when I was able to reproduce the bug on my iPhone. If this would be useful to share, please let me know where I should send it. Happy to provide any additional help -- this has caused thousands of user crashes for my app, and I'm happy to do whatever I can to get this fixed as soon as possible.

https://github.com/nkalupahana/indexeddb-crash-repro
Files of note are AppDelegate.swift, App.tsx, and db.ts.

Still present iOS 18.1

[Louis Spieckerhoff]

We invited our users to do a survey which included questions about this problem.

78% of all users use our app daily; and 48% of them use an apple device (iPhone, iPad).
About 41% of those apple users also reported, that they often get logged out or need to set up the app again in order to use it. <- this relates to the storage problem

All questions were asked in relation to the last 6 months; e.g. "How often have you used our app in the last 6 months?"

Our app is part of a b2b solution and is used mostly by the employees of a business (our customer). Some employees are obligated to use our app in their daily business.
It is frustrating and time consuming for them because of this bug.

Please pay a little more attention to this one.

[Louis Spieckerhoff]

Created attachment 473234 [details]
List of devices with iOS 18.2 with asyncStorage error

Even with iOS 18.2 this error is still around... we will migrate to SQLite soon. The background runner will still us CapacitorKV which - i guess - still depends on localStorage, but thats okay for us. At least the user wont be making a bad experience anymore.

I ended up notifying the user to force close the app. It's a terrible user experience, but it's in Apple's hands.

Also AFAIK localStorage is unaffected. Just Indexeddb. Louis, what sqlite adapter are you using?

https://github.com/user-attachments/assets/9bb111ec-7523-4738-aa2c-2120fc8ad167

[Louis Spieckerhoff]

Sorry - meant IndexedDB. I will give the community one a try (https://github.com/capacitor-community/sqlite). Do you know a better one?

At the moment, we also show an alert to the user to force close the app and reopen...

The migration from IndexedDB to SQLite will be funny - not, I guess.

[Nisala]

I believe that the bug can be resolved just by reloading in the WebView, instead of needing to force close + re-open the app. I recently rolled out a change in my app that catches the error and reloads the page. If the error occurs again within 10 seconds, it sends me an error on Sentry, and so far I haven't received any errors (and I've stopped hearing about issues from my users) so I believe the solution works. Here's my implementation -- I'm using Dexie.js useLiveQuery, so I'm using a React error boundary to catch the error. https://github.com/nkalupahana/baseline/pull/414/files

This is by no means a good solution -- reloading the app is not an acceptable long-term solution to this problem. But it's not very noticeable on my app, so it's a good workaround for anyone else facing this problem while the bug remains unfixed.

We now also saw this in our logs although sometimes the message is "Attempt to get a record from database without an in-progress transaction" instead of " instead of "Connection to Indexed Database server lost."

We're still (or again?) seeing this on iOS 18.3 and 18.3.1. (We're using https://github.com/Kinto/kinto.js, which uses IndexedDB internally.)

Still happening on 18.3.2.

Still seeing this on iOS 18.4. Here are our failure counts by version in a recent random 24-hour period.
iOS 18.3 - 33635
iOS 18.4 - 32616
Mac OS X 10.15 - 7612
We have not been able to repro so we don't even know for sure what to tell customers. Do you reload the browser, restart the app, restart the phone? Can it be a transient issue for some? Is there a way you can repro this/force it to happen on a Mac OS X?

I am encountering the same issue, and implemented a workaround.
I use a window visibility callback to perform a DB healthcheck on visible, wrapped with a Promise that serve as a mutex
Any other code that need to run when the app become visible will wait on that Promise, so nothing will be executed if the DB is not accessible.
If the promise is rejected, it serve as a signal that the page will be soon reloaded, so state can be saved and restored after reload

If the DB is not accessible I reload the page after some delay

To prevent data loss, I use local store as a temp storage so it can be available after reload. This includes the exception I got back from the DB, so I can keep track of the errors

So far it seems as if the workaround is 100% reliable, however it would be great to see this issue fixed properly

[Lincoln Baxter, III]

(In reply to yonatan.goraly from comment #21)
> I am encountering the same issue, and implemented a workaround.
> I use a window visibility callback to perform a DB healthcheck on visible,
> wrapped with a Promise that serve as a mutex
> Any other code that need to run when the app become visible will wait on
> that Promise, so nothing will be executed if the DB is not accessible.
> If the promise is rejected, it serve as a signal that the page will be soon
> reloaded, so state can be saved and restored after reload
> 
> If the DB is not accessible I reload the page after some delay
> 
> To prevent data loss, I use local store as a temp storage so it can be
> available after reload. This includes the exception I got back from the DB,
> so I can keep track of the errors
> 
> So far it seems as if the workaround is 100% reliable, however it would be
> great to see this issue fixed properly

How do you reload the page in this scenario? What actual method do you call? Just want to make sure I am trying this the right way. Thanks for the potential workaround! I'm going to try this.

window.location.reload()

[Mark Langer]

We could observe that once the background WebKit Network Service crashes, a simple reload of the page does not resolve the issue. One has to fully quit Safari on MacOS and relaunch it. Then, the DB connection opens again.

As a user, this feels quite cumbersome. And buggy too.

[Laszlo]

Common Apple, I just saw you did improvements on webview for localstorage but nothing about this! It's been a full year. This is a really sad state that PWA's are left in the dust for this long.

[Mark Langer]

@Nisala: Could you update the Safari version 17 → 18 so that Apple has this issue on the radar when filtering for persisting bugs?

[Mark Langer]

@Apple, pretty pretty please? Can you at least assign someone that looks into this?

We've already posted some logs from MacOS hinting at possible reasons in issue https://bugs.webkit.org/show_bug.cgi?id=282093

Any brave engineers out there willing to tackle this bug?

[Laszlo]

(In reply to Mark Langer from comment #27)
> @Apple, pretty pretty please? Can you at least assign someone that looks
> into this?
> 
> We've already posted some logs from MacOS hinting at possible reasons in
> issue https://bugs.webkit.org/show_bug.cgi?id=282093
> 
> Any brave engineers out there willing to tackle this bug?

good luck, this goes against their native app push with their app store. This would be a conflict of interest for them so this will sit around for a long time before its dealt with or forced on them

(In reply to Louis Spieckerhoff from comment #15)
> Sorry - meant IndexedDB. I will give the community one a try
> (https://github.com/capacitor-community/sqlite). Do you know a better one?
> 
> At the moment, we also show an alert to the user to force close the app and
> reopen...
> 
> The migration from IndexedDB to SQLite will be funny - not, I guess.

Have you migrated to SQLite , if so have you adopted SQLite with OPFS wasm option. Does that helps in persisting storage better than indexedDB ?

[Louis Spieckerhoff]

(In reply to p.gunal333 from comment #29)
> (In reply to Louis Spieckerhoff from comment #15)
> > Sorry - meant IndexedDB. I will give the community one a try
> > (https://github.com/capacitor-community/sqlite). Do you know a better one?
> > 
> > At the moment, we also show an alert to the user to force close the app and
> > reopen...
> > 
> > The migration from IndexedDB to SQLite will be funny - not, I guess.
> 
> Have you migrated to SQLite , if so have you adopted SQLite with OPFS wasm
> option. Does that helps in persisting storage better than indexedDB ?

Yes sir, we have successfully migrated to SQLite with the default plugin as you mentioned: https://github.com/capacitor-community/sqlite

It takes a short moment to initialize the database connection; you should strictly follow the example implementations in the repo.

You should also consider to implement a queue for your queries, because penetrating the SQLite db with too many queries at the same time can lead to OOM on Android really fast.