Bug 197050 - IndexedDB: An internal error was encountered in the Indexed Database server
Summary: IndexedDB: An internal error was encountered in the Indexed Database server
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: JavaScriptCore (show other bugs)
Version: Safari 12
Hardware: Unspecified iOS 12
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
: 224804 (view as bug list)
Depends on: 196372 215239
Blocks:
  Show dependency treegraph
 
Reported: 2019-04-17 21:59 PDT by mrschmidt
Modified: 2021-06-18 08:16 PDT (History)
33 users (show)

See Also:


Attachments
com.apple.WebKit.Networking crash report for IndexedDb background issue (43.34 KB, text/plain)
2019-05-07 11:21 PDT, Michael Lehenabuer
no flags Details
Failed to execute 'transaction' on 'IDBDatabase': The database connection is closing. (56.74 KB, text/plain)
2020-08-04 07:05 PDT, Istvan Tabanyi
no flags Details
UnknownError: An internal error was encountered in the Indexed Database server (50.30 KB, text/plain)
2020-08-19 14:40 PDT, Simon Bluhm
no flags Details
iOS_14_5_com.apple.WebKit.Networking-2021-04-15-121002.ips (60.69 KB, text/plain)
2021-04-20 03:10 PDT, Tim
no flags Details
iOS_12_4_com.apple.WebKit.Networking-2021-04-13-110655.ips (44.50 KB, text/plain)
2021-04-20 03:11 PDT, Tim
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description mrschmidt 2019-04-17 21:59:33 PDT
We have received reports from the users of the Firebase SDK for Firestore (see https://github.com/firebase/firebase-js-sdk) that they are seeing the following exception starting with iOS 12:

"An internal error was encountered in the Indexed Database server"

The user report states that bringing the Web app into the background and back into the foreground triggers this crash. More details can be found here: https://github.com/firebase/firebase-js-sdk/issues/1670

There are some other reports that this issue is related to storing Uint8Arrays in Object Stores that use auto-incrementing keys. While we do use auto-incrementing keys in Firestore, we do not store Uint8Arrays.
Comment 1 Radar WebKit Bug Importer 2019-04-17 22:11:25 PDT
<rdar://problem/50005532>
Comment 2 David Fahlander 2019-04-25 16:19:30 PDT
Got Dexie issue related to this one: https://github.com/dfahlander/Dexie.js/issues/832
Comment 3 David Fahlander 2019-04-25 16:21:31 PDT
Related Cordova issue:

https://github.com/ionic-team/cordova-plugin-ionic-webview/issues/354
Comment 4 Michael Lehenabuer 2019-04-25 17:51:07 PDT
This problem is being reported by more and more Firestore customers. I've created a small repro page that just creates an IndexedDb transaction, does 10 writes, and repeats every 100ms.  If you send safari to the background repeatedly while it's running, you'll hit the bug easily.

Repro steps:

1. Open http://mike-shared.firebaseapp.com/ios-crash.html in iOS 12.2 Safari 
2. Press Home button.
3. Open safari again (just tap the app icon).
4. Repeat steps 2 and 3 until number on the page stops incrementing. On my iPhone 6S it usually happens within ~5 tries; most I've had to do is ~12.

Result:
Eventually the number on the page will stop incrementing. If you look in the JS Console you'll see:

[Error] put onerror – UnknownError: An internal error was encountered in the Indexed Database server
	(anonymous function) (ios-crash.html:63)

At this point, IndexedDb is completely broken.  Subsequent transactions fail saying that IndexedDb is closing, and if you try to open IndexedDb again, the request just hangs.

Let me know if you need anything else to investigate.  Thanks!
Comment 5 Michael Lehenabuer 2019-04-30 10:14:45 PDT
As an update, I've tried a couple workarounds which do not work:

1. Re-opening the database after hitting this bug results in an "UnknownError: Connection to Indexed Database server lost. Refresh the page to try again."

2. Listening to visibilitychange events and attempting to stop queuing operations when visibilitystate == false does not work because in the case where the bug repros, no visibilitychange event is received before Safari is paused in the background.  The event arrives only after restoring Safari to the foreground, but then it is too late and you get the "An internal error was encountered in the Indexed Database server" error.

I've uploaded a modified repro at http://mike-shared.firebaseapp.com/ios-crash2.html which contains these workarounds and additional logging.

Given there are seemingly no workarounds, it would be very helpful if somebody could acknowledge that this bug has been seen and is being investigated. Thanks!
Comment 6 tgangso 2019-05-06 07:19:29 PDT
This is a serious issue and should get top priority.
Comment 7 Sihui Liu 2019-05-06 09:06:20 PDT
(In reply to Michael Lehenabuer from comment #5)
> As an update, I've tried a couple workarounds which do not work:
> 
> 1. Re-opening the database after hitting this bug results in an
> "UnknownError: Connection to Indexed Database server lost. Refresh the page
> to try again."
> 
> 2. Listening to visibilitychange events and attempting to stop queuing
> operations when visibilitystate == false does not work because in the case
> where the bug repros, no visibilitychange event is received before Safari is
> paused in the background.  The event arrives only after restoring Safari to
> the foreground, but then it is too late and you get the "An internal error
> was encountered in the Indexed Database server" error.
> 
> I've uploaded a modified repro at
> http://mike-shared.firebaseapp.com/ios-crash2.html which contains these
> workarounds and additional logging.
> 
> Given there are seemingly no workarounds, it would be very helpful if
> somebody could acknowledge that this bug has been seen and is being
> investigated. Thanks!

Hi Michael,

Thank you for the bug report.

Does IDB behave correctly after you put Safari foreground and refresh the web page?
We do have bug report about network process crashing when backgrounding Safari because database file cannot be locked during process suspension. We have proposed a fix(https://bugs.webkit.org/show_bug.cgi?id=196372).

If that's the case, you should see crash report about com.apple.WebKit.Networking. And you should expect IDB to work if you refresh page and load again.

Thanks!
Comment 8 Michael Lehenabuer 2019-05-07 09:02:56 PDT
Hi Sihui,

Thanks very much for the response.  From the description, https://bugs.webkit.org/show_bug.cgi?id=196372 does sound very much like it could be the root cause which is great news. IDB *does* work fine after refreshing the page.

Where exactly would I look to see the com.apple.WebKit.Networking crash report to confirm that it is happening as well?

Thanks,
Michael
Comment 9 Chris Dumez 2019-05-07 09:07:10 PDT
(In reply to Michael Lehenabuer from comment #8)
> Hi Sihui,
> 
> Thanks very much for the response.  From the description,
> https://bugs.webkit.org/show_bug.cgi?id=196372 does sound very much like it
> could be the root cause which is great news. IDB *does* work fine after
> refreshing the page.
> 
> Where exactly would I look to see the com.apple.WebKit.Networking crash
> report to confirm that it is happening as well?

The crash report should show up under:
Settings > Privacy > Analytics > Analytics Data.
Comment 10 Michael Lehenabuer 2019-05-07 11:21:05 PDT
Created attachment 369304 [details]
com.apple.WebKit.Networking crash report for IndexedDb background issue

Thanks. I do see a number of crash reports for com.apple.WebKit.Networking from 2019-04-24 and 2019-04-25 which is when I was originally reproducing the bug. Strangely though, if I reproduce the bug again now (using my http://mike-shared.firebaseapp.com/ios-crash.html repro) I do *not* see a new crash report, even though the bug still reproduces.  I'm not sure what the difference is.

I've attached one of the reports from 04-25 if you want to verify it is the same bug.  I can't explain why I'm not getting new crash reports though.

Thanks!
Comment 11 Jason Kealey 2019-05-14 05:55:19 PDT
I could reproduce easily on iOS 12.2 yesterday on Safari, Chrome and Firefox using 
http://mike-shared.firebaseapp.com/ios-crash.html

Today, on 12.3, I am unable to make any of the browsers crash right after upgrading iOS. I have not done extensive testing to see if this persists after prolonged browser use.
Comment 12 Michael Lehenabuer 2019-05-14 08:08:43 PDT
Thanks Jason!  It seems like everything is adding up that this is a dupe of https://bugs.webkit.org/show_bug.cgi?id=196372 and has been addressed in iOS 12.3 which is great news.  So I think you can probably go ahead and close this issue.  Thanks!
Comment 13 Stefan Sechelmann 2019-05-16 12:25:30 PDT
The issue is still present in 12.3 stable as observed today in a WKWebView: first occurs

AbortError: UnknownError An internal error was encountered in the Indexed Database server Mozilla/5.0 (iPhone; CPU iPhone OS 12_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko)

then any further access would result in

InvalidStateError: Failed to execute 'transaction' on 'IDBDatabase': The database connection is closing. Mozilla/5.0 (iPhone; CPU iPhone OS 12_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko)

I will observe the stats for our app for the next days.
Comment 14 Sihui Liu 2019-05-16 14:20:44 PDT
Hi all,

The fix for https://bugs.webkit.org/show_bug.cgi?id=196372 is not in iOS(iOS 12.3) yet, so you may still see the bug. For temporary workaround, you could try reloading page or re-opening the IDB database after you encounter the error.

The bug leads to process crash, so the old IDB database connection to crashed process would fail and we need to establish a new connection by re-opening.
Comment 15 Michael Lehenabuer 2019-05-16 14:23:42 PDT
Thanks Sihui,

Do you know what release it's going to be in?  We're planning to disable our usage of IndexedDb on iOS 12.2 to avoid this crash. I was really hoping we'd only have to disable it for iOS 12.2, but if the fix isn't going to make it into iOS 12.3 we may need to reevaluate.
Comment 16 Sihui Liu 2019-05-16 15:15:21 PDT
(In reply to Michael Lehenabuer from comment #15)
> Thanks Sihui,
> 
> Do you know what release it's going to be in?  We're planning to disable our
> usage of IndexedDb on iOS 12.2 to avoid this crash. I was really hoping we'd
> only have to disable it for iOS 12.2, but if the fix isn't going to make it
> into iOS 12.3 we may need to reevaluate.


Hi Michael,

We are not allowed to comment on future iOS release. If you want to know whether the fix is in a released iOS build (including beta), I can tell you that. The fix is not in any iOS build yet.
Comment 17 Johan Baversjo 2019-05-20 13:18:34 PDT
(In reply to Sihui Liu from comment #16)
> (In reply to Michael Lehenabuer from comment #15)
> > Thanks Sihui,
> > 
> > Do you know what release it's going to be in?  We're planning to disable our
> > usage of IndexedDb on iOS 12.2 to avoid this crash. I was really hoping we'd
> > only have to disable it for iOS 12.2, but if the fix isn't going to make it
> > into iOS 12.3 we may need to reevaluate.
> 
> 
> Hi Michael,
> 
> We are not allowed to comment on future iOS release. If you want to know
> whether the fix is in a released iOS build (including beta), I can tell you
> that. The fix is not in any iOS build yet.

Hello Sihui,
Thank you for your fix.
Is it the case that this fix is also not in iOS 12.4 beta 2?
Comment 18 Sihui Liu 2019-05-20 13:55:14 PDT
(In reply to Johan Baversjo from comment #17)
> (In reply to Sihui Liu from comment #16)
> > (In reply to Michael Lehenabuer from comment #15)
> > > Thanks Sihui,
> > > 
> > > Do you know what release it's going to be in?  We're planning to disable our
> > > usage of IndexedDb on iOS 12.2 to avoid this crash. I was really hoping we'd
> > > only have to disable it for iOS 12.2, but if the fix isn't going to make it
> > > into iOS 12.3 we may need to reevaluate.
> > 
> > 
> > Hi Michael,
> > 
> > We are not allowed to comment on future iOS release. If you want to know
> > whether the fix is in a released iOS build (including beta), I can tell you
> > that. The fix is not in any iOS build yet.
> 
> Hello Sihui,
> Thank you for your fix.
> Is it the case that this fix is also not in iOS 12.4 beta 2?

Yes, it's not in iOS 12.4 beta 2.
Comment 19 Tim 2019-06-03 07:59:48 PDT
Can anyone confirm that the fix is in the iOS 12.4 beta 3 or, asking in advance, the first iOS 13 beta?
Comment 20 Sihui Liu 2019-06-05 14:07:48 PDT
Fix is in iOS 13 beta, you should not see the same crash or IDB error when bringing backgrounded app to foreground.
Comment 21 E.J. Daly 2019-06-10 17:08:18 PDT
Hi Sihui,

Can you confirm if this fix is also in iOS 12.4 please?

Thanks
Comment 22 Sihui Liu 2019-06-10 17:52:06 PDT
(In reply to E.J. Daly from comment #21)
> Hi Sihui,
> 
> Can you confirm if this fix is also in iOS 12.4 please?
> 
> Thanks

Hi E.J, this is not in iOS 12.4.
Comment 23 E.J. Daly 2019-06-10 18:28:45 PDT
Thanks Sihui,

Is there still a possibility of it getting to iOS 12 or has that been finalized yet? 

If not, it's going to be remain an issue we need to consider for iPhone ~6 users I guess?

The issue itself is particularly problematic for apps added to home screen, which can't be refreshed / forced to quit in iOS12. So if the app gets into a bad state because of this - it's very difficult to get out of it (without restarting the phone or doing a memory reset or re-installing the PWA)
Comment 24 Sihui Liu 2019-06-12 14:35:41 PDT
(In reply to E.J. Daly from comment #23)
> Thanks Sihui,
> 
> Is there still a possibility of it getting to iOS 12 or has that been
> finalized yet? 
> 
> If not, it's going to be remain an issue we need to consider for iPhone ~6
> users I guess?
> 
> The issue itself is particularly problematic for apps added to home screen,
> which can't be refreshed / forced to quit in iOS12. So if the app gets into
> a bad state because of this - it's very difficult to get out of it (without
> restarting the phone or doing a memory reset or re-installing the PWA)

That should be finalized. Can you try re-opening the database?
Comment 25 Mark 2019-06-27 04:32:17 PDT
We have been struggling with this problem on our WKWebview based app which has local sync database. The error is resulting in the loss of customer's data so is pretty serious for us, we have tried trapping it and reconnecting to the IndexedDb but this does not seem to resolve the issue. The only way currently to move on is to completely reload the webview but doing this means we are loosing the state that needs to be saved to IndexedDb. Many of our users use older iOS devices so it needs to be a iOS 12 fix for us, otherwise we will have to inform a large number of our users that due to a iOS 12 bug we can no longer support their device!
Comment 26 Jake Teton-Landis 2019-08-20 18:06:01 PDT
On iOS 13 beta, we're still seeing the following error:

"UnknownError: Connection to Indexed Database server lost. Refresh the page to try again"

Is this expected to be fixed? Is there another ticket which tracks the "...Refresh the page to try again" bug?

Like Mark, I work on an app that uses IndexedDB to persist critical user edits, so we cannot "refresh the page" without losing in-memory data.
Comment 27 Tim 2019-09-16 07:47:38 PDT
Sihui Liu, can you confirm the bug (or correctly the bug for id 196372) is fixed and available in the iOS 13 GM?

Thanks!
Comment 28 Sihui Liu 2019-09-16 08:56:05 PDT
(In reply to Tim from comment #27)
> Sihui Liu, can you confirm the bug (or correctly the bug for id 196372) is
> fixed and available in the iOS 13 GM?
> 
> Thanks!

Hi,

The fix for bug (https://bugs.webkit.org/show_bug.cgi?id=196372) is in iOS 13 GM Seed. It should fix the issue where IDB internal error showed up when backgrounding app and then bringing it to foreground.
Comment 29 mrschmidt 2019-10-08 14:04:56 PDT
We are seeing new issues with iOS 13 and believe that the patch for this issue may be one of the causes. It appears that the following sequence of operations is now possible:

1) The Firestore SDK opens a new IndexedDb transaction/
2) The Safari tab goes in the background.
3) WebKit terminates the network, which stops the IDBServer (https://trac.webkit.org/browser/trunk/Source/WebKit/NetworkProcess/NetworkProcess.cpp#L2117)
4) The app goes back in the foreground and resumes its operation. Firestore tries to use the transaction it opened in step 1 and receives "Attempt to get a record from database without an in-progress transaction"

I filed https://bugs.webkit.org/show_bug.cgi?id=202705
Comment 30 zac spitzer 2020-03-16 14:35:45 PDT
I'm seeing this problem with multiple iOS 13 users, from what I can tell, 
there were no open transactions before the browser went to the background.

The tab gets focussed again after a few minutes and then "Connection to Indexed Database server lost." gets thrown next time IndexedDB is accessed, 

via a new open, new transaction, put(), no auto incrementing keys in use.
Comment 31 Piotr 2020-04-10 23:38:51 PDT
I can confirm that this bug still exists in iOS 13.3.1.
Comment 32 Brady Eidson 2020-06-09 09:14:49 PDT
Whatever is still happening here to some users is different than what was originally reported.

So for those who still see this happen on iOS 13.x - Please attach com.apple.WebKit.Networking crash logs.
Comment 33 Istvan Tabanyi 2020-08-04 07:05:47 PDT
Created attachment 405919 [details]
Failed to execute 'transaction' on 'IDBDatabase': The database connection is closing.

I still see a *lot* of these on latest iOS with our app. I can not reproduce it by normal usage, but recently I found a pretty reliable way to trigger it. Just start writing records into indexeddb, put the app into background, wait a bit, then repeat, sometimes rapidly switch between foreground and background. Eventually I get a "Failed to execute 'transaction' on 'IDBDatabase': The database connection is closing."
While this is a bit odd way, our users have a lot of trouble because of this, so I guess it just requires some nice timing with putting the app to background.

I also got some kind of failsafe mechanism in place as I automatically retry failed writes a few times. Now another strange issue happens here as during writes I get some "UnknownError: An internal error was encountered in the Indexed Database server", the retry logic trigger, but on the next write request I got ConstraintError, as the data was actually written into indexeddb store previously.

This bug is really killing us, please tell me whatever more info you need to address this. Happy to debug it if you give me some points how to start, as I said I can repro it pretty easily now.
Comment 34 zac spitzer 2020-08-04 07:12:00 PDT
which version of iOS? 

I have only seen this from users running older versions, 13.5.1 being the latest
Comment 35 Istvan Tabanyi 2020-08-04 07:15:45 PDT
13.5.1, I got at least three different AbortError and UnknownError in the last hour with the method I described. I am sure about the two errors which I mentioned.
Comment 36 zac spitzer 2020-08-04 07:22:34 PDT
I'm afraid your only option is to sniff the UA and tell em to upgrade their iOS
Comment 37 Istvan Tabanyi 2020-08-04 07:27:03 PDT
Sorry, but upgrade to where? I can reproduce it on 13.5.1.
Comment 38 zac spitzer 2020-08-04 08:11:35 PDT
13.6 is the latest release of iOS
Comment 39 Istvan Tabanyi 2020-08-05 00:40:34 PDT
Oh, nice, I saw only 13.5.1, restarted the phone and the update appeared. I will try to reproduce it on 13.6, cheers.
Comment 40 Simon Bluhm 2020-08-19 14:40:08 PDT
Created attachment 406874 [details]
UnknownError: An internal error was encountered in the Indexed Database server
Comment 41 Simon Bluhm 2020-08-19 14:40:41 PDT
I have not been able to consistently reproduce this and not sure what exactly is causing this but users of my app have reported several "Failed to execute 'transaction' on 'IDBDatabase': The database connection is closing" errors.

To investigate further, I added the IDBDatabase close event to my code and it seems to fire a lot on iOS 13.3.1 but I have also seen it on iOS 13_6. This is usually on opening the app after a period of inactivity. The code now automatically reloads the app which allows access to IndexedDB again.

I have updated to iOS 14 beta 5 to see if I can reproduce the error on there but I now seem to get the "UnknownError: An internal error was encountered in the Indexed Database server" error again. Only encountered this after updating my web app using skipWaiting in the Service Worker. After the page reloaded, I could no longer access IndexedDB and eventually got the error message above. I have attached the com.apple.WebKit.Networking crash log.

Still trying to create a small repro. Once I can reproduce something I will post a link.
Comment 42 contact 2020-10-02 12:39:37 PDT
I've been unable to reproduce this  myself, but lots of other people do seem to have the issue:
https://github.com/johannesjo/super-productivity/labels/db
Comment 43 David Gaynor 2020-10-22 08:58:43 PDT
Is this bug related: https://bugs.webkit.org/show_bug.cgi?id=216769? Should we track the patch listed there as the expected fix?
Comment 44 Sihui Liu 2020-10-22 09:27:12 PDT
(In reply to David Gaynor from comment #43)
> Is this bug related: https://bugs.webkit.org/show_bug.cgi?id=216769? Should
> we track the patch listed there as the expected fix?

No this bugs should is not related. If you are seeing "An internal error was encountered in the Indexed Database server", it's very likely that WebKit process crahses (may or may not due to IndexedDB). Please check and attach com.apple.WebKit.Networking crash files if there is any (on iOS, go to Settings > Privacy > Analytics > Analytics Data). 

We keep fixing networking process crashes, so it would be nice if you can try on a latest OS build. It's also useful if you can provide steps to reproduce (with your test app, web page, etc) on an older build.
Comment 45 Lee Robertson 2021-01-22 06:35:58 PST
This is still an issue
Comment 46 zac spitzer 2021-01-22 08:25:27 PST
Lee, with which version are you seeing this error? the latest release?
Comment 47 Lee Robertson 2021-01-22 08:49:03 PST
I was able to reproduce on and ipad using version 13.4.1, but we have user's report it on version 14+
Comment 48 Lee Robertson 2021-01-22 11:38:04 PST
(In reply to zac spitzer from comment #46)
> Lee, with which version are you seeing this error? the latest release?


I was able to reproduce on an ipad using version 13.4.1, but we have user's report it on version 14+.

I will say it took a lot of effort to get this to occur. The error occurs when I try to delete a database that is used to store attachments (images, files, etc..).
Comment 49 Valentin Funk 2021-04-08 00:17:33 PDT
I have just received this error in our Bugtracking tool for iOS Mobile Safari 14.0.3 using the popular idb and idb-keyval libraries.
Comment 50 Frank 2021-04-15 10:09:26 PDT
On iOS 14.4.2 with Safari 14.0.3 I am as well recieving the error "UnknownError: Connection to Indexed Database server lost. Refresh the page to try again.".
It's a wide spread error for our users and we require a fix urgently!

We are consuming within a Crodova environment and error is coming randomly. Sometimes it seems to appear more frequently if you put the app into the background and then back to foreground again. But this might also be a wrong observation.

With iOS first versions of iOS 14 this popped up already.

Are there any known workarounds?
Comment 51 Tim 2021-04-20 03:10:05 PDT
Created attachment 426532 [details]
iOS_14_5_com.apple.WebKit.Networking-2021-04-15-121002.ips

I've attached further crash logs and hope that these will be sufficient to finally narrow down the bug.
Comment 52 Tim 2021-04-20 03:11:26 PDT
Created attachment 426533 [details]
iOS_12_4_com.apple.WebKit.Networking-2021-04-13-110655.ips
Comment 53 Alexey Proskuryakov 2021-04-20 09:47:55 PDT
*** Bug 224804 has been marked as a duplicate of this bug. ***
Comment 54 Lincoln Baxter, III 2021-04-23 09:08:04 PDT
We are also seeing this error both on iOS 12.5.2 and also on 14.4.2:

https://sentry.io/share/issue/aee6bb3031b740db8a7ca6bcb68b6bf2/

94.44%	
iOS 12.5.2
26 minutes ago

5.56%	
iOS 14.4.2
2 hours ago

`Error: Uncaught (in promise): UnknownError: Connection to Indexed Database server lost. Refresh the page to try again`
Comment 55 Lee Robertson 2021-06-18 08:16:48 PDT
Does anyone know if the new connection bug in 14.6 is related to this issue?