Bug 115761 – Race condition with DatabaseTracker::deleteAllDatabases and web workers

• Component: WebCore Misc.
• Status: ASSIGNED

[Chris Dalton]

When DatabaseTracker::deleteAllDatabases is called from the main thread, it ends up calling through DatabaseTracker::deleteDatabaseFile to Database::markAsDeletedAndClose, on all the relevant open databases (including those open by web workers).

markAsDeletedAndClose first makes sure that databaseContext()->databaseThread()->terminationRequested() is false, and then schedules and waits for a DatabaseClose task.

This has two problems:

1) It's possible for the worker thread to request termination AFTER markAsDeletedAndClose checks terminationRequested, but BEFORE it schedules and waits for the task. (WorkerThreadShutdownStartTask::performTask -> DatabaseManager::stopDatabases -> DatabaseContext::stopDatabases -> DatabaseThread::requestTermination.) In this scenario, the task will never execute and the main thread will hang.

2) It's possible for markAsDeletedAndClose to check terminationRequested AFTER the worker thread has requested termination, but BEFORE the database is fully closed. In this it's possible to delete the database file while a web worker still has it open.

We need a thread-safe way of making sure all relevant databases are closed before deleting the file.

[Mark Lam]

Im already working on something in this area.  Will investigate this issue as well.

[Simon Fraser (smfr)]

Maybe the cause of this frequent crash on the bots?
http://build.webkit.org/results/Apple%20MountainLion%20Debug%20WK2%20(Tests)/r158872%20(14016)/fast/workers/storage/open-database-while-transaction-in-progress-crash-log.txt

[Radar WebKit Bug Importer]

<rdar://problem/15419881>

[Mark Lam]

(In reply to comment #2)
> Maybe the cause of this frequent crash on the bots?
> http://build.webkit.org/results/Apple%20MountainLion%20Debug%20WK2%20(Tests)/r158872%20(14016)/fast/workers/storage/open-database-while-transaction-in-progress-crash-log.txt

Tip from smfr: Look at main thread and thread 17.

[Brady Eidson]

Tried to reproduce this multiple times with `run-webkit-tests fast/workers/storage -g -2` and `run-webkit-tests fast/workers/storage -2` to no avail.

I think it's a strong possibility that the WAL journal_mode has made this race condition more likely to rear its ugly head.

[Ahmad Saleem]

Something similar was fixed in Blink:

https://chromium.googlesource.com/chromium/blink/+/8462425975e2d8bc28b02d9c2416ddc9c8520aab

It was about hang happening while detaching frame in SQLTransactionCoordinator::shutdown().

This might not be directly related bug but it is only bug I got when I was trying to search for "deleteAllDatabases".

[Ahmad Saleem]

(In reply to Ahmad Saleem from comment #6)
> Something similar was fixed in Blink:
> 
> https://chromium.googlesource.com/chromium/blink/+/
> 8462425975e2d8bc28b02d9c2416ddc9c8520aab
> 
> It was about hang happening while detaching frame in
> SQLTransactionCoordinator::shutdown().
> 
> This might not be directly related bug but it is only bug I got when I was
> trying to search for "deleteAllDatabases".

https://searchfox.org/wubkat/rev/3e2262b3476ecfb5666a35bcdd746d474c061b0d/Source/WebCore/Modules/webdatabase/SQLTransactionCoordinator.cpp#132