Bug 204379 - IndexedDB: lock-based implementation of suspension
Summary: IndexedDB: lock-based implementation of suspension
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Sihui Liu
URL:
Keywords: InRadar
Depends on:
Blocks: 203971
  Show dependency treegraph
 
Reported: 2019-11-19 14:34 PST by Sihui Liu
Modified: 2019-11-22 11:15 PST (History)
7 users (show)

See Also:


Attachments
Patch (31.35 KB, patch)
2019-11-19 14:45 PST, Sihui Liu
no flags Details | Formatted Diff | Diff
Patch (44.85 KB, patch)
2019-11-20 17:13 PST, Sihui Liu
no flags Details | Formatted Diff | Diff
Patch (44.86 KB, patch)
2019-11-20 17:23 PST, Sihui Liu
no flags Details | Formatted Diff | Diff
Patch (45.80 KB, patch)
2019-11-21 09:15 PST, Sihui Liu
no flags Details | Formatted Diff | Diff
Patch (99.39 KB, patch)
2019-11-21 16:51 PST, Sihui Liu
no flags Details | Formatted Diff | Diff
Patch for landing (96.27 KB, patch)
2019-11-22 10:55 PST, Sihui Liu
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Sihui Liu 2019-11-19 14:34:54 PST
When process is about to suspend, we would:
1. suspend the database thread (by making it wait on a condition variable)
2. (wait on main thread until 1 finishes) stop all transactions on the main thread
To implement this, database thread would check whether it needs to suspend *after each task*. If it needs to, it would wake up the main thread, and wait on suspend condition.

However, if the database thread is already suspended (e.g. waiting on lock) in the middle of some task, the thread will not get to the point of checking suspension state and waking up main thread. Main thread would be stuck.
Comment 1 Sihui Liu 2019-11-19 14:45:54 PST
Created attachment 383907 [details]
Patch
Comment 2 Brady Eidson 2019-11-20 11:04:57 PST
Comment on attachment 383907 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=383907&action=review

> Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.h:273
> +    Lock m_backingStoreLock;

This change adds fragility.

I think it would make it easier to avoid future mistakes if the backing store itself had a notion of a lock that it could assert is locked for each backing store operation.

I can think of at least two ways to accomplish this:

1 - Since UniqueIDBDatabase needs a lock for the setting/getting of the backing store itself, add a second lock to the backing store.
2 - Pass a Lock& to the backing store constructor so it can ASSERT the lock is appropriately held.
Comment 3 Brady Eidson 2019-11-20 11:07:37 PST
Comment on attachment 383907 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=383907&action=review

> Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:2473
> +    if (m_isSuspended)
> +        return;
> +    m_isSuspended = true;
> +
> +    m_backingStoreLock.lock();
> +

This change adds fragility.

To make it harder to introduce future mistakes I think we should make this be ASSERT-crazy.

If not suspended, assert the lock is not held.

> Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:2508
> +void UniqueIDBDatabase::resume()
> +{
> +    ASSERT(isMainThread());
> +
> +    if (!m_isSuspended)
> +        return;
> +
> +    m_isSuspended = false;
> +    m_backingStoreLock.unlock();
> +}

This change adds fragility.

To make it harder to introduce future mistakes I think we should make this be ASSERT-crazy.

If suspended, assert the lock is held
Comment 4 Sihui Liu 2019-11-20 17:13:18 PST
Created attachment 384006 [details]
Patch
Comment 5 Sihui Liu 2019-11-20 17:23:54 PST
Created attachment 384007 [details]
Patch
Comment 6 Sihui Liu 2019-11-21 09:15:28 PST
Created attachment 384059 [details]
Patch
Comment 7 Geoffrey Garen 2019-11-21 11:24:45 PST
A design pattern that can help you remember to take the lock is to require a lock holder reference as an argument to each function that wants the lock to be held. 

That way, the compiler will verify that you did acquire the lock before calling the function. (Technically, you can still make a mistake and lock a different lock of the same type, but that mistake is very rare.)

If we do it that way, we don't really need the backing store to hold the lock as a data member or assert that it is locked; the type of the locker can only be instantiated by locking the lock.
Comment 8 Geoffrey Garen 2019-11-21 11:26:49 PST
It would also be nice to add a comment next to the declaration of the backing store lock that explains that its purpose is to synchronize against process suspension (even though the backing store is usually only accessed on one thread).

Maybe we could even rename it to backingStoreSuspensionLock.
Comment 9 Geoffrey Garen 2019-11-21 11:57:10 PST
Here's an alternative solution to consider: What if, before calling the quota manager on the main thread, the database thread pre-emptively suspended its own task queue?

Like so (similar to CrossThreadTaskHandler::taskRunLoop()):

            m_suspendedLock.lock();
            ASSERT(!m_suspended);
            m_suspended = true;
            m_suspendedCondition.notifyOne();
            m_suspendedLock.unlock();

And then, upon reply, it checked whether to suspend?

Like so (duplicating CrossThreadTaskHandler::taskRunLoop()):

        Locker<Lock> shouldSuspendLocker(m_shouldSuspendLock);
        while (m_shouldSuspend) {
            m_suspendedLock.lock();
            if (!m_suspended) {
                m_suspended = true;
                m_suspendedCondition.notifyOne();
            }
            m_suspendedLock.unlock();
            m_shouldSuspendCondition.wait(m_shouldSuspendLock);
        }

I believe, if you did that, you would solve the conundrum of waiting for acknowledgement from the database thread because the database thread would acknowledge being suspended preemptively before it went to sleep; and it would honor any request to suspend after it woke up.

(Of course, rather than duplicating code from CrossThreadTaskHandler::taskRunLoop(), I would probably make helper functions.)
Comment 10 Sihui Liu 2019-11-21 16:51:54 PST
Created attachment 384106 [details]
Patch
Comment 11 Geoffrey Garen 2019-11-21 17:18:17 PST
Comment on attachment 384106 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=384106&action=review

r=me

> Source/WebCore/Modules/indexeddb/server/IDBServer.h:196
> +    HashSet<UniqueIDBDatabase*> m_allUniqueIDBDatabases;

It would be better to use a WeakHashSet. That way, you don't need registerDatabase / unregisterDatabase, and you have pointer safety by default.

> Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.h:273
> +    Lock m_backingStoreLock;

Would be nice to add a comment explaining that the purpose of this lock is to synchronize vs suspension.
Comment 12 Sihui Liu 2019-11-22 10:55:28 PST
Created attachment 384172 [details]
Patch for landing
Comment 13 WebKit Commit Bot 2019-11-22 11:14:29 PST
Comment on attachment 384172 [details]
Patch for landing

Clearing flags on attachment: 384172

Committed r252787: <https://trac.webkit.org/changeset/252787>
Comment 14 WebKit Commit Bot 2019-11-22 11:14:30 PST
All reviewed patches have been landed.  Closing bug.
Comment 15 Radar WebKit Bug Importer 2019-11-22 11:15:20 PST
<rdar://problem/57434641>