Bug 195688 – Include WAL and SHM file size in IDB database size computation

• Status: RESOLVED
• Resolution: FIXED
• Component: WebCore Misc.

[youenn fablet]

Include WAL and SHM file size in IDB database size computation

[youenn fablet]

Created attachment 364553 [details]
Patch

[EWS Watchlist]

Comment on attachment 364553 [details]
Patch

Attachment 364553 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/11495927

New failing tests:
storage/websql/open-database-while-transaction-in-progress.html
storage/websql/transaction-database-expand-quota.html

[EWS Watchlist]

Created attachment 364595 [details]
Archive of layout-test-results from ews100 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100  Port: mac-highsierra  Platform: Mac OS X 10.13.6

[EWS Watchlist]

Comment on attachment 364553 [details]
Patch

Attachment 364553 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/11495899

New failing tests:
imported/w3c/web-platform-tests/IndexedDB/idb-binary-key-roundtrip.htm
storage/websql/transaction-database-expand-quota.html

[EWS Watchlist]

Created attachment 364597 [details]
Archive of layout-test-results from ews106 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews106  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6

[youenn fablet]

Created attachment 364714 [details]
Patch

[EWS Watchlist]

Comment on attachment 364714 [details]
Patch

Attachment 364714 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/11510027

New failing tests:
imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.htm
imported/w3c/web-platform-tests/IndexedDB/idb-binary-key-roundtrip.htm
imported/w3c/web-platform-tests/IndexedDB/keypath-special-identifiers.htm
imported/w3c/web-platform-tests/IndexedDB/idbcursor_iterating.htm
imported/w3c/web-platform-tests/IndexedDB/idbcursor-continuePrimaryKey-exceptions.htm

[EWS Watchlist]

Created attachment 364728 [details]
Archive of layout-test-results from ews104 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6

[EWS Watchlist]

Comment on attachment 364714 [details]
Patch

Attachment 364714 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/11511213

New failing tests:
imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.htm
imported/w3c/web-platform-tests/IndexedDB/idb-binary-key-roundtrip.htm
imported/w3c/web-platform-tests/IndexedDB/keypath-special-identifiers.htm
imported/w3c/web-platform-tests/IndexedDB/idbcursor_iterating.htm
imported/w3c/web-platform-tests/IndexedDB/idbcursor-continuePrimaryKey-exceptions.htm

[EWS Watchlist]

Created attachment 364753 [details]
Archive of layout-test-results from ews121 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews121  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6

[youenn fablet]

Created attachment 364799 [details]
Patch

[youenn fablet]

Created attachment 364806 [details]
Patch

[youenn fablet]

Created attachment 364824 [details]
Patch

[Alex Christensen]

Comment on attachment 364824 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=364824&action=review

> Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:826
> +            callback(WTFMove(error.value()));

*error
Couldn't all these checks be moved to waitForRequestSpaceCompletion?

> Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:934
> +        this->clearObjectStoreAfetQuotaCheck(transaction, objectStoreIdentifier, WTFMove(callback));

After

[youenn fablet]

(In reply to Alex Christensen from comment #14)
> Comment on attachment 364824 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=364824&action=review
> 
> > Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:826
> > +            callback(WTFMove(error.value()));
> 
> *error

OK

> Couldn't all these checks be moved to waitForRequestSpaceCompletion?

Not really, the callbacks have different signatures so we would need to have a templated waitForRequestSpaceCompletion, taking a Function instead of a CompletionHandler.


> > Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:934
> > +        this->clearObjectStoreAfetQuotaCheck(transaction, objectStoreIdentifier, WTFMove(callback));
> 
> After

OK

[youenn fablet]

Created attachment 365408 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 365408 [details]
Patch

Clearing flags on attachment: 365408

Committed r243270: <https://trac.webkit.org/changeset/243270>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/49088756>

[Darin Adler]

Comment on attachment 364824 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=364824&action=review

> LayoutTests/ChangeLog:9
> +        Bump quota for this test since WAL files are not included in quota computation.

Do you mean "now included" rather than "not included"?

[Shawn Roberts]

storage/indexeddb/closed-cursor-private.html is crashing on Mac Debug on the Bots.

Crashing started to happen around this revision so I do not know if it could be related.

I cannot repro locally. 

Dashboard :

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=storage%2Findexeddb%2Fclosed-cursor-private.html

Crash Log :

https://build.webkit.org/results/Apple%20High%20Sierra%20Debug%20WK2%20(Tests)/r243270%20(7135)/storage/indexeddb/closed-cursor-private-crash-log.txt

0   com.apple.JavaScriptCore      	0x0000000127fab0f0 WTFCrash + 16 (Assertions.cpp:305)
1   com.apple.WebCore             	0x00000001154157cb WTFCrashWithInfo(int, char const*, char const*, int) + 27
2   com.apple.WebCore             	0x0000000116d22d89 WebCore::IDBServer::UniqueIDBDatabaseConnection::didCommitTransaction(WebCore::IDBServer::UniqueIDBDatabaseTransaction&, WebCore::IDBError const&) + 265 (UniqueIDBDatabaseConnection.cpp:184)
3   com.apple.WebCore             	0x0000000116d7de36 WebCore::IDBServer::UniqueIDBDatabaseTransaction::commit()::$_24::operator()(WebCore::IDBError const&) const + 86 (UniqueIDBDatabaseTransaction.cpp:120)
4   com.apple.WebCore             	0x0000000116d7dd44 WTF::Function<void (WebCore::IDBError const&)>::CallableWrapper<WebCore::IDBServer::UniqueIDBDatabaseTransaction::commit()::$_24>::call(WebCore::IDBError const&) + 52 (Function.h:102)
5   com.apple.WebCore             	0x0000000116d08460 WTF::Function<void (WebCore::IDBError const&)>::operator()(WebCore::IDBError const&) const + 192 (Function.h:57)
6   com.apple.WebCore             	0x0000000116d6db3d auto WebCore::IDBServer::UniqueIDBDatabase::commitTransaction(WebCore::IDBServer::UniqueIDBDatabaseTransaction&, WTF::Function<void (WebCore::IDBError const&)>)::$_18::operator()<WTF::Optional<WebCore::IDBError> >(WTF::Optional<WebCore::IDBError>) + 93 (UniqueIDBDatabase.cpp:1611)

[youenn fablet]

(In reply to Darin Adler from comment #20)
> Comment on attachment 364824 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=364824&action=review
> 
> > LayoutTests/ChangeLog:9
> > +        Bump quota for this test since WAL files are not included in quota computation.
> 
> Do you mean "now included" rather than "not included"?

Yes

[youenn fablet]

> 0   com.apple.JavaScriptCore      	0x0000000127fab0f0 WTFCrash + 16
> (Assertions.cpp:305)
> 1   com.apple.WebCore             	0x00000001154157cb WTFCrashWithInfo(int,
> char const*, char const*, int) + 27
> 2   com.apple.WebCore             	0x0000000116d22d89
> WebCore::IDBServer::UniqueIDBDatabaseConnection::
> didCommitTransaction(WebCore::IDBServer::UniqueIDBDatabaseTransaction&,
> WebCore::IDBError const&) + 265 (UniqueIDBDatabaseConnection.cpp:184)
> 3   com.apple.WebCore             	0x0000000116d7de36
> WebCore::IDBServer::UniqueIDBDatabaseTransaction::commit()::$_24::
> operator()(WebCore::IDBError const&) const + 86
> (UniqueIDBDatabaseTransaction.cpp:120)
> 4   com.apple.WebCore             	0x0000000116d7dd44 WTF::Function<void
> (WebCore::IDBError
> const&)>::CallableWrapper<WebCore::IDBServer::UniqueIDBDatabaseTransaction::
> commit()::$_24>::call(WebCore::IDBError const&) + 52 (Function.h:102)
> 5   com.apple.WebCore             	0x0000000116d08460 WTF::Function<void
> (WebCore::IDBError const&)>::operator()(WebCore::IDBError const&) const +
> 192 (Function.h:57)
> 6   com.apple.WebCore             	0x0000000116d6db3d auto
> WebCore::IDBServer::UniqueIDBDatabase::commitTransaction(WebCore::IDBServer::
> UniqueIDBDatabaseTransaction&, WTF::Function<void (WebCore::IDBError
> const&)>)::$_18::operator()<WTF::Optional<WebCore::IDBError>
> >(WTF::Optional<WebCore::IDBError>) + 93 (UniqueIDBDatabase.cpp:1611)

The debug assertion might need to be updated.
We should probably change it to ASSERT(m_transactionMap.contains(transactionIdentifier) || !error.isNull());

[youenn fablet]

Filed https://bugs.webkit.org/show_bug.cgi?id=196101 to handle the debug assertion.