Bug 195688

Summary:

Include WAL and SHM file size in IDB database size computation

Product:

WebKit

Reporter:

youenn fablet <youennf>

Component:

WebCore Misc.

Assignee:

youenn fablet <youennf>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

achristensen, alecflett, beidson, cdumez, commit-queue, darin, ews-watchlist, jsbell, rniwa, sihui_liu, sroberts, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

WebKit Nightly Build

Hardware:

Unspecified

OS:

Unspecified

Bug Depends on:

Bug Blocks:

195707

Attachments:

Description	Flags
Patch	ews-watchlist: commit-queue-
Archive of layout-test-results from ews100 for mac-highsierra	none
Archive of layout-test-results from ews106 for mac-highsierra-wk2	none
Patch	none
Archive of layout-test-results from ews104 for mac-highsierra-wk2	none
Archive of layout-test-results from ews121 for ios-simulator-wk2	none
Patch	none
Patch	none
Patch	none
Patch	none

Description youenn fablet 2019-03-13 11:30:51 PDT

Include WAL and SHM file size in IDB database size computation

Comment 1 youenn fablet 2019-03-13 11:35:52 PDT

Created attachment 364553 [details]
Patch

Comment 2 EWS Watchlist 2019-03-13 17:35:40 PDT

Comment on attachment 364553 [details]
Patch

Attachment 364553 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/11495927

New failing tests:
storage/websql/open-database-while-transaction-in-progress.html
storage/websql/transaction-database-expand-quota.html

Comment 3 EWS Watchlist 2019-03-13 17:35:41 PDT

Created attachment 364595 [details]
Archive of layout-test-results from ews100 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100  Port: mac-highsierra  Platform: Mac OS X 10.13.6

Comment 4 EWS Watchlist 2019-03-13 17:37:59 PDT

Comment on attachment 364553 [details]
Patch

Attachment 364553 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/11495899

New failing tests:
imported/w3c/web-platform-tests/IndexedDB/idb-binary-key-roundtrip.htm
storage/websql/transaction-database-expand-quota.html

Comment 5 EWS Watchlist 2019-03-13 17:38:01 PDT

Created attachment 364597 [details]
Archive of layout-test-results from ews106 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews106  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6

Comment 6 youenn fablet 2019-03-14 16:27:11 PDT

Created attachment 364714 [details]
Patch

Comment 7 EWS Watchlist 2019-03-14 17:26:46 PDT

Comment on attachment 364714 [details]
Patch

Attachment 364714 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/11510027

New failing tests:
imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.htm
imported/w3c/web-platform-tests/IndexedDB/idb-binary-key-roundtrip.htm
imported/w3c/web-platform-tests/IndexedDB/keypath-special-identifiers.htm
imported/w3c/web-platform-tests/IndexedDB/idbcursor_iterating.htm
imported/w3c/web-platform-tests/IndexedDB/idbcursor-continuePrimaryKey-exceptions.htm

Comment 8 EWS Watchlist 2019-03-14 17:26:49 PDT

Created attachment 364728 [details]
Archive of layout-test-results from ews104 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6

Comment 9 EWS Watchlist 2019-03-14 19:03:29 PDT

Comment on attachment 364714 [details]
Patch

Attachment 364714 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/11511213

New failing tests:
imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.htm
imported/w3c/web-platform-tests/IndexedDB/idb-binary-key-roundtrip.htm
imported/w3c/web-platform-tests/IndexedDB/keypath-special-identifiers.htm
imported/w3c/web-platform-tests/IndexedDB/idbcursor_iterating.htm
imported/w3c/web-platform-tests/IndexedDB/idbcursor-continuePrimaryKey-exceptions.htm

Comment 10 EWS Watchlist 2019-03-14 19:03:31 PDT

Created attachment 364753 [details]
Archive of layout-test-results from ews121 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews121  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6

Comment 11 youenn fablet 2019-03-15 08:24:56 PDT

Created attachment 364799 [details]
Patch

Comment 12 youenn fablet 2019-03-15 09:35:17 PDT

Created attachment 364806 [details]
Patch

Comment 13 youenn fablet 2019-03-15 13:05:55 PDT

Created attachment 364824 [details]
Patch

Comment 14 Alex Christensen 2019-03-18 10:21:46 PDT

Comment on attachment 364824 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=364824&action=review

> Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:826
> +            callback(WTFMove(error.value()));

*error
Couldn't all these checks be moved to waitForRequestSpaceCompletion?

> Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:934
> +        this->clearObjectStoreAfetQuotaCheck(transaction, objectStoreIdentifier, WTFMove(callback));

After

Comment 15 youenn fablet 2019-03-20 14:48:02 PDT

(In reply to Alex Christensen from comment #14)
> Comment on attachment 364824 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=364824&action=review
> 
> > Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:826
> > +            callback(WTFMove(error.value()));
> 
> *error

OK

> Couldn't all these checks be moved to waitForRequestSpaceCompletion?

Not really, the callbacks have different signatures so we would need to have a templated waitForRequestSpaceCompletion, taking a Function instead of a CompletionHandler.


> > Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:934
> > +        this->clearObjectStoreAfetQuotaCheck(transaction, objectStoreIdentifier, WTFMove(callback));
> 
> After

OK

Comment 16 youenn fablet 2019-03-20 14:59:55 PDT

Created attachment 365408 [details]
Patch

Comment 17 WebKit Commit Bot 2019-03-20 18:24:57 PDT

Comment on attachment 365408 [details]
Patch

Clearing flags on attachment: 365408

Committed r243270: <https://trac.webkit.org/changeset/243270>

Comment 18 WebKit Commit Bot 2019-03-20 18:24:59 PDT

All reviewed patches have been landed.  Closing bug.

Comment 19 Radar WebKit Bug Importer 2019-03-20 18:25:24 PDT

<rdar://problem/49088756>

Comment 20 Darin Adler 2019-03-21 08:37:16 PDT

Comment on attachment 364824 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=364824&action=review

> LayoutTests/ChangeLog:9
> +        Bump quota for this test since WAL files are not included in quota computation.

Do you mean "now included" rather than "not included"?

Comment 21 Shawn Roberts 2019-03-21 11:04:14 PDT

storage/indexeddb/closed-cursor-private.html is crashing on Mac Debug on the Bots.

Crashing started to happen around this revision so I do not know if it could be related.

I cannot repro locally. 

Dashboard :

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=storage%2Findexeddb%2Fclosed-cursor-private.html

Crash Log :

https://build.webkit.org/results/Apple%20High%20Sierra%20Debug%20WK2%20(Tests)/r243270%20(7135)/storage/indexeddb/closed-cursor-private-crash-log.txt

0   com.apple.JavaScriptCore      	0x0000000127fab0f0 WTFCrash + 16 (Assertions.cpp:305)
1   com.apple.WebCore             	0x00000001154157cb WTFCrashWithInfo(int, char const*, char const*, int) + 27
2   com.apple.WebCore             	0x0000000116d22d89 WebCore::IDBServer::UniqueIDBDatabaseConnection::didCommitTransaction(WebCore::IDBServer::UniqueIDBDatabaseTransaction&, WebCore::IDBError const&) + 265 (UniqueIDBDatabaseConnection.cpp:184)
3   com.apple.WebCore             	0x0000000116d7de36 WebCore::IDBServer::UniqueIDBDatabaseTransaction::commit()::$_24::operator()(WebCore::IDBError const&) const + 86 (UniqueIDBDatabaseTransaction.cpp:120)
4   com.apple.WebCore             	0x0000000116d7dd44 WTF::Function<void (WebCore::IDBError const&)>::CallableWrapper<WebCore::IDBServer::UniqueIDBDatabaseTransaction::commit()::$_24>::call(WebCore::IDBError const&) + 52 (Function.h:102)
5   com.apple.WebCore             	0x0000000116d08460 WTF::Function<void (WebCore::IDBError const&)>::operator()(WebCore::IDBError const&) const + 192 (Function.h:57)
6   com.apple.WebCore             	0x0000000116d6db3d auto WebCore::IDBServer::UniqueIDBDatabase::commitTransaction(WebCore::IDBServer::UniqueIDBDatabaseTransaction&, WTF::Function<void (WebCore::IDBError const&)>)::$_18::operator()<WTF::Optional<WebCore::IDBError> >(WTF::Optional<WebCore::IDBError>) + 93 (UniqueIDBDatabase.cpp:1611)

Comment 22 youenn fablet 2019-03-21 11:13:42 PDT

(In reply to Darin Adler from comment #20)
> Comment on attachment 364824 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=364824&action=review
> 
> > LayoutTests/ChangeLog:9
> > +        Bump quota for this test since WAL files are not included in quota computation.
> 
> Do you mean "now included" rather than "not included"?

Yes

Comment 23 youenn fablet 2019-03-21 12:08:08 PDT

> 0   com.apple.JavaScriptCore      	0x0000000127fab0f0 WTFCrash + 16
> (Assertions.cpp:305)
> 1   com.apple.WebCore             	0x00000001154157cb WTFCrashWithInfo(int,
> char const*, char const*, int) + 27
> 2   com.apple.WebCore             	0x0000000116d22d89
> WebCore::IDBServer::UniqueIDBDatabaseConnection::
> didCommitTransaction(WebCore::IDBServer::UniqueIDBDatabaseTransaction&,
> WebCore::IDBError const&) + 265 (UniqueIDBDatabaseConnection.cpp:184)
> 3   com.apple.WebCore             	0x0000000116d7de36
> WebCore::IDBServer::UniqueIDBDatabaseTransaction::commit()::$_24::
> operator()(WebCore::IDBError const&) const + 86
> (UniqueIDBDatabaseTransaction.cpp:120)
> 4   com.apple.WebCore             	0x0000000116d7dd44 WTF::Function<void
> (WebCore::IDBError
> const&)>::CallableWrapper<WebCore::IDBServer::UniqueIDBDatabaseTransaction::
> commit()::$_24>::call(WebCore::IDBError const&) + 52 (Function.h:102)
> 5   com.apple.WebCore             	0x0000000116d08460 WTF::Function<void
> (WebCore::IDBError const&)>::operator()(WebCore::IDBError const&) const +
> 192 (Function.h:57)
> 6   com.apple.WebCore             	0x0000000116d6db3d auto
> WebCore::IDBServer::UniqueIDBDatabase::commitTransaction(WebCore::IDBServer::
> UniqueIDBDatabaseTransaction&, WTF::Function<void (WebCore::IDBError
> const&)>)::$_18::operator()<WTF::Optional<WebCore::IDBError>
> >(WTF::Optional<WebCore::IDBError>) + 93 (UniqueIDBDatabase.cpp:1611)

The debug assertion might need to be updated.
We should probably change it to ASSERT(m_transactionMap.contains(transactionIdentifier) || !error.isNull());

Comment 24 youenn fablet 2019-03-21 12:16:21 PDT

Filed https://bugs.webkit.org/show_bug.cgi?id=196101 to handle the debug assertion.