Bug 195688 - Include WAL and SHM file size in IDB database size computation
Summary: Include WAL and SHM file size in IDB database size computation
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebCore Misc. (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: youenn fablet
URL:
Keywords: InRadar
Depends on:
Blocks: 195707
  Show dependency treegraph
 
Reported: 2019-03-13 11:30 PDT by youenn fablet
Modified: 2019-03-21 12:16 PDT (History)
12 users (show)

See Also:


Attachments
Patch (2.01 KB, patch)
2019-03-13 11:35 PDT, youenn fablet
ews-watchlist: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews100 for mac-highsierra (2.56 MB, application/zip)
2019-03-13 17:35 PDT, EWS Watchlist
no flags Details
Archive of layout-test-results from ews106 for mac-highsierra-wk2 (2.88 MB, application/zip)
2019-03-13 17:38 PDT, EWS Watchlist
no flags Details
Patch (3.58 KB, patch)
2019-03-14 16:27 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews104 for mac-highsierra-wk2 (3.21 MB, application/zip)
2019-03-14 17:26 PDT, EWS Watchlist
no flags Details
Archive of layout-test-results from ews121 for ios-simulator-wk2 (2.62 MB, application/zip)
2019-03-14 19:03 PDT, EWS Watchlist
no flags Details
Patch (24.39 KB, patch)
2019-03-15 08:24 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch (16.29 KB, patch)
2019-03-15 09:35 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch (20.01 KB, patch)
2019-03-15 13:05 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch (27.02 KB, patch)
2019-03-20 14:59 PDT, youenn fablet
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description youenn fablet 2019-03-13 11:30:51 PDT
Include WAL and SHM file size in IDB database size computation
Comment 1 youenn fablet 2019-03-13 11:35:52 PDT
Created attachment 364553 [details]
Patch
Comment 2 EWS Watchlist 2019-03-13 17:35:40 PDT
Comment on attachment 364553 [details]
Patch

Attachment 364553 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/11495927

New failing tests:
storage/websql/open-database-while-transaction-in-progress.html
storage/websql/transaction-database-expand-quota.html
Comment 3 EWS Watchlist 2019-03-13 17:35:41 PDT
Created attachment 364595 [details]
Archive of layout-test-results from ews100 for mac-highsierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100  Port: mac-highsierra  Platform: Mac OS X 10.13.6
Comment 4 EWS Watchlist 2019-03-13 17:37:59 PDT
Comment on attachment 364553 [details]
Patch

Attachment 364553 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/11495899

New failing tests:
imported/w3c/web-platform-tests/IndexedDB/idb-binary-key-roundtrip.htm
storage/websql/transaction-database-expand-quota.html
Comment 5 EWS Watchlist 2019-03-13 17:38:01 PDT
Created attachment 364597 [details]
Archive of layout-test-results from ews106 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews106  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6
Comment 6 youenn fablet 2019-03-14 16:27:11 PDT
Created attachment 364714 [details]
Patch
Comment 7 EWS Watchlist 2019-03-14 17:26:46 PDT
Comment on attachment 364714 [details]
Patch

Attachment 364714 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/11510027

New failing tests:
imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.htm
imported/w3c/web-platform-tests/IndexedDB/idb-binary-key-roundtrip.htm
imported/w3c/web-platform-tests/IndexedDB/keypath-special-identifiers.htm
imported/w3c/web-platform-tests/IndexedDB/idbcursor_iterating.htm
imported/w3c/web-platform-tests/IndexedDB/idbcursor-continuePrimaryKey-exceptions.htm
Comment 8 EWS Watchlist 2019-03-14 17:26:49 PDT
Created attachment 364728 [details]
Archive of layout-test-results from ews104 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6
Comment 9 EWS Watchlist 2019-03-14 19:03:29 PDT
Comment on attachment 364714 [details]
Patch

Attachment 364714 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/11511213

New failing tests:
imported/w3c/web-platform-tests/IndexedDB/key-conversion-exceptions.htm
imported/w3c/web-platform-tests/IndexedDB/idb-binary-key-roundtrip.htm
imported/w3c/web-platform-tests/IndexedDB/keypath-special-identifiers.htm
imported/w3c/web-platform-tests/IndexedDB/idbcursor_iterating.htm
imported/w3c/web-platform-tests/IndexedDB/idbcursor-continuePrimaryKey-exceptions.htm
Comment 10 EWS Watchlist 2019-03-14 19:03:31 PDT
Created attachment 364753 [details]
Archive of layout-test-results from ews121 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews121  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6
Comment 11 youenn fablet 2019-03-15 08:24:56 PDT
Created attachment 364799 [details]
Patch
Comment 12 youenn fablet 2019-03-15 09:35:17 PDT
Created attachment 364806 [details]
Patch
Comment 13 youenn fablet 2019-03-15 13:05:55 PDT
Created attachment 364824 [details]
Patch
Comment 14 Alex Christensen 2019-03-18 10:21:46 PDT
Comment on attachment 364824 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=364824&action=review

> Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:826
> +            callback(WTFMove(error.value()));

*error
Couldn't all these checks be moved to waitForRequestSpaceCompletion?

> Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:934
> +        this->clearObjectStoreAfetQuotaCheck(transaction, objectStoreIdentifier, WTFMove(callback));

After
Comment 15 youenn fablet 2019-03-20 14:48:02 PDT
(In reply to Alex Christensen from comment #14)
> Comment on attachment 364824 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=364824&action=review
> 
> > Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:826
> > +            callback(WTFMove(error.value()));
> 
> *error

OK

> Couldn't all these checks be moved to waitForRequestSpaceCompletion?

Not really, the callbacks have different signatures so we would need to have a templated waitForRequestSpaceCompletion, taking a Function instead of a CompletionHandler.


> > Source/WebCore/Modules/indexeddb/server/UniqueIDBDatabase.cpp:934
> > +        this->clearObjectStoreAfetQuotaCheck(transaction, objectStoreIdentifier, WTFMove(callback));
> 
> After

OK
Comment 16 youenn fablet 2019-03-20 14:59:55 PDT
Created attachment 365408 [details]
Patch
Comment 17 WebKit Commit Bot 2019-03-20 18:24:57 PDT
Comment on attachment 365408 [details]
Patch

Clearing flags on attachment: 365408

Committed r243270: <https://trac.webkit.org/changeset/243270>
Comment 18 WebKit Commit Bot 2019-03-20 18:24:59 PDT
All reviewed patches have been landed.  Closing bug.
Comment 19 Radar WebKit Bug Importer 2019-03-20 18:25:24 PDT
<rdar://problem/49088756>
Comment 20 Darin Adler 2019-03-21 08:37:16 PDT
Comment on attachment 364824 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=364824&action=review

> LayoutTests/ChangeLog:9
> +        Bump quota for this test since WAL files are not included in quota computation.

Do you mean "now included" rather than "not included"?
Comment 21 Shawn Roberts 2019-03-21 11:04:14 PDT
storage/indexeddb/closed-cursor-private.html is crashing on Mac Debug on the Bots.

Crashing started to happen around this revision so I do not know if it could be related.

I cannot repro locally. 

Dashboard :

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=storage%2Findexeddb%2Fclosed-cursor-private.html

Crash Log :

https://build.webkit.org/results/Apple%20High%20Sierra%20Debug%20WK2%20(Tests)/r243270%20(7135)/storage/indexeddb/closed-cursor-private-crash-log.txt

0   com.apple.JavaScriptCore      	0x0000000127fab0f0 WTFCrash + 16 (Assertions.cpp:305)
1   com.apple.WebCore             	0x00000001154157cb WTFCrashWithInfo(int, char const*, char const*, int) + 27
2   com.apple.WebCore             	0x0000000116d22d89 WebCore::IDBServer::UniqueIDBDatabaseConnection::didCommitTransaction(WebCore::IDBServer::UniqueIDBDatabaseTransaction&, WebCore::IDBError const&) + 265 (UniqueIDBDatabaseConnection.cpp:184)
3   com.apple.WebCore             	0x0000000116d7de36 WebCore::IDBServer::UniqueIDBDatabaseTransaction::commit()::$_24::operator()(WebCore::IDBError const&) const + 86 (UniqueIDBDatabaseTransaction.cpp:120)
4   com.apple.WebCore             	0x0000000116d7dd44 WTF::Function<void (WebCore::IDBError const&)>::CallableWrapper<WebCore::IDBServer::UniqueIDBDatabaseTransaction::commit()::$_24>::call(WebCore::IDBError const&) + 52 (Function.h:102)
5   com.apple.WebCore             	0x0000000116d08460 WTF::Function<void (WebCore::IDBError const&)>::operator()(WebCore::IDBError const&) const + 192 (Function.h:57)
6   com.apple.WebCore             	0x0000000116d6db3d auto WebCore::IDBServer::UniqueIDBDatabase::commitTransaction(WebCore::IDBServer::UniqueIDBDatabaseTransaction&, WTF::Function<void (WebCore::IDBError const&)>)::$_18::operator()<WTF::Optional<WebCore::IDBError> >(WTF::Optional<WebCore::IDBError>) + 93 (UniqueIDBDatabase.cpp:1611)
Comment 22 youenn fablet 2019-03-21 11:13:42 PDT
(In reply to Darin Adler from comment #20)
> Comment on attachment 364824 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=364824&action=review
> 
> > LayoutTests/ChangeLog:9
> > +        Bump quota for this test since WAL files are not included in quota computation.
> 
> Do you mean "now included" rather than "not included"?

Yes
Comment 23 youenn fablet 2019-03-21 12:08:08 PDT
> 0   com.apple.JavaScriptCore      	0x0000000127fab0f0 WTFCrash + 16
> (Assertions.cpp:305)
> 1   com.apple.WebCore             	0x00000001154157cb WTFCrashWithInfo(int,
> char const*, char const*, int) + 27
> 2   com.apple.WebCore             	0x0000000116d22d89
> WebCore::IDBServer::UniqueIDBDatabaseConnection::
> didCommitTransaction(WebCore::IDBServer::UniqueIDBDatabaseTransaction&,
> WebCore::IDBError const&) + 265 (UniqueIDBDatabaseConnection.cpp:184)
> 3   com.apple.WebCore             	0x0000000116d7de36
> WebCore::IDBServer::UniqueIDBDatabaseTransaction::commit()::$_24::
> operator()(WebCore::IDBError const&) const + 86
> (UniqueIDBDatabaseTransaction.cpp:120)
> 4   com.apple.WebCore             	0x0000000116d7dd44 WTF::Function<void
> (WebCore::IDBError
> const&)>::CallableWrapper<WebCore::IDBServer::UniqueIDBDatabaseTransaction::
> commit()::$_24>::call(WebCore::IDBError const&) + 52 (Function.h:102)
> 5   com.apple.WebCore             	0x0000000116d08460 WTF::Function<void
> (WebCore::IDBError const&)>::operator()(WebCore::IDBError const&) const +
> 192 (Function.h:57)
> 6   com.apple.WebCore             	0x0000000116d6db3d auto
> WebCore::IDBServer::UniqueIDBDatabase::commitTransaction(WebCore::IDBServer::
> UniqueIDBDatabaseTransaction&, WTF::Function<void (WebCore::IDBError
> const&)>)::$_18::operator()<WTF::Optional<WebCore::IDBError>
> >(WTF::Optional<WebCore::IDBError>) + 93 (UniqueIDBDatabase.cpp:1611)

The debug assertion might need to be updated.
We should probably change it to ASSERT(m_transactionMap.contains(transactionIdentifier) || !error.isNull());
Comment 24 youenn fablet 2019-03-21 12:16:21 PDT
Filed https://bugs.webkit.org/show_bug.cgi?id=196101 to handle the debug assertion.