Bug 143193 - Modern IDB: ObjectStore Blob Support
Summary: Modern IDB: ObjectStore Blob Support
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKit2 (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Brady Eidson
URL:
Keywords:
Depends on: 156041 156068 156242 156248 156264 156321 156367 156523 156640
Blocks: 149117
  Show dependency treegraph
 
Reported: 2015-03-29 09:03 PDT by Joonghun Park
Modified: 2016-04-19 11:12 PDT (History)
8 users (show)

See Also:


Attachments
Patch v1 (72.58 KB, patch)
2016-04-15 13:24 PDT, Brady Eidson
achristensen: review+
buildbot: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews101 for mac-yosemite (817.63 KB, application/zip)
2016-04-15 14:08 PDT, Build Bot
no flags Details
Patch for EWS exploration (72.67 KB, patch)
2016-04-15 14:27 PDT, Brady Eidson
buildbot: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews115 for mac-yosemite (851.90 KB, application/zip)
2016-04-15 14:30 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews103 for mac-yosemite (923.59 KB, application/zip)
2016-04-15 15:11 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews116 for mac-yosemite (853.76 KB, application/zip)
2016-04-15 15:33 PDT, Build Bot
no flags Details
More EWS experimentation (74.07 KB, patch)
2016-04-15 15:44 PDT, Brady Eidson
buildbot: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews102 for mac-yosemite (777.96 KB, application/zip)
2016-04-15 16:28 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews114 for mac-yosemite (860.82 KB, application/zip)
2016-04-15 16:41 PDT, Build Bot
no flags Details
Patch for EWS exploration (74.07 KB, patch)
2016-04-15 16:47 PDT, Brady Eidson
buildbot: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews102 for mac-yosemite (1.14 MB, application/zip)
2016-04-15 17:12 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews116 for mac-yosemite (857.89 KB, application/zip)
2016-04-15 18:00 PDT, Build Bot
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Joonghun Park 2015-03-29 09:03:28 PDT
Add IndexedDB Blob support which allows IndexedDB API to be able to store and retrieve a Blob without converting it to a Base64 string.
Comment 1 Brady Eidson 2016-04-15 13:24:13 PDT
Created attachment 276501 [details]
Patch v1

This patch needs review, but won't land quite yet - https://bugs.webkit.org/show_bug.cgi?id=156640 is the last remaining blocking bug before this lands, and I'm working on it right now.
Comment 2 Build Bot 2016-04-15 14:08:46 PDT
Comment on attachment 276501 [details]
Patch v1

Attachment 276501 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/1164996

New failing tests:
imported/blink/storage/indexeddb/empty-blob-file.html
Comment 3 Build Bot 2016-04-15 14:08:48 PDT
Created attachment 276508 [details]
Archive of layout-test-results from ews101 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews101  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 4 Alex Christensen 2016-04-15 14:19:11 PDT
Comment on attachment 276501 [details]
Patch v1

View in context: https://bugs.webkit.org/attachment.cgi?id=276501&action=review

Code changes look good.  Look into the failing test before landing, after the other patch is landed.

> LayoutTests/imported/blink/storage/indexeddb/blob-basics-metadata.html:106
> +        cursor.continue();
> +//        evalAndLog("cursor.continue();");

this doesn't look good.  I guess it exists like this in blink, though...
Comment 5 Brady Eidson 2016-04-15 14:21:42 PDT
The Yosemite bot failure is super mysterious.

In my reading of the code, there's exactly one place it could come from, and that one place shouldn't ever really happen:

RefPtr<IDBRequest> IDBObjectStore::putOrAdd(JSC::ExecState& state, JSC::JSValue value, RefPtr<IDBKey> key, IndexedDB::ObjectStoreOverwriteMode overwriteMode, InlineKeyCheck inlineKeyCheck, ExceptionCodeWithMessage& ec)
{
    LOG(IndexedDB, "IDBObjectStore::putOrAdd");

    auto context = scriptExecutionContextFromExecState(&state);
    if (!context) {
        ec.code = IDBDatabaseException::UnknownError;
        return nullptr;
    }
...
Comment 6 Brady Eidson 2016-04-15 14:22:51 PDT
(In reply to comment #4)
> Comment on attachment 276501 [details]
> Patch v1
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=276501&action=review
> 
> Code changes look good.  Look into the failing test before landing, after
> the other patch is landed.

Yup, looking into it now.

> 
> > LayoutTests/imported/blink/storage/indexeddb/blob-basics-metadata.html:106
> > +        cursor.continue();
> > +//        evalAndLog("cursor.continue();");
> 
> this doesn't look good.  I guess it exists like this in blink, though...

By calling it natively, the exception is uncaught and aborts the test.
Calling with evalAndLog, the exception is caught, and then the test times out.

Prefer to leave the original in place for future diffing against the blink test.
Comment 7 Brady Eidson 2016-04-15 14:27:04 PDT
Might have gone without saying, but to be explicit: Can't reproduce that failure locally.

To validate my theory on what the bot is seeing, uploading a new patch for EWS.
Comment 8 Brady Eidson 2016-04-15 14:27:32 PDT
Created attachment 276512 [details]
Patch for EWS exploration
Comment 9 Build Bot 2016-04-15 14:30:40 PDT
Comment on attachment 276501 [details]
Patch v1

Attachment 276501 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1165023

New failing tests:
imported/blink/storage/indexeddb/empty-blob-file.html
Comment 10 Build Bot 2016-04-15 14:30:43 PDT
Created attachment 276513 [details]
Archive of layout-test-results from ews115 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews115  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 11 Brady Eidson 2016-04-15 14:38:11 PDT
AHA.
stderr to the rescue

ERROR: Failed copying File contents to a Blob temporary file for storage in IndexedDB (/Volumes/Data/EWS/WebKit/LayoutTests/imported/blink/storage/indexeddb/resources/empty.txt to /var/folders/7n/s62c21h54p549lcpn5v012mc0000gn/T/DumpRenderTree-rzL3dC/BlobHd0Wr3)
/Volumes/Data/EWS/WebKit/Source/WebCore/platform/network/BlobRegistryImpl.cpp(320) : auto WebCore::BlobRegistryImpl::writeBlobsToTemporaryFiles(const Vector<WTF::String> &, std::function<void (const Vector<String> &)>)::(anonymous class)::operator()() const
Comment 12 Build Bot 2016-04-15 15:11:28 PDT
Comment on attachment 276512 [details]
Patch for EWS exploration

Attachment 276512 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/1165220

New failing tests:
imported/blink/storage/indexeddb/empty-blob-file.html
Comment 13 Build Bot 2016-04-15 15:11:33 PDT
Created attachment 276516 [details]
Archive of layout-test-results from ews103 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 14 Build Bot 2016-04-15 15:33:35 PDT
Comment on attachment 276512 [details]
Patch for EWS exploration

Attachment 276512 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1165226

New failing tests:
imported/blink/storage/indexeddb/empty-blob-file.html
Comment 15 Build Bot 2016-04-15 15:33:39 PDT
Created attachment 276517 [details]
Archive of layout-test-results from ews116 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 16 Brady Eidson 2016-04-15 15:44:03 PDT
Created attachment 276518 [details]
More EWS experimentation
Comment 17 Brady Eidson 2016-04-15 15:48:19 PDT
Hmmm, I'm developing a working theory here - This is an empty file, literally 0 bytes.

I wonder if the behavior of our filesystem calls are subtley different on Yosemite vs other OSXs with regards to their responses on 0 byte files.

Hopefully the logging in the patch I just uploaded will tell us this, and if so, it should be an easy fix.
Comment 18 Build Bot 2016-04-15 16:28:03 PDT
Comment on attachment 276518 [details]
More EWS experimentation

Attachment 276518 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/1165509

New failing tests:
imported/blink/storage/indexeddb/empty-blob-file.html
Comment 19 Build Bot 2016-04-15 16:28:07 PDT
Created attachment 276520 [details]
Archive of layout-test-results from ews102 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 20 Build Bot 2016-04-15 16:41:00 PDT
Comment on attachment 276518 [details]
More EWS experimentation

Attachment 276518 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1165517

New failing tests:
imported/blink/storage/indexeddb/empty-blob-file.html
Comment 21 Build Bot 2016-04-15 16:41:04 PDT
Created attachment 276523 [details]
Archive of layout-test-results from ews114 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews114  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 22 Brady Eidson 2016-04-15 16:47:28 PDT
Wow, okay.

Can't even open the source file...

One more EWS test.
Comment 23 Brady Eidson 2016-04-15 16:47:47 PDT
Created attachment 276525 [details]
Patch for EWS exploration
Comment 24 Build Bot 2016-04-15 17:12:54 PDT
Comment on attachment 276525 [details]
Patch for EWS exploration

Attachment 276525 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/1165661

New failing tests:
imported/blink/storage/indexeddb/empty-blob-file.html
Comment 25 Build Bot 2016-04-15 17:12:58 PDT
Created attachment 276529 [details]
Archive of layout-test-results from ews102 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 26 Brady Eidson 2016-04-15 17:16:33 PDT
On a whim, I just tried stashing my changes and then applying the patch locally, using svn-apply.

The empty.txt file - which is in the patch - is not created on the filesystem.

So attempts to reference it do, of course, fail.

This isn't a bug with the patch, but rather a bug with svn-apply.

This patch will be fine as-is when I actually push the changed, but just cannot be svn-applied.

*sigh*

Filed https://bugs.webkit.org/show_bug.cgi?id=156649 for the svn-apply issue.
Comment 27 Build Bot 2016-04-15 18:00:51 PDT
Comment on attachment 276525 [details]
Patch for EWS exploration

Attachment 276525 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/1165754

New failing tests:
imported/blink/storage/indexeddb/empty-blob-file.html
Comment 28 Build Bot 2016-04-15 18:00:53 PDT
Created attachment 276534 [details]
Archive of layout-test-results from ews116 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 29 Brady Eidson 2016-04-19 07:07:15 PDT
r156640 landed overnight.

Once I'm in the office, will re-run tests locally and land this.
Comment 30 Brady Eidson 2016-04-19 09:58:46 PDT
Almost certainly lingering edge cases, but the primary functional is definitely done:
http://trac.webkit.org/changeset/199730
Comment 31 Brady Eidson 2016-04-19 11:12:15 PDT
Release build fix in https://trac.webkit.org/changeset/199732