Bug 129642 - StorageTracker::deleteOrigin being called off the main thread (ASSERTs in inspector/test-harness-trivially-works.html test)
Summary: StorageTracker::deleteOrigin being called off the main thread (ASSERTs in ins...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKit Misc. (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Brady Eidson
URL:
Keywords: InRadar, MakingBotsRed
Depends on:
Blocks: 129217
  Show dependency treegraph
 
Reported: 2014-03-03 16:59 PST by Alexey Proskuryakov
Modified: 2014-09-26 13:19 PDT (History)
9 users (show)

See Also:


Attachments
Patch (5.12 KB, patch)
2014-09-25 15:43 PDT, Brian Burg
beidson: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexey Proskuryakov 2014-03-03 16:59:34 PST
inspector/test-harness-trivially-works.html frequently asserts:

http://build.webkit.org/results/Apple%20Mavericks%20Debug%20WK1%20(Tests)/r165013%20(3509)/inspector/test-harness-trivially-works-crash-log.txt

Thread 25 Crashed:: WebCore: LocalStorage
0   com.apple.JavaScriptCore      	0x0000000101d74b2a WTFCrash + 42 (Assertions.cpp:333)
1   com.apple.WebCore             	0x00000001050e4db2 WebCore::StorageTracker::deleteOrigin(WebCore::SecurityOrigin*) + 130 (StorageTracker.cpp:455)
2   com.apple.WebCore             	0x00000001050e402d WebCore::StorageTracker::deleteOriginWithIdentifier(WTF::String const&) + 61 (StorageTracker.cpp:449)
3   com.apple.WebCore             	0x00000001050cd815 WebCore::StorageAreaSync::deleteEmptyDatabase() + 421 (StorageAreaSync.cpp:517)
4   com.apple.WebCore             	0x00000001050cfac2 WTF::FunctionWrapper<void (WebCore::StorageAreaSync::*)()>::operator()(WebCore::StorageAreaSync*) + 114 (Functional.h:218)
5   com.apple.WebCore             	0x00000001050cfa45 WTF::BoundFunctionImpl<WTF::FunctionWrapper<void (WebCore::StorageAreaSync::*)()>, void (WebCore::StorageAreaSync*)>::operator()() + 53 (Functional.h:496)
Comment 1 Radar WebKit Bug Importer 2014-03-03 17:00:49 PST
<rdar://problem/16217802>
Comment 2 Timothy Hatcher 2014-03-04 09:51:11 PST
StorageTracker.cpp:455: ASSERT(isMainThread());

This is an indication of a treading issue or the assertion is bad. It does not seem Inspector specific, this test does nothing more than test the Inspector test harness. The Inspector is closing when this happens.
Comment 3 Alexey Proskuryakov 2014-03-04 10:09:06 PST
What should we do in the meanwhile? Would skipping this particular test be sufficient? Or does an assertion  in "trivially works" test mean that all Inspector tests are affected?
Comment 4 BJ Burg 2014-03-04 10:15:00 PST
(In reply to comment #3)
> What should we do in the meanwhile? Would skipping this particular test be sufficient? Or does an assertion  in "trivially works" test mean that all Inspector tests are affected?

It would affect tests in LayoutTests/inspector/ but not LayoutTests/inspector-protocol. The latter uses a dummy page and doesn't actually open up an inspector instance through WK/WK2.

I was working to move tests in LayoutTests/inspector-protocol/ to LayoutTests/inspector, so enabling those tests after moving them to the different test harness is blocked by this flakiness.
Comment 5 BJ Burg 2014-03-04 10:16:25 PST
(In reply to comment #4)
> (In reply to comment #3)
> > What should we do in the meanwhile? Would skipping this particular test be sufficient? Or does an assertion  in "trivially works" test mean that all Inspector tests are affected?

So, it could be skipped for now to make bots green, but it should be addressed ASAP to unblock writing replay tests.
Comment 6 BJ Burg 2014-03-04 10:57:40 PST
Marked skipped in <https://trac.webkit.org/changeset/165061>
Comment 7 Joseph Pecoraro 2014-03-04 11:18:53 PST
(In reply to comment #2)
> StorageTracker.cpp:455: ASSERT(isMainThread());

Aren't we on the main thread?!
Comment 8 Joseph Pecoraro 2014-03-04 11:20:09 PST
Oh, I see: Crashed Thread:  25  WebCore: LocalStorage
Comment 9 Brady Eidson 2014-03-10 09:50:56 PDT
(In reply to comment #2)
>this test does nothing more than test the Inspector test harness. 

What is "the inspector test harness"?  DRT?  WebKitTestRunner?  Or some other executable?

I ask, because a WebKit app that enables LocalStorage is also required to explicitly shut down local storage before it exits.  DRT and WKTR do this.
Comment 10 Brady Eidson 2014-03-10 09:51:35 PDT
(In reply to comment #9)
> (In reply to comment #2)
> >this test does nothing more than test the Inspector test harness. 
> 
> What is "the inspector test harness"?  DRT?  WebKitTestRunner?  Or some other executable?
> 
> I ask, because a WebKit app that enables LocalStorage is also required to explicitly shut down local storage before it exits.  DRT and WKTR do this.

I guess I misunderstood the significance of "Inspector test harness" as this just appears to be DRT crashing.
Comment 11 Timothy Hatcher 2014-03-12 11:38:31 PDT
(In reply to comment #10)
> (In reply to comment #9)
> > (In reply to comment #2)
> > >this test does nothing more than test the Inspector test harness. 
> > 
> > What is "the inspector test harness"?  DRT?  WebKitTestRunner?  Or some other executable?
> > 
> > I ask, because a WebKit app that enables LocalStorage is also required to explicitly shut down local storage before it exits.  DRT and WKTR do this.
> 
> I guess I misunderstood the significance of "Inspector test harness" as this just appears to be DRT crashing.

Yes, "Inspector test harness" just runs inside DRT and WKTR.
Comment 12 Alexey Proskuryakov 2014-03-29 23:05:35 PDT
Saw this on inspector/dom/content-flow-content-removal.html today. Did we get more tests that use the harness?
Comment 13 Brian Burg 2014-03-31 14:25:34 PDT
(In reply to comment #12)
> Saw this on inspector/dom/content-flow-content-removal.html today. Did we get more tests that use the harness?

Everything in the inspector/ directory was ported at the same time to the new test "harness". I think any test that uses a real inspector instance–everything under inspector/–could hit this assertion.
Comment 14 Brian Burg 2014-09-25 15:43:33 PDT
Created attachment 238679 [details]
Patch
Comment 15 Brady Eidson 2014-09-25 15:50:50 PDT
We really need an easy way to take sudden termination assertions while all these main thread operations are still outstanding.

This introduces a new one, but there's so many already that it's okay for now.
Comment 16 Alexey Proskuryakov 2014-09-25 16:24:23 PDT
Comment on attachment 238679 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=238679&action=review

> Source/WebCore/storage/StorageAreaSync.cpp:526
> +            String databaseIdentifierCopy = m_databaseIdentifier.isolatedCopy();
> +            callOnMainThread([databaseIdentifierCopy] {

This is not thread safe. A new copy of the string is made for the capture, and databaseIdentifierCopy races with it for derefing.
Comment 17 Brian Burg 2014-09-25 19:20:10 PDT
Committed r173991: <http://trac.webkit.org/changeset/173991>
Comment 18 Alexey Proskuryakov 2014-09-26 00:02:31 PDT
In the landed patch, what guarantees that "this" object is still alive when code executes on the main thread? I'm concerned that this might have introduced a potential use-after-free security bug.

If you just needed to pass a string, you could make an isolated StringImpl*, and manually delete it when done.
Comment 19 Brian Burg 2014-09-26 10:19:03 PDT
(In reply to comment #16)
> (From update of attachment 238679 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=238679&action=review
> 
> > Source/WebCore/storage/StorageAreaSync.cpp:526
> > +            String databaseIdentifierCopy = m_databaseIdentifier.isolatedCopy();
> > +            callOnMainThread([databaseIdentifierCopy] {
> 
> This is not thread safe. A new copy of the string is made for the capture, and databaseIdentifierCopy races with it for derefing.

I might be dense, but.. if the local is instead a String&& databaseIdentifierCopy, then this should cause capture to transfer the StringImpl ref from databesIdentifierCopy to the captured version, right?
Comment 20 Alexey Proskuryakov 2014-09-26 10:45:16 PDT
> if the local is instead a String&& databaseIdentifierCopy

I don't think that would compile.
Comment 21 Anders Carlsson 2014-09-26 11:01:14 PDT
(In reply to comment #20)
> > if the local is instead a String&& databaseIdentifierCopy
> 
> I don't think that would compile.

C++11 doesn't support capture-by-move in lambdas. C++14 does.
Comment 22 Brady Eidson 2014-09-26 11:29:26 PDT
(In reply to comment #17)
> Committed r173991: <http://trac.webkit.org/changeset/173991>

I might have r+'ed, but based on Alexey's feedback afterwards I don't think this patch should have been landed as-is.
Comment 23 Alexey Proskuryakov 2014-09-26 11:45:59 PDT
It's not evident from the bug, but Brian did account for my feedback when landing. That was still not quite correct, so he followed up with a better fix in <http://trac.webkit.org/changeset/174014>.
Comment 24 Brady Eidson 2014-09-26 13:03:49 PDT
(In reply to comment #23)
> It's not evident from the bug, but Brian did account for my feedback when landing. That was still not quite correct, so he followed up with a better fix in <http://trac.webkit.org/changeset/174014>.

Ack, my apologies - Saw the continued chatter without actually looking at the landed patch :)
Comment 25 Brian Burg 2014-09-26 13:19:54 PDT
(In reply to comment #24)
> (In reply to comment #23)
> > It's not evident from the bug, but Brian did account for my feedback when landing. That was still not quite correct, so he followed up with a better fix in <http://trac.webkit.org/changeset/174014>.
> 
> Ack, my apologies - Saw the continued chatter without actually looking at the landed patch :)

I assumed that commit-queue would add a link to the fixup commit here, but I must have used dcommit or something. Sorry for the confusion!