ASSERTION FAILED: contextConnection under WebCore::SWServer::terminateWorkerInternal

[Ryan Haddad]

Created attachment 344337 [details]
Crash log

The following assertion failure was seen on High Sierra WK2 with LayoutTest http/tests/workers/service/postmessage-after-sw-process-crash.https.html:

ASSERTION FAILED: contextConnection
./workers/service/server/SWServer.cpp(630) : void WebCore::SWServer::terminateWorkerInternal(WebCore::SWServerWorker &, WebCore::SWServer::TerminationMode)
1   0x1152562b9 WTFCrash
2   0x108fceafd WebCore::SWServer::terminateWorkerInternal(WebCore::SWServerWorker&, WebCore::SWServer::TerminationMode)
3   0x108fce922 WebCore::SWServer::terminateWorker(WebCore::SWServerWorker&)
4   0x108fcbc16 WebCore::SWServerWorker::terminate()
5   0x108fc90ac WebCore::SWServerRegistration::clear()
6   0x108fc8cbf WebCore::SWServer::clearAll(WTF::CompletionHandler<void ()>&&)
7   0x100b2fc7f WebKit::StorageProcess::deleteWebsiteData(PAL::SessionID, WTF::OptionSet<WebKit::WebsiteDataType>, WTF::WallTime, unsigned long long)
8   0x100b53e37 void IPC::callMemberFunctionImpl<WebKit::StorageProcess, void (WebKit::StorageProcess::*)(PAL::SessionID, WTF::OptionSet<WebKit::WebsiteDataType>, WTF::WallTime, unsigned long long), std::__1::tuple<PAL::SessionID, WTF::OptionSet<WebKit::WebsiteDataType>, WTF::WallTime, unsigned long long>, 0ul, 1ul, 2ul, 3ul>(WebKit::StorageProcess*, void (WebKit::StorageProcess::*)(PAL::SessionID, WTF::OptionSet<WebKit::WebsiteDataType>, WTF::WallTime, unsigned long long), std::__1::tuple<PAL::SessionID, WTF::OptionSet<WebKit::WebsiteDataType>, WTF::WallTime, unsigned long long>&&, std::__1::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul>)
9   0x100b53cc0 void IPC::callMemberFunction<WebKit::StorageProcess, void (WebKit::StorageProcess::*)(PAL::SessionID, WTF::OptionSet<WebKit::WebsiteDataType>, WTF::WallTime, unsigned long long), std::__1::tuple<PAL::SessionID, WTF::OptionSet<WebKit::WebsiteDataType>, WTF::WallTime, unsigned long long>, std::__1::integer_sequence<unsigned long, 0ul, 1ul, 2ul, 3ul> >(std::__1::tuple<PAL::SessionID, WTF::OptionSet<WebKit::WebsiteDataType>, WTF::WallTime, unsigned long long>&&, WebKit::StorageProcess*, void (WebKit::StorageProcess::*)(PAL::SessionID, WTF::OptionSet<WebKit::WebsiteDataType>, WTF::WallTime, unsigned long long))
10  0x100b519ff void IPC::handleMessage<Messages::StorageProcess::DeleteWebsiteData, WebKit::StorageProcess, void (WebKit::StorageProcess::*)(PAL::SessionID, WTF::OptionSet<WebKit::WebsiteDataType>, WTF::WallTime, unsigned long long)>(IPC::Decoder&, WebKit::StorageProcess*, void (WebKit::StorageProcess::*)(PAL::SessionID, WTF::OptionSet<WebKit::WebsiteDataType>, WTF::WallTime, unsigned long long))
11  0x100b50cb6 WebKit::StorageProcess::didReceiveStorageProcessMessage(IPC::Connection&, IPC::Decoder&)
12  0x100b2cd4b WebKit::StorageProcess::didReceiveMessage(IPC::Connection&, IPC::Decoder&)
13  0x10052ada3 IPC::Connection::dispatchMessage(IPC::Decoder&)
14  0x10051e2a8 IPC::Connection::dispatchMessage(std::__1::unique_ptr<IPC::Decoder, std::__1::default_delete<IPC::Decoder> >)
15  0x10052b8c4 IPC::Connection::dispatchOneIncomingMessage()
16  0x100543458 IPC::Connection::enqueueIncomingMessage(std::__1::unique_ptr<IPC::Decoder, std::__1::default_delete<IPC::Decoder> >)::$_14::operator()()
17  0x100543369 WTF::Function<void ()>::CallableWrapper<IPC::Connection::enqueueIncomingMessage(std::__1::unique_ptr<IPC::Decoder, std::__1::default_delete<IPC::Decoder> >)::$_14>::call()
18  0x11527b48b WTF::Function<void ()>::operator()() const
19  0x1152cfb63 WTF::RunLoop::performWork()
20  0x1152d0464 WTF::RunLoop::performWork(void*)
21  0x7fff2fdd6d81 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__
22  0x7fff2fe8e65c __CFRunLoopDoSource0
23  0x7fff2fdb9d30 __CFRunLoopDoSources0
24  0x7fff2fdb91ad __CFRunLoopRun
25  0x7fff2fdb8a07 CFRunLoopRunSpecific
26  0x7fff31ec2f26 -[NSRunLoop(NSRunLoop) runMode:beforeDate:]
27  0x7fff31ec2dfe -[NSRunLoop(NSRunLoop) run]
28  0x7fff584b7f2b _xpc_objc_main
29  0x7fff584b6baa xpc_main
30  0x10029e0ab main
31  0x7fff5815d015 start
LEAK: 4 WebPageProxy

[Radar WebKit Bug Importer]

<rdar://problem/41853270>

[Chris Dumez]

void WebSWServerToContextConnection::connectionClosed()
{
    // FIXME: Do what here...?
}

Shouldn't we the associated workers as terminated here?

[Ryan Haddad]

This appears to be a flaky crash, I have not attempted to repro locally:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fworkers%2Fservice%2Fpostmessage-after-sw-process-crash.https.html

[Chris Dumez]

SWServerWorker::contextConnection() relies on SWServerToContextConnection::connectionForOrigin(securityOrigin()).

if it returns null, it means the SWServerToContextConnection object was destroyed.
So this indicates that it is possible for SWServerToContextConnection to get destroyed but its SWServerWorkers still think they're running.

[Chris Dumez]

(In reply to Chris Dumez from comment #4)
> SWServerWorker::contextConnection() relies on
> SWServerToContextConnection::connectionForOrigin(securityOrigin()).
> 
> if it returns null, it means the SWServerToContextConnection object was
> destroyed.
> So this indicates that it is possible for SWServerToContextConnection to get
> destroyed but its SWServerWorkers still think they're running.

SWServerToContextConnection objects can get destroyed as a result of:
1. StorageProcess::swContextConnectionMayNoLongerBeNeeded() -> there is no longer any client for this origin.
2. StorageProcess::connectionToContextProcessWasClosed() -> Context process terminated or crashed

[Chris Dumez]

StorageProcess::connectionToContextProcessWasClosed() calls:
    for (auto& swServer : m_swServers.values())
        swServer->markAllWorkersForOriginAsTerminated(securityOrigin);

So I would expect its workers not to be in running state.

StorageProcess::swContextConnectionMayNoLongerBeNeeded() failed to mark the workers as terminated though..

[Chris Dumez]

Created attachment 344342 [details]
Patch

[EWS Watchlist]

Comment on attachment 344342 [details]
Patch

Attachment 344342 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/8447738

New failing tests:
accessibility/mac/set-value-editable-dispatch-events.html

[EWS Watchlist]

Created attachment 344357 [details]
Archive of layout-test-results from ews113 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-sierra  Platform: Mac OS X 10.12.6

[Ryan Haddad]

(In reply to Build Bot from comment #8)
> Comment on attachment 344342 [details]
> Patch
> 
> Attachment 344342 [details] did not pass mac-debug-ews (mac):
> Output: https://webkit-queues.webkit.org/results/8447738
> 
> New failing tests:
> accessibility/Mac/set-value-editable-dispatch-events.html
This failure is unrelated (and currently being investigated).

[youenn fablet]

Comment on attachment 344342 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=344342&action=review

> Source/WebKit/ChangeLog:9
> +        Make sure we mark corresponding SWServerWorkers as terminated when we terminated a

s/terminated/terminate/

[Chris Dumez]

Committed r233598: <https://trac.webkit.org/changeset/233598>