Bug 236794 – [iOS] Tests with incomplete UIScripts cause flaky crashes under WebKit::RemoteLayerTreeDrawingAreaProxy::commitLayerTree()

• Status: ASSIGNED
• Component: New Bugs

[Ryan Haddad]

Created attachment 452399 [details]
crash log

editing/spelling/editing-word-with-marker-2.html has become a flaky crash on iOS release bots

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   org.webkit.WebKitTestRunnerApp	0x000000010b137a13 WTFCrashWithInfo(int, char const*, char const*, int) + 19
1   org.webkit.WebKitTestRunnerApp	0x000000010b16ab00 WTR::TestInvocation::runUISideScriptImmediately(OpaqueWKError const*, void*) + 98
2   com.apple.WebKit              	0x0000000116215394 operator() + 11 (Function.h:82) [inlined]
3   com.apple.WebKit              	0x0000000116215394 WebKit::GenericCallback<>::performCallbackWithReturnValue() + 40 (GenericCallback.h:109)
4   com.apple.WebKit              	0x00000001162114af performCallback + 5 (GenericCallback.h:114) [inlined]
5   com.apple.WebKit              	0x00000001162114af WebKit::RemoteLayerTreeDrawingAreaProxy::commitLayerTree(WebKit::RemoteLayerTreeTransaction const&, WebKit::RemoteScrollingCoordinatorTransaction const&) + 593 (RemoteLayerTreeDrawingAreaProxy.mm:287)
6   com.apple.WebKit              	0x0000000115f258b3 callMemberFunctionImpl<WebKit::RemoteLayerTreeDrawingAreaProxy, void (WebKit::RemoteLayerTreeDrawingAreaProxy::*)(const WebKit::RemoteLayerTreeTransaction &, const WebKit::RemoteScrollingCoordinatorTransaction &), std::tuple<WebKit::RemoteLayerTreeTransaction, WebKit::RemoteScrollingCoordinatorTransaction>, 0, 1> + 22 (HandleMessage.h:125) [inlined]
7   com.apple.WebKit              	0x0000000115f258b3 callMemberFunction<WebKit::RemoteLayerTreeDrawingAreaProxy, void (WebKit::RemoteLayerTreeDrawingAreaProxy::*)(const WebKit::RemoteLayerTreeTransaction &, const WebKit::RemoteScrollingCoordinatorTransaction &), std::tuple<WebKit::RemoteLayerTreeTransaction, WebKit::RemoteScrollingCoordinatorTransaction>, std::integer_sequence<unsigned long, 0, 1> > + 22 (HandleMessage.h:131) [inlined]
8   com.apple.WebKit              	0x0000000115f258b3 handleMessage<Messages::RemoteLayerTreeDrawingAreaProxy::CommitLayerTree, WebKit::RemoteLayerTreeDrawingAreaProxy, void (WebKit::RemoteLayerTreeDrawingAreaProxy::*)(const WebKit::RemoteLayerTreeTransaction &, const WebKit::RemoteScrollingCoordinatorTransaction &)> + 47 (HandleMessage.h:196) [inlined]
9   com.apple.WebKit              	0x0000000115f258b3 WebKit::RemoteLayerTreeDrawingAreaProxy::didReceiveMessage(IPC::Connection&, IPC::Decoder&) + 273 (RemoteLayerTreeDrawingAreaProxyMessageReceiver.cpp:44)
10  com.apple.WebKit              	0x00000001161b15ca IPC::MessageReceiverMap::dispatchMessage(IPC::Connection&, IPC::Decoder&) + 224 (MessageReceiverMap.cpp:129)
11  com.apple.WebKit              	0x00000001163889ae WebKit::WebProcessProxy::didReceiveMessage(IPC::Connection&, IPC::Decoder&) + 24 (WebProcessProxy.cpp:859)
12  com.apple.WebKit              	0x00000001161aac9a IPC::Connection::dispatchMessage(std::__1::unique_ptr<IPC::Decoder, std::__1::default_delete<IPC::Decoder> >) + 238 (Connection.cpp:1137)
13  com.apple.WebKit              	0x00000001161aa3bd IPC::Connection::dispatchIncomingMessages() + 377 (Connection.cpp:1241)
14  com.apple.JavaScriptCore      	0x000000010f7f418f operator() + 9 (Function.h:82) [inlined]
15  com.apple.JavaScriptCore      	0x000000010f7f418f WTF::RunLoop::performWork() + 431 (RunLoop.cpp:133)
16  com.apple.JavaScriptCore      	0x000000010f7f4a62 WTF::RunLoop::performWork(void*) + 34 (RunLoopCF.cpp:46)
17  com.apple.CoreFoundation      	0x000000010ef61e25 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
18  com.apple.CoreFoundation      	0x000000010ef61d1d __CFRunLoopDoSource0 + 180
19  com.apple.CoreFoundation      	0x000000010ef611f2 __CFRunLoopDoSources0 + 242
20  com.apple.CoreFoundation      	0x000000010ef5b951 __CFRunLoopRun + 875
21  com.apple.CoreFoundation      	0x000000010ef5b103 CFRunLoopRunSpecific + 567
22  com.apple.Foundation          	0x000000010d98e41c -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 213
23  org.webkit.WebKitTestRunnerApp	0x000000010b15cf7e WTR::TestController::platformRunUntil(bool&, WTF::Seconds) + 184
24  org.webkit.WebKitTestRunnerApp	0x000000010b166743 WTR::TestInvocation::invoke() + 393
25  org.webkit.WebKitTestRunnerApp	0x000000010b14db92 WTR::TestController::runTest(char const*) + 330
26  org.webkit.WebKitTestRunnerApp	0x000000010b14de24 WTR::TestController::runTestingServerLoop() + 128
27  org.webkit.WebKitTestRunnerApp	0x000000010b148dd1 WTR::TestController::TestController(int, char const**) + 479
28  org.webkit.WebKitTestRunnerApp	0x000000010b137ae8 -[WebKitTestRunnerApp _runTestController] + 40
29  com.apple.Foundation          	0x000000010d9b758c __NSThreadPerformPerform + 207
30  com.apple.CoreFoundation      	0x000000010ef61e25 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
31  com.apple.CoreFoundation      	0x000000010ef61d1d __CFRunLoopDoSource0 + 180
32  com.apple.CoreFoundation      	0x000000010ef61254 __CFRunLoopDoSources0 + 340
33  com.apple.CoreFoundation      	0x000000010ef5b951 __CFRunLoopRun + 875
34  com.apple.CoreFoundation      	0x000000010ef5b103 CFRunLoopRunSpecific + 567
35  com.apple.GraphicsServices    	0x000000010bbd4cd3 GSEventRunModal + 139
36  com.apple.UIKitCore           	0x000000011f1ebe63 -[UIApplication _run] + 928
37  com.apple.UIKitCore           	0x000000011f1f0a53 UIApplicationMain + 101
38  org.webkit.WebKitTestRunnerApp	0x000000010b137be6 main + 65
39  dyld_sim                      	0x000000010b258e1e start_sim + 10
40  ???                           	0x0000000000000001 0 + 1
41  ???                           	0x0000000000000002 0 + 2


https://results.webkit.org/?suite=layout-tests&test=editing%2Fspelling%2Fediting-word-with-marker-2.html

[Ryan Haddad]

Based on test history, this appears to have regressed somewhere between r289765 and r289784

[Radar WebKit Bug Importer]

<rdar://problem/89100788>

[Ryan Haddad]

https://trac.webkit.org/log/webkit/?action=stop_on_copy&mode=stop_on_copy&rev=289784&stop_rev=289765&limit=100&verbose=on

[Simon Fraser (smfr)]

Seems like this is:
        RELEASE_ASSERT(TestController::singleton().isCurrentInvocation(invocation));

[Robert Jenner]

This has been slowing down iOS EWS and has been flagged as a flaky crash:
https://ews-build.webkit.org/#/builders/68/builds/9053


I have marked expectations for this here:
https://trac.webkit.org/changeset/290462/webkit

[Alexey Proskuryakov]

Seeing more tests with this stack trace, related.

[Dawn Morningstar]

*** Bug 237221 has been marked as a duplicate of this bug. ***

[Ryan Haddad]

*** Bug 238229 has been marked as a duplicate of this bug. ***

[Dawn Morningstar]

*** Bug 238229 has been marked as a duplicate of this bug. ***

[Dawn Morningstar]

*** Bug 238284 has been marked as a duplicate of this bug. ***

[Dawn Morningstar]

*** Bug 238301 has been marked as a duplicate of this bug. ***

[Simon Fraser (smfr)]

Unable to reproduce with

run-webkit-tests --ios-simulator --release imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-052.html editing/spelling/spellcheck-async-remove-frame.html imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-050.html  editing/spelling/spellcheck-api-crash.html editing/spelling/editing-word-with-marker-2.html --iterations=20

[Simon Fraser (smfr)]

Aha, I did get it once with --ios-simulator --release imported/w3c/web-platform-tests/css/css-contain/content-visibility/ editing/spelling --iterations=5

[Dawn Morningstar]

(In reply to Simon Fraser (smfr) from comment #13)
> Aha, I did get it once with --ios-simulator --release
> imported/w3c/web-platform-tests/css/css-contain/content-visibility/
> editing/spelling --iterations=5

Fantastic! I was also unable to reproduce locally. Very Very Flaky.

[Wenson Hsieh]

This happens when a test finishes with unfired UI script callbacks.

One (pretty trivial) way to reproduce this to run editing/spelling/editing-word-with-marker-1.html in a loop (it reproduces about every other iteration).

[Simon Fraser (smfr)]

editing/spelling/editing-word-with-marker-1.html has a bunch of missing 'awaits'.

[Simon Fraser (smfr)]

Created attachment 455970 [details]
Path for EWS to see how many tests are bad

[Simon Fraser (smfr)]

iOS results shows that the following tests can cause this crash:

editing/spelling/editing-word-with-marker-1.html
fast/events/ios/pdf-modifer-key-down-crash.html
fast/media/mq-inverted-colors-live-update-for-listener.html
fast/media/mq-prefers-contrast-live-update-for-listener.html
fast/media/mq-prefers-reduced-motion-live-update-for-listener.html
imported/w3c/web-platform-tests/clipboard-apis/async-raw-write-read.tentative.https.html
imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-047.html
imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-048.html
imported/w3c/web-platform-tests/html/user-activation/no-activation-thru-escape-key.html
platform/ios/fast/scrolling/find-text-in-overflow-node-indicator-position-limit.html
platform/ios/fast/scrolling/find-text-in-overflow-node-indicator-position.html
swipe/main-frame-pinning-requirement.html

[Rob Buis]

(In reply to Simon Fraser (smfr) from comment #18)
> iOS results shows that the following tests can cause this crash:
> 
> imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-
> visibility-047.html
> imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-
> visibility-048.html

Note that content-visibility-048.html currently is a bit broken, in the sense that
it relies on scroll-to-text-fragment WPT dir which is not yet imported. The only
other similarity here that I see is both these using test-driver. No idea if that
is related to the reported crash though.

[Ryan Haddad]

*** Bug 240836 has been marked as a duplicate of this bug. ***

[Karl Rackler]

REPRODUCTION STEPS
I can reproduce this crash on ToT r294877

Command: 
run-webkit-tests --ios-simulator --no-retry --child-processes 1 imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-048.html imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-052.html

[Simon Fraser (smfr)]

We have a clear understanding of the bug and how to fix it, so no need for more repro info.

[Ryan Haddad]

(In reply to Simon Fraser (smfr) from comment #22)
> We have a clear understanding of the bug and how to fix it, so no need for
> more repro info.
Great! Do you think it can be fixed soon, or should we skip all the tests you called out in comment #18?

[Karl Rackler]

(In reply to Simon Fraser (smfr) from comment #22)
> We have a clear understanding of the bug and how to fix it, so there is no need for
> more repro info.

Hello Simon,
Understood.  It looks like Robert created bug 237159 and set up test expectations for imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-049.html.  
That seems to shuffle the test order from imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-049.html to imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-048.html imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-052.html and fail the next test in order.  That is where I noticed it on Bug 240836. 
 
I did not discover this bug until later and added my repro steps.  
Since we have a clear understanding of the issue, I will not add test expectations for imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-052.html and hold for the fix unless you advise otherwise.  Thank you for being so helpful on this!

[Ryan Haddad]

*** Bug 237159 has been marked as a duplicate of this bug. ***

[Nikos Mouchtaris]

Pull request: https://github.com/WebKit/WebKit/pull/1916

[Simon Fraser (smfr)]

These should be resolved by updating WPT for the relevant directories:
imported/w3c/web-platform-tests/clipboard-apis/async-raw-write-read.tentative.https.html
imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-047.html
imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-048.html
imported/w3c/web-platform-tests/html/user-activation/no-activation-thru-escape-key.html

[Simon Fraser (smfr)]

Created attachment 461389 [details]
Retest to see if any failures remain

[Simon Fraser (smfr)]

Created attachment 461390 [details]
Retest

[Simon Fraser (smfr)]

Created attachment 461392 [details]
Retest

[Simon Fraser (smfr)]

Pull request: https://github.com/WebKit/WebKit/pull/3151

[Simon Fraser (smfr)]

These tests still need fixing:

accessibility/ios-simulator/accessibility-make-first-responder.html
fast/forms/textfield-outline.html
imported/w3c/web-platform-tests/css/css-contain/content-visibility/content-visibility-048.html
imported/w3c/web-platform-tests/html/user-activation/no-activation-thru-escape-key.html