Bug 160782 – REGRESSION (r204226): LayoutTest editing/deleting/delete-empty-line-breaks-at-end-of-textarea.html "crashing" without a crashlog

• Component: New Bugs
• Status: RESOLVED FIXED

[Ryan Haddad]

LayoutTest editing/deleting/delete-empty-line-breaks-at-end-of-textarea.html "crashing" without a crashlog

Most recent "crash":
https://build.webkit.org/builders/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/builds/7275

Flakiness dashboard:
https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=editing%2Fdeleting%2Fdelete-empty-line-breaks-at-end-of-textarea.html

[Jonathan Bedard]

Let's wait on https://bugs.webkit.org/show_bug.cgi?id=160779, I suspect that this is the same bug we were seeing there, although I can't be sure because the --no-sample-on-timeout disables the output we would see before the test gets marked as having crashed.

Even if https://bugs.webkit.org/show_bug.cgi?id=160779 is the issue we're seeing here, we are still going to have a timeout occurring.  Why is this suite disabling sample on timeout?

[Ryan Haddad]

It looks like this happened again after https://trac.webkit.org/changeset/204514 landed

https://build.webkit.org/builders/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/builds/8541

[Jonathan Bedard]

I guess send that Radar back my way.  https://trac.webkit.org/changeset/204514 definitely fixed a problem, I guess it just wasn't this one.

[Ryan Haddad]

(In reply to comment #2)
> It looks like this happened again after
> https://trac.webkit.org/changeset/204514 landed
> 
> https://build.webkit.org/builders/
> Apple%20El%20Capitan%20Release%20WK2%20(Tests)/builds/8541

Sorry, that link should have been:
https://build.webkit.org/builders/Apple%20El%20Capitan%20Release%20WK2%20%28Tests%29/builds/8539

[Jonathan Bedard]

Taking a closer look at the history of editing/deleting/delete-empty-line-breaks-at-end-of-textarea.html, this seems to be a separate problem from radar 27786762.  I'm not sure what's causing this test to be marked as crashed, it's quite possible this is a true crash and it simply can't match the crash log.

Taking a look at https://build.webkit.org/builders/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/builds/8296/steps/layout-test/logs/stdio (which is where this bug first starts showing up) we can see process 78827 crashing, then 78817 is killed, then an attempt to find the crashlog of 78827 (which would seem to fail, otherwise the results would be attached) and then 78827 crashing for a second time in the next test.  This second piece should not be possible, as 78827 should have already crashed.

[Jonathan Bedard]

https://bugs.webkit.org/show_bug.cgi?id=160943 should help debug this.

[Jonathan Bedard]

https://build.webkit.org/builders/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/builds/8583 has this bug again.  None of the logging added in <http://trac.webkit.org/changeset/204577> caught it, so somehow, a test is being erroneously marked as crashed outside of the framework we are using for marking tests as crashed.

[Jonathan Bedard]

Created attachment 286374 [details]
Patch

[Alexey Proskuryakov]

Comment on attachment 286374 [details]
Patch

Discussed in person.

[Jonathan Bedard]

Some additional information on this: This particular bug does not seem to be in the testing rig, but rather the test itself.  This particular error also seems to be related to the preceding test, editing/deleting/delete-emoji.html.  In short, the WebProcess is exiting unexpectedly, but with an exit() instead of an abort(), meaning no crash-log is generated.

On one occasion, I have managed to reproduced this error locally, but without the debugger enabled.

[Ryan Haddad]

Marked test as flaky on El Capitan in http://trac.webkit.org/projects/webkit/changeset/204650 to get the bots back to green.

[Alexey Proskuryakov]

What happens here is that editing/deleting/delete-emoji.html is extremely slow to repaint, and sometimes it repaints after finishing. When this happens, WebKitTestRunner cannot reset WebContent process for the next test, and attempts to terminate it, and to launch a new one.

However, calling WKPageTerminate() results in a processDidCrash() callback, and then WebKitTestRunner itself terminates with an exit(). It seems suspicious that we get a processDidCrash() when intentionally terminating.

    frame #12: 0x00007fff9857c767 libsystem_c.dylib`exit + 55
    frame #13: 0x00000001002f5b30 WebKitTestRunner`WTR::TestController::processDidCrash(this=0x00007fff5f9667a0) + 576 at TestController.cpp:1730
    frame #14: 0x00000001002daf2c WebKitTestRunner`WTR::TestController::processDidCrash(page=0x000061d000039880, clientInfo=0x00007fff5f9667a0) + 28 at TestController.cpp:1518
    frame #15: 0x00000001099ef958 WebKit`WKPageSetPageNavigationClient::NavigationClient::processDidCrash(this=0x0000610000049540, page=0x000061d000039898) + 264 at WKPage.cpp:2325
    frame #16: 0x000000010902b6c8 WebKit`WebKit::WebPageProxy::processDidCrash(this=0x000061d000039898) + 792 at WebPageProxy.cpp:5190
    frame #17: 0x00000001095cce52 WebKit`WebKit::WebProcessProxy::didClose(this=0x0000619000035780, (null)=0x000061400009d240) + 434 at WebProcessProxy.cpp:541
    frame #18: 0x00000001095d19d5 WebKit`WebKit::WebProcessProxy::requestTermination(this=0x0000619000035780) + 197 at WebProcessProxy.cpp:808
    frame #19: 0x0000000109000366 WebKit`WebKit::WebPageProxy::terminateProcess(this=0x000061d000039898) + 230 at WebPageProxy.cpp:2355
    frame #20: 0x00000001099bfdfd WebKit`::WKPageTerminate(pageRef=0x000061d000039880) + 29 at WKPage.cpp:409
    frame #21: 0x00000001002e2c8d WebKitTestRunner`WTR::TestController::terminateWebContentProcess(this=0x00007fff5f9667a0) + 109 at TestController.cpp:805
    frame #22: 0x00000001003389ec WebKitTestRunner`WTR::TestInvocation::invoke(this=0x0000611000079d00) + 2940 at TestInvocation.cpp:180
    frame #23: 0x00000001002ef2b9 WebKitTestRunner`WTR::TestController::runTest(this=<unavailable>, inputLine=<unavailable>) + 4873 at TestController.cpp:1098
    frame #24: 0x00000001002f1f5c WebKitTestRunner`WTR::TestController::runTestingServerLoop(this=0x00007fff5f9667a0) + 524 at TestController.cpp:1115
    frame #25: 0x00000001002d8a2f WebKitTestRunner`WTR::TestController::run(this=0x00007fff5f9667a0) + 159 at TestController.cpp:1123
    frame #26: 0x00000001002d7645 WebKitTestRunner`WTR::TestController::TestController(this=<unavailable>, argc=<unavailable>, argv=<unavailable>) + 8069 at TestController.cpp:126
    frame #27: 0x00000001002d8d73 WebKitTestRunner`WTR::TestController::TestController(this=0x00007fff5f9667a0, argc=2, argv=0x00007fff5f966a30) + 35 at TestController.cpp:123

[Alexey Proskuryakov]

What's still unclear is why this only started to happen recently, around August 5th.

[Alexey Proskuryakov]

Found it, regressed with <https://trac.webkit.org/changeset/204226>.

The crash a few hours prior to that was a real one, a gamepad patch was making lots of tests crash.

[Jonathan Bedard]

Created attachment 286738 [details]
Patch

[Chris Dumez]

Comment on attachment 286738 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=286738&action=review

> Source/WebKit2/ChangeLog:8
> +        Backing out r204226.

r204243 is the last version of this change.

[Chris Dumez]

Comment on attachment 286738 [details]
Patch

This should also remove the change to WebPageProxy, please see:
https://trac.webkit.org/changeset/204243

[Chris Dumez]

I did a roll out in <http://trac.webkit.org/changeset/204841>. Just unskip the test.

[Jonathan Bedard]

Created attachment 286747 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 286747 [details]
Patch

Clearing flags on attachment: 286747

Committed r204845: <http://trac.webkit.org/changeset/204845>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.