Bug 152390 – testRunner.runUIScript crashes while running multiple tests in a row that invokes the same UIScript

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Jiewen Tan]

I have created a series of tests: /http/tests/contentdispositionattachmentsandbox/referer-header-stripped* which will include a js: http/tests/contentdispositionattachmentsandbox/resources/referer-header-stripped.js. In the JS, it has a navigation function to simulate human interaction, as follows:
function navigation() {
    // Due to the sandbox, it's not possible to run script in the iframe or even access its contentDocument.
    var element = document.getElementsByTagName("iframe")[0];
    var x = element.offsetLeft + 10;
    var y = element.offsetTop + 10;

    if (window.testRunner) {
        if (window.eventSender) {
            eventSender.mouseMoveTo(x, y);
            eventSender.mouseDown();
            eventSender.mouseUp();
        }

        if (testRunner.runUIScript)
            testRunner.runUIScript("(function() { uiController.singleTapAtPoint(" + x + ", " + y + "); })()");
    }
}

All the tests if run separately, will pass in iOS-simulator-wk2. Yet, if they are executed by the command: run-webkit-tests --ios-simulator ./http/tests/contentdispositionattachmentsandbox/referer-header-stripped* in a row, WebKitTestRunner will then crash. It could crash in any one of the test. Here are two examples:
Regressions: Unexpected crashes (2)
  http/tests/contentdispositionattachmentsandbox/referer-header-stripped-with-meta-referer-no-referrer.html [ Crash ]
  http/tests/contentdispositionattachmentsandbox/referer-header-stripped-with-meta-referer-unsafe-url.html [ Crash ]
Regressions: Unexpected crashes (2)
  http/tests/contentdispositionattachmentsandbox/referer-header-stripped-with-meta-referer-origin.html [ Crash ]
  http/tests/contentdispositionattachmentsandbox/referer-header-stripped.html [ Crash ]
And the all crashes at the same location, here is the crash log:
Crashed Thread:        0  Dispatch queue: com.apple.main-thread

Exception Type:        EXC_BAD_ACCESS (SIGSEGV)
Exception Codes:       EXC_I386_GPFLT
Exception Note:        EXC_CORPSE_NOTIFY

Application Specific Information:
CRASHING TEST: /contentdispositionattachmentsandbox/referer-header-stripped-with-meta-referer-unsafe-url.html
CoreSimulator 201.3 - Device: iPhone 5s WebKit Tester0 - Runtime: iOS 9.2 (13D11) - DeviceType: iPhone 5s

Thread 0 Crashed:: Dispatch queue: com.apple.main-thread
0   WebKitTestRunnerApp           	0x00000001087e8924 bool WTF::IdentityHashTranslator<WTF::IntHash<unsigned int> >::equal<unsigned int, unsigned int>(unsigned int const&, unsigned int const&) + 20 (HashTable.h:284)
1   WebKitTestRunnerApp           	0x00000001087e92a8 WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task>* WTF::HashTable<unsigned int, WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task>, WTF::KeyValuePairKeyExtractor<WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task> >, WTF::IntHash<unsigned int>, WTF::HashMap<unsigned int, WTR::UIScriptContext::Task, WTF::IntHash<unsigned int>, WTF::HashTraits<unsigned int>, WTF::HashTraits<WTR::UIScriptContext::Task> >::KeyValuePairTraits, WTF::HashTraits<unsigned int> >::lookup<WTF::IdentityHashTranslator<WTF::IntHash<unsigned int> >, unsigned int>(unsigned int const&) + 168 (HashTable.h:622)
2   WebKitTestRunnerApp           	0x00000001087ea8ff WTF::HashTableIterator<unsigned int, WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task>, WTF::KeyValuePairKeyExtractor<WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task> >, WTF::IntHash<unsigned int>, WTF::HashMap<unsigned int, WTR::UIScriptContext::Task, WTF::IntHash<unsigned int>, WTF::HashTraits<unsigned int>, WTF::HashTraits<WTR::UIScriptContext::Task> >::KeyValuePairTraits, WTF::HashTraits<unsigned int> > WTF::HashTable<unsigned int, WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task>, WTF::KeyValuePairKeyExtractor<WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task> >, WTF::IntHash<unsigned int>, WTF::HashMap<unsigned int, WTR::UIScriptContext::Task, WTF::IntHash<unsigned int>, WTF::HashTraits<unsigned int>, WTF::HashTraits<WTR::UIScriptContext::Task> >::KeyValuePairTraits, WTF::HashTraits<unsigned int> >::find<WTF::IdentityHashTranslator<WTF::IntHash<unsigned int> >, unsigned int>(unsigned int const&) + 79 (HashTable.h:999)
3   WebKitTestRunnerApp           	0x00000001087ea874 WTF::HashTable<unsigned int, WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task>, WTF::KeyValuePairKeyExtractor<WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task> >, WTF::IntHash<unsigned int>, WTF::HashMap<unsigned int, WTR::UIScriptContext::Task, WTF::IntHash<unsigned int>, WTF::HashTraits<unsigned int>, WTF::HashTraits<WTR::UIScriptContext::Task> >::KeyValuePairTraits, WTF::HashTraits<unsigned int> >::find(unsigned int const&) + 36 (HashTable.h:392)
4   WebKitTestRunnerApp           	0x00000001087ea67f WTF::HashMap<unsigned int, WTR::UIScriptContext::Task, WTF::IntHash<unsigned int>, WTF::HashTraits<unsigned int>, WTF::HashTraits<WTR::UIScriptContext::Task> >::find(unsigned int const&) + 47 (HashMap.h:242)
5   WebKitTestRunnerApp           	0x00000001087e7c3e WTF::HashMap<unsigned int, WTR::UIScriptContext::Task, WTF::IntHash<unsigned int>, WTF::HashTraits<unsigned int>, WTF::HashTraits<WTR::UIScriptContext::Task> >::take(unsigned int const&) + 46 (HashMap.h:389)
6   WebKitTestRunnerApp           	0x00000001087e7210 WTR::UIScriptContext::asyncTaskComplete(unsigned int) + 48 (UIScriptContext.cpp:97)
7   WebKitTestRunnerApp           	0x00000001087af4a6 ___ZN3WTR18UIScriptController16singleTapAtPointEllPK13OpaqueJSValue_block_invoke + 38 (UIScriptControllerIOS.mm:84)
8   WebKitTestRunnerApp           	0x00000001087a9081 -[HIDEventGenerator markerEventReceived:] + 225 (HIDEventGenerator.mm:426)
9   WebKitTestRunnerApp           	0x00000001087a6b4d -[WebKitTestRunnerApp _handleHIDEvent:] + 61 (mainIOS.mm:78)
10  com.apple.UIKit               	0x000000010b39b532 _UIApplicationHandleEventQueue + 4695
11  com.apple.CoreFoundation      	0x000000010f9b2a31 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
12  com.apple.CoreFoundation      	0x000000010f9a88d7 __CFRunLoopDoSources0 + 423
13  com.apple.CoreFoundation      	0x000000010f9a7e13 __CFRunLoopRun + 867
14  com.apple.CoreFoundation      	0x000000010f9a7828 CFRunLoopRunSpecific + 488
15  com.apple.Foundation          	0x000000010f0452f1 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 267
16  WebKitTestRunnerApp           	0x00000001087edb56 WTR::TestController::platformRunUntil(bool&, double) + 262 (TestControllerCocoa.mm:110)
17  WebKitTestRunnerApp           	0x00000001087c1ab9 WTR::TestController::runUntil(bool&, double) + 73 (TestController.cpp:1106)
18  WebKitTestRunnerApp           	0x00000001087dcdd1 WTR::TestInvocation::invoke() + 977 (TestInvocation.cpp:148)
19  WebKitTestRunnerApp           	0x00000001087c73db WTR::TestController::runTest(char const*) + 1659 (TestController.cpp:1067)
20  WebKitTestRunnerApp           	0x00000001087c80d8 WTR::TestController::runTestingServerLoop() + 184 (TestController.cpp:1083)
21  WebKitTestRunnerApp           	0x00000001087bdc57 WTR::TestController::run() + 55 (TestController.cpp:1091)
22  WebKitTestRunnerApp           	0x00000001087bd7e6 WTR::TestController::TestController(int, char const**) + 1478 (TestController.cpp:143)
23  WebKitTestRunnerApp           	0x00000001087bde23 WTR::TestController::TestController(int, char const**) + 35 (TestController.cpp:144)
24  WebKitTestRunnerApp           	0x00000001087a66ff -[WebKitTestRunnerApp _runTestController] + 47 (mainIOS.mm:45)
25  com.apple.Foundation          	0x000000010f041067 __NSThreadPerformPerform + 283
26  com.apple.CoreFoundation      	0x000000010f9b2a31 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17
27  com.apple.CoreFoundation      	0x000000010f9a88d7 __CFRunLoopDoSources0 + 423
28  com.apple.CoreFoundation      	0x000000010f9a7e13 __CFRunLoopRun + 867
29  com.apple.CoreFoundation      	0x000000010f9a7828 CFRunLoopRunSpecific + 488
30  com.apple.GraphicsServices    	0x00000001129d0ad2 GSEventRunModal + 161
31  com.apple.UIKit               	0x000000010b3a1610 UIApplicationMain + 171
32  WebKitTestRunnerApp           	0x00000001087a6ba3 main + 67 (mainIOS.mm:87)
33  libdyld.dylib                 	0x000000011030392d start + 1

[Ryan Haddad]

Tests skipped on ios-simulator with <http://trac.webkit.org/changeset/194225>

[Simon Fraser (smfr)]

Jiewen, why don't you try to fix this?

[Jiewen Tan]

(In reply to comment #2)
> Jiewen, why don't you try to fix this?

I did multiple attempts to fix the test case in the morning with Wenson. However, none of them works. Therefore, Let's skip these test cases for the iOS simulator. I will come back and try to fix it later on.

[Jiewen Tan]

<rdar://problem/23948321>

[Alexey Proskuryakov]

Created attachment 268478 [details]
proposed fix

This fixes bugs in WebKitTestRunner, so the tests pass reliably for me.

However there is still something wrong with the tests too - the fact that completion sometimes only happens after a test finishes - yet the test passes - means that we don't actually test the operation whose completion it is! I will not be looking into that, but someone should.

[WebKit Commit Bot]

Comment on attachment 268478 [details]
proposed fix

Clearing flags on attachment: 268478

Committed r194721: <http://trac.webkit.org/changeset/194721>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Jiewen Tan]

(In reply to comment #5)
> Created attachment 268478 [details]
> proposed fix
> 
> This fixes bugs in WebKitTestRunner, so the tests pass reliably for me.
> 
> However there is still something wrong with the tests too - the fact that
> completion sometimes only happens after a test finishes - yet the test
> passes - means that we don't actually test the operation whose completion it
> is! I will not be looking into that, but someone should.

I might be able to take a step into it.

[Jiewen Tan]

Bug 153140 is created to track the test case issue.