I have created a series of tests: /http/tests/contentdispositionattachmentsandbox/referer-header-stripped* which will include a js: http/tests/contentdispositionattachmentsandbox/resources/referer-header-stripped.js. In the JS, it has a navigation function to simulate human interaction, as follows: function navigation() { // Due to the sandbox, it's not possible to run script in the iframe or even access its contentDocument. var element = document.getElementsByTagName("iframe")[0]; var x = element.offsetLeft + 10; var y = element.offsetTop + 10; if (window.testRunner) { if (window.eventSender) { eventSender.mouseMoveTo(x, y); eventSender.mouseDown(); eventSender.mouseUp(); } if (testRunner.runUIScript) testRunner.runUIScript("(function() { uiController.singleTapAtPoint(" + x + ", " + y + "); })()"); } } All the tests if run separately, will pass in iOS-simulator-wk2. Yet, if they are executed by the command: run-webkit-tests --ios-simulator ./http/tests/contentdispositionattachmentsandbox/referer-header-stripped* in a row, WebKitTestRunner will then crash. It could crash in any one of the test. Here are two examples: Regressions: Unexpected crashes (2) http/tests/contentdispositionattachmentsandbox/referer-header-stripped-with-meta-referer-no-referrer.html [ Crash ] http/tests/contentdispositionattachmentsandbox/referer-header-stripped-with-meta-referer-unsafe-url.html [ Crash ] Regressions: Unexpected crashes (2) http/tests/contentdispositionattachmentsandbox/referer-header-stripped-with-meta-referer-origin.html [ Crash ] http/tests/contentdispositionattachmentsandbox/referer-header-stripped.html [ Crash ] And the all crashes at the same location, here is the crash log: Crashed Thread: 0 Dispatch queue: com.apple.main-thread Exception Type: EXC_BAD_ACCESS (SIGSEGV) Exception Codes: EXC_I386_GPFLT Exception Note: EXC_CORPSE_NOTIFY Application Specific Information: CRASHING TEST: /contentdispositionattachmentsandbox/referer-header-stripped-with-meta-referer-unsafe-url.html CoreSimulator 201.3 - Device: iPhone 5s WebKit Tester0 - Runtime: iOS 9.2 (13D11) - DeviceType: iPhone 5s Thread 0 Crashed:: Dispatch queue: com.apple.main-thread 0 WebKitTestRunnerApp 0x00000001087e8924 bool WTF::IdentityHashTranslator<WTF::IntHash<unsigned int> >::equal<unsigned int, unsigned int>(unsigned int const&, unsigned int const&) + 20 (HashTable.h:284) 1 WebKitTestRunnerApp 0x00000001087e92a8 WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task>* WTF::HashTable<unsigned int, WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task>, WTF::KeyValuePairKeyExtractor<WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task> >, WTF::IntHash<unsigned int>, WTF::HashMap<unsigned int, WTR::UIScriptContext::Task, WTF::IntHash<unsigned int>, WTF::HashTraits<unsigned int>, WTF::HashTraits<WTR::UIScriptContext::Task> >::KeyValuePairTraits, WTF::HashTraits<unsigned int> >::lookup<WTF::IdentityHashTranslator<WTF::IntHash<unsigned int> >, unsigned int>(unsigned int const&) + 168 (HashTable.h:622) 2 WebKitTestRunnerApp 0x00000001087ea8ff WTF::HashTableIterator<unsigned int, WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task>, WTF::KeyValuePairKeyExtractor<WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task> >, WTF::IntHash<unsigned int>, WTF::HashMap<unsigned int, WTR::UIScriptContext::Task, WTF::IntHash<unsigned int>, WTF::HashTraits<unsigned int>, WTF::HashTraits<WTR::UIScriptContext::Task> >::KeyValuePairTraits, WTF::HashTraits<unsigned int> > WTF::HashTable<unsigned int, WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task>, WTF::KeyValuePairKeyExtractor<WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task> >, WTF::IntHash<unsigned int>, WTF::HashMap<unsigned int, WTR::UIScriptContext::Task, WTF::IntHash<unsigned int>, WTF::HashTraits<unsigned int>, WTF::HashTraits<WTR::UIScriptContext::Task> >::KeyValuePairTraits, WTF::HashTraits<unsigned int> >::find<WTF::IdentityHashTranslator<WTF::IntHash<unsigned int> >, unsigned int>(unsigned int const&) + 79 (HashTable.h:999) 3 WebKitTestRunnerApp 0x00000001087ea874 WTF::HashTable<unsigned int, WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task>, WTF::KeyValuePairKeyExtractor<WTF::KeyValuePair<unsigned int, WTR::UIScriptContext::Task> >, WTF::IntHash<unsigned int>, WTF::HashMap<unsigned int, WTR::UIScriptContext::Task, WTF::IntHash<unsigned int>, WTF::HashTraits<unsigned int>, WTF::HashTraits<WTR::UIScriptContext::Task> >::KeyValuePairTraits, WTF::HashTraits<unsigned int> >::find(unsigned int const&) + 36 (HashTable.h:392) 4 WebKitTestRunnerApp 0x00000001087ea67f WTF::HashMap<unsigned int, WTR::UIScriptContext::Task, WTF::IntHash<unsigned int>, WTF::HashTraits<unsigned int>, WTF::HashTraits<WTR::UIScriptContext::Task> >::find(unsigned int const&) + 47 (HashMap.h:242) 5 WebKitTestRunnerApp 0x00000001087e7c3e WTF::HashMap<unsigned int, WTR::UIScriptContext::Task, WTF::IntHash<unsigned int>, WTF::HashTraits<unsigned int>, WTF::HashTraits<WTR::UIScriptContext::Task> >::take(unsigned int const&) + 46 (HashMap.h:389) 6 WebKitTestRunnerApp 0x00000001087e7210 WTR::UIScriptContext::asyncTaskComplete(unsigned int) + 48 (UIScriptContext.cpp:97) 7 WebKitTestRunnerApp 0x00000001087af4a6 ___ZN3WTR18UIScriptController16singleTapAtPointEllPK13OpaqueJSValue_block_invoke + 38 (UIScriptControllerIOS.mm:84) 8 WebKitTestRunnerApp 0x00000001087a9081 -[HIDEventGenerator markerEventReceived:] + 225 (HIDEventGenerator.mm:426) 9 WebKitTestRunnerApp 0x00000001087a6b4d -[WebKitTestRunnerApp _handleHIDEvent:] + 61 (mainIOS.mm:78) 10 com.apple.UIKit 0x000000010b39b532 _UIApplicationHandleEventQueue + 4695 11 com.apple.CoreFoundation 0x000000010f9b2a31 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17 12 com.apple.CoreFoundation 0x000000010f9a88d7 __CFRunLoopDoSources0 + 423 13 com.apple.CoreFoundation 0x000000010f9a7e13 __CFRunLoopRun + 867 14 com.apple.CoreFoundation 0x000000010f9a7828 CFRunLoopRunSpecific + 488 15 com.apple.Foundation 0x000000010f0452f1 -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 267 16 WebKitTestRunnerApp 0x00000001087edb56 WTR::TestController::platformRunUntil(bool&, double) + 262 (TestControllerCocoa.mm:110) 17 WebKitTestRunnerApp 0x00000001087c1ab9 WTR::TestController::runUntil(bool&, double) + 73 (TestController.cpp:1106) 18 WebKitTestRunnerApp 0x00000001087dcdd1 WTR::TestInvocation::invoke() + 977 (TestInvocation.cpp:148) 19 WebKitTestRunnerApp 0x00000001087c73db WTR::TestController::runTest(char const*) + 1659 (TestController.cpp:1067) 20 WebKitTestRunnerApp 0x00000001087c80d8 WTR::TestController::runTestingServerLoop() + 184 (TestController.cpp:1083) 21 WebKitTestRunnerApp 0x00000001087bdc57 WTR::TestController::run() + 55 (TestController.cpp:1091) 22 WebKitTestRunnerApp 0x00000001087bd7e6 WTR::TestController::TestController(int, char const**) + 1478 (TestController.cpp:143) 23 WebKitTestRunnerApp 0x00000001087bde23 WTR::TestController::TestController(int, char const**) + 35 (TestController.cpp:144) 24 WebKitTestRunnerApp 0x00000001087a66ff -[WebKitTestRunnerApp _runTestController] + 47 (mainIOS.mm:45) 25 com.apple.Foundation 0x000000010f041067 __NSThreadPerformPerform + 283 26 com.apple.CoreFoundation 0x000000010f9b2a31 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17 27 com.apple.CoreFoundation 0x000000010f9a88d7 __CFRunLoopDoSources0 + 423 28 com.apple.CoreFoundation 0x000000010f9a7e13 __CFRunLoopRun + 867 29 com.apple.CoreFoundation 0x000000010f9a7828 CFRunLoopRunSpecific + 488 30 com.apple.GraphicsServices 0x00000001129d0ad2 GSEventRunModal + 161 31 com.apple.UIKit 0x000000010b3a1610 UIApplicationMain + 171 32 WebKitTestRunnerApp 0x00000001087a6ba3 main + 67 (mainIOS.mm:87) 33 libdyld.dylib 0x000000011030392d start + 1
Tests skipped on ios-simulator with <http://trac.webkit.org/changeset/194225>
Jiewen, why don't you try to fix this?
(In reply to comment #2) > Jiewen, why don't you try to fix this? I did multiple attempts to fix the test case in the morning with Wenson. However, none of them works. Therefore, Let's skip these test cases for the iOS simulator. I will come back and try to fix it later on.
<rdar://problem/23948321>
Created attachment 268478 [details] proposed fix This fixes bugs in WebKitTestRunner, so the tests pass reliably for me. However there is still something wrong with the tests too - the fact that completion sometimes only happens after a test finishes - yet the test passes - means that we don't actually test the operation whose completion it is! I will not be looking into that, but someone should.
Comment on attachment 268478 [details] proposed fix Clearing flags on attachment: 268478 Committed r194721: <http://trac.webkit.org/changeset/194721>
All reviewed patches have been landed. Closing bug.
(In reply to comment #5) > Created attachment 268478 [details] > proposed fix > > This fixes bugs in WebKitTestRunner, so the tests pass reliably for me. > > However there is still something wrong with the tests too - the fact that > completion sometimes only happens after a test finishes - yet the test > passes - means that we don't actually test the operation whose completion it > is! I will not be looking into that, but someone should. I might be able to take a step into it.
Bug 153140 is created to track the test case issue.