TestWebKitAPI.WebKit.WKHTTPCookieStoreWithoutProcessPool 2018-05-21 12:14:45.165 TestWebKitAPI[42919:17694317] *** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'Completion handler passed to -[CookieUIDelegate webView:runJavaScriptAlertPanelWithMessage:initiatedByFrame:completionHandler:] was not called' *** First throw call stack: ( 0 CoreFoundation 0x0000000111cab1e6 __exceptionPreprocess + 294 1 libobjc.A.dylib 0x00000001112ad031 objc_exception_throw + 48 2 CoreFoundation 0x0000000111d20975 +[NSException raise:format:] + 197 3 WebKit 0x00000001092d55bf _ZN6WebKit28CompletionHandlerCallCheckerD2Ev + 129 4 WebKit 0x00000001092df11c _ZNK3WTF20ThreadSafeRefCountedIN6WebKit28CompletionHandlerCallCheckerELNS_17DestructionThreadE0EE5derefEv + 36 5 WebKit 0x00000001093f0193 _ZZN3WTF8BlockPtrIFvvEE12fromCallableIZN6WebKit10UIDelegate8UIClient18runJavaScriptAlertEPNS4_12WebPageProxyERKNS_6StringEPNS4_13WebFrameProxyERKN7WebCore18SecurityOriginDataEONS_8FunctionIS1_EEE3$_1EES2_T_ENUlPKvE_8__invokeESO_ + 29 6 libsystem_blocks.dylib 0x00000001122ea98a _Block_release + 111 7 WebKit 0x00000001093ee344 _ZN6WebKit10UIDelegate8UIClient18runJavaScriptAlertEPNS_12WebPageProxyERKN3WTF6StringEPNS_13WebFrameProxyERKN7WebCore18SecurityOriginDataEONS4_8FunctionIFvvEEE + 376 8 WebKit 0x00000001094e7aa7 _ZN6WebKit12WebPageProxy18runJavaScriptAlertEyRKN7WebCore18SecurityOriginDataERKN3WTF6StringEONS5_17CompletionHandlerIFvvEEE + 199 9 WebKit 0x000000010950c4b9 _ZN3IPC20handleMessageDelayedIN8Messages12WebPageProxy18RunJavaScriptAlertEN6WebKit12WebPageProxyEMS5_FvyRKN7WebCore18SecurityOriginDataERKN3WTF6StringEONSA_17CompletionHandlerIFvvEEEEEEvRNS_10ConnectionERNS_7DecoderERNSt3__110unique_ptrINS_7EncoderENSO_14default_deleteISQ_EEEEPT0_T1_ + 175 10 WebKit 0x000000010930cde7 _ZN3IPC18MessageReceiverMap19dispatchSyncMessageERNS_10ConnectionERNS_7DecoderERNSt3__110unique_ptrINS_7EncoderENS5_14default_deleteIS7_EEEE + 141 11 WebKit 0x0000000109568e94 _ZN6WebKit15WebProcessProxy21didReceiveSyncMessageERN3IPC10ConnectionERNS1_7DecoderERNSt3__110unique_ptrINS1_7EncoderENS6_14default_deleteIS8_EEEE + 28 12 WebKit 0x00000001092d81a9 _ZN3IPC10Connection19dispatchSyncMessageERNS_7DecoderE + 209 13 WebKit 0x00000001092d5b20 _ZN3IPC10Connection15dispatchMessageENSt3__110unique_ptrINS_7DecoderENS1_14default_deleteIS3_EEEE + 104 14 WebKit 0x00000001092d83d1 _ZN3IPC10Connection18dispatchOneMessageEv + 177 15 JavaScriptCore 0x000000010836d9dc _ZN3WTF7RunLoop11performWorkEv + 236 16 JavaScriptCore 0x000000010836dc82 _ZN3WTF7RunLoop11performWorkEPv + 34 17 CoreFoundation 0x0000000111c4dbb1 __CFRUNLOOP_IS_CALLING_OUT_TO_A_SOURCE0_PERFORM_FUNCTION__ + 17 18 CoreFoundation 0x0000000111c324af __CFRunLoopDoSources0 + 271 19 CoreFoundation 0x0000000111c31a6f __CFRunLoopRun + 1263 20 CoreFoundation 0x0000000111c3130b CFRunLoopRunSpecific + 635 21 Foundation 0x0000000110ca8b4a -[NSRunLoop(NSRunLoop) runMode:beforeDate:] + 274 22 TestWebKitAPI 0x0000000107e56d54 _ZN13TestWebKitAPI4Util3runEPb + 119 23 TestWebKitAPI 0x0000000107e84eb9 _ZN47WebKit_WKHTTPCookieStoreWithoutProcessPool_Test8TestBodyEv + 1849 24 TestWebKitAPI 0x0000000107eb7018 _ZN7testing4Test3RunEv + 92 25 TestWebKitAPI 0x0000000107eb74d6 _ZN7testing8internal12TestInfoImpl3RunEv + 180 26 TestWebKitAPI 0x0000000107eb784e _ZN7testing8TestCase3RunEv + 196 27 TestWebKitAPI 0x0000000107ebaab8 _ZN7testing8internal12UnitTestImpl11RunAllTestsEv + 614 28 TestWebKitAPI 0x0000000107e070a4 _ZN13TestWebKitAPI15TestsController3runEiPPc + 120 29 TestWebKitAPI 0x0000000107e9c0d8 main + 322 30 libdyld.dylib 0x0000000112278955 start + 1 ) libc++abi.dylib: terminating with uncaught exception of type NSException Child process terminated with signal 6: Abort trap https://build.webkit.org/builders/Apple%20iOS%2011%20Simulator%20Release%20WK2%20%28Tests%29/builds/5109
It looks like this started with https://trac.webkit.org/changeset/231999/webkit
ASSERT_TRUE(cookies == "PersistentCookieName=CookieValue; SessionCookieName=CookieValue" || cookies == "SessionCookieName=CookieValue; PersistentCookieName=CookieValue"); We could try EXPECT_TRUE instead of ASSERT_TRUE here, to make sure the completionHandler gets called. However, I think this means the string comparison fails and the test is thus still flaky.
Created attachment 340990 [details] Patch
Comment on attachment 340990 [details] Patch r=me
Comment on attachment 340990 [details] Patch Clearing flags on attachment: 340990 Committed r232083: <https://trac.webkit.org/changeset/232083>
All reviewed patches have been landed. Closing bug.
<rdar://problem/40468716>
The test is no longer crashing on iOS after this change, but now it is failing again: TestWebKitAPI.WebKit.WKHTTPCookieStoreWithoutProcessPool /Volumes/Data/slave/ios-simulator-11-release/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKHTTPCookieStore.mm:381 Value of: cookies == "PersistentCookieName=CookieValue; SessionCookieName=CookieValue" || cookies == "SessionCookieName=CookieValue; PersistentCookieName=CookieValue" Actual: false Expected: true https://build.webkit.org/builders/Apple%20iOS%2011%20Simulator%20Release%20WK2%20%28Tests%29/builds/5186
Reopening to attach new patch.
Created attachment 341206 [details] Patch
The TestWebKitAPI.WebKit.WKHTTPCookieStoreWithoutProcessPool test is still failing on the iOS 11 Simulator WK2 tests. For example: <https://build.webkit.org/builders/Apple%20iOS%2011%20Simulator%20Release%20WK2%20(Tests)/builds/5259> Ran 1590 tests of 1611 with 1589 successful ------------------------------ Test suite failed Failed TestWebKitAPI.WebKit.WKHTTPCookieStoreWithoutProcessPool /Volumes/Data/slave/ios-simulator-11-release/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKHTTPCookieStore.mm:436 Value of: cookies == "PersistentCookieName=CookieValue; SessionCookieName=CookieValue" || cookies == "SessionCookieName=CookieValue; PersistentCookieName=CookieValue" Actual: false Expected: true Testing completed, Exit status: 3
(In reply to David Kilzer (:ddkilzer) from comment #11) > The TestWebKitAPI.WebKit.WKHTTPCookieStoreWithoutProcessPool test is still > failing on the iOS 11 Simulator WK2 tests. For example: > > <https://build.webkit.org/builders/ > Apple%20iOS%2011%20Simulator%20Release%20WK2%20(Tests)/builds/5259> > > Ran 1590 tests of 1611 with 1589 successful > ------------------------------ > Test suite failed > > Failed > > TestWebKitAPI.WebKit.WKHTTPCookieStoreWithoutProcessPool > > > /Volumes/Data/slave/ios-simulator-11-release/build/Tools/TestWebKitAPI/Tests/ > WebKitCocoa/WKHTTPCookieStore.mm:436 > Value of: cookies == "PersistentCookieName=CookieValue; > SessionCookieName=CookieValue" || cookies == "SessionCookieName=CookieValue; > PersistentCookieName=CookieValue" > Actual: false > Expected: true > > > Testing completed, Exit status: 3 Oops! Didn't see Comment #8 before posting this.
Comment on attachment 341206 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=341206&action=review r=me, but consider making changes in feedback. > Tools/TestWebKitAPI/Tests/WebKitCocoa/WKHTTPCookieStore.mm:380 > - auto cookies = String(message.UTF8String); > - EXPECT_TRUE(cookies == "PersistentCookieName=CookieValue; SessionCookieName=CookieValue" || cookies == "SessionCookieName=CookieValue; PersistentCookieName=CookieValue"); > + EXPECT_STREQ("PersistentCookieName=CookieValue,SessionCookieName=CookieValue", message.UTF8String); Much better to use EXPECT_STREQ() here so we can see how the strings differ when they fail! Reminder: The expected value will change from using a comma to a semi-colon if #2 is changed below. > Tools/TestWebKitAPI/Tests/WebKitCocoa/WKHTTPCookieStore.mm:401 > - NSString *alertCookieHTML = @"<script>alert(document.cookie);</script>"; > + NSString *alertCookieHTML = @"<script>var cookies = document.cookie.split(';'); for (let i = 0; i < cookies.length; i ++) {cookies[i] = cookies[i].trim();} cookies.sort(); alert(cookies);</script>"; This is okay, but two things: 1. There should be spaces around the '{' and '}' brackets in the inline JavaScript. 2. It looks like alert(cookies) uses a comma to separate cookies; I would prefer you use join(';') so the list still looks cookie-like. NSString *alertCookieHTML = @"<script>var cookies = document.cookie.split(';'); for (let i = 0; i < cookies.length; i ++) { cookies[i] = cookies[i].trim(); } cookies.sort(); alert(cookies.join(';'));</script>";
Created attachment 341464 [details] Patch
Comment on attachment 341464 [details] Patch r=me
Any reason not to land this? It would be good to get this test passing on the bots.
(In reply to Ryan Haddad from comment #16) > Any reason not to land this? It would be good to get this test passing on > the bots. Looking at the patch, I do not think it will fix the flakiness/failure on the bots. It will merely provide more useful output when failing.
Comment on attachment 341464 [details] Patch Clearing flags on attachment: 341464 Committed r232357: <https://trac.webkit.org/changeset/232357>
Here is the failure output now: TestWebKitAPI.WebKit.WKHTTPCookieStoreWithoutProcessPool /Volumes/Data/slave/ios-simulator-11-release/build/Tools/TestWebKitAPI/Tests/WebKitCocoa/WKHTTPCookieStore.mm:471 Value of: message.UTF8String Actual: "SessionCookieName=CookieValue" Expected: "PersistentCookieName=CookieValue; SessionCookieName=CookieValue" https://build.webkit.org/builders/Apple%20iOS%2011%20Simulator%20Release%20WK2%20%28Tests%29/builds/5348/steps/run-api-tests/logs/stdio
What is the current state of this bug? It seems like the first patch here got reverted in https://trac.webkit.org/r234730
We disabled this test on iOS as the change is breaking some apps. The problem is that network process may not have access to the cookie storage file the UI process is using because of sandbox.