RESOLVED FIXED 196006
Flaky API Test TestWebKitAPI.KeyboardInputTests.KeyboardTypeForInput 
https://bugs.webkit.org/show_bug.cgi?id=196006
Summary Flaky API Test TestWebKitAPI.KeyboardInputTests.KeyboardTypeForInput
Aakash Jain
Reported 2019-03-20 07:43:08 PDT
TestWebKitAPI.KeyboardInputTests.KeyboardTypeForInput seems flaky. In https://ews-build.webkit-uat.org/#/builders/20/builds/2040, the test Failed in run-api-tests step. However, in the immediately next retry step (re-run-api-tests), it passed.
Attachments
Add attachment proposed patch, testcase, etc.
Aakash Jain
Comment 1 2019-03-20 09:17:47 PDT
This test was added in https://trac.webkit.org/changeset/235201
Aakash Jain
Comment 2 2019-03-21 14:00:32 PDT
Happened again in https://ews-build.webkit-uat.org/#/builders/20/builds/2198 TestWebKitAPI.KeyboardInputTests.KeyboardTypeForInput 2019-03-21 12:08:07.471 TestWebKitAPI[32628:65791725] Displayed 7 for <input type='search' inputmode='tel' pattern=''>. Expected 5. 2019-03-21 12:08:09.257 TestWebKitAPI[32628:65791725] Displayed 5 for <input type='tel' inputmode='numeric' pattern=''>. Expected 2. /Volumes/Data/worker/iOS-11-Simulator-Build-EWS/build/Tools/TestWebKitAPI/Tests/ios/KeyboardInputTestsIOS.mm:413 Value of: runTest(inputType, inputMode, pattern, (UIKeyboardType)keyboardType.intValue) Actual: false Expected: true /Volumes/Data/worker/iOS-11-Simulator-Build-EWS/build/Tools/TestWebKitAPI/Tests/ios/KeyboardInputTestsIOS.mm:413 Value of: runTest(inputType, inputMode, pattern, (UIKeyboardType)keyboardType.intValue) Actual: false Expected: true
Aakash Jain
Comment 3 2019-03-22 06:17:09 PDT
Happening very frequently, e.g.: https://ews-build.webkit-uat.org/#/builders/20/builds/2232 https://ews-build.webkit-uat.org/#/builders/20/builds/2224 https://ews-build.webkit-uat.org/#/builders/20/builds/2205 TestWebKitAPI.KeyboardInputTests.KeyboardTypeForInput 2019-03-21 17:39:00.638 TestWebKitAPI[20388:66464848] Displayed 8 for <input type='search' inputmode='url' pattern=''>. Expected 3. /Volumes/Data/worker/iOS-12-Simulator-Build-EWS/build/Tools/TestWebKitAPI/Tests/ios/KeyboardInputTestsIOS.mm:413 Value of: runTest(inputType, inputMode, pattern, (UIKeyboardType)keyboardType.intValue) Actual: false Expected: true TestWebKitAPI.KeyboardInputTests.KeyboardTypeForInput 2019-03-21 13:58:33.025 TestWebKitAPI[5458:274706] Displayed 3 for <input type='text' inputmode='' pattern=''>. Expected 0. /Volumes/Data/worker/iOS-12-Simulator-Build-EWS/build/Tools/TestWebKitAPI/Tests/ios/KeyboardInputTestsIOS.mm:413 Value of: runTest(inputType, inputMode, pattern, (UIKeyboardType)keyboardType.intValue) Actual: false Expected: true
Aakash Jain
Comment 4 2019-03-24 04:52:24 PDT
Can we prioritize this? I am working on EWS for API tests, and this flaky failure is creating problem (e.g.: https://ews-build.webkit-uat.org/#/builders/20/builds/2256).
Wenson Hsieh
Comment 5 2019-03-25 14:41:42 PDT
I can repro this somewhat reliably by running the test for a dozen or so iterations in a loop.
Aakash Jain
Comment 6 2019-04-04 04:41:36 PDT
Happening very frequently, Recent occurrences: https://ews-build.webkit-uat.org/#/builders/20/builds/2630 https://ews-build.webkit-uat.org/#/builders/20/builds/2624 https://ews-build.webkit-uat.org/#/builders/20/builds/2623 https://ews-build.webkit-uat.org/#/builders/20/builds/2616 https://ews-build.webkit-uat.org/#/builders/20/builds/2614 https://ews-build.webkit-uat.org/#/builders/20/builds/2612 https://ews-build.webkit-uat.org/#/builders/20/builds/2604 https://ews-build.webkit-uat.org/#/builders/20/builds/2599 https://ews-build.webkit-uat.org/#/builders/20/builds/2594 https://ews-build.webkit-uat.org/#/builders/20/builds/2593 https://ews-build.webkit-uat.org/#/builders/20/builds/2591 https://ews-build.webkit-uat.org/#/builders/20/builds/2588 https://ews-build.webkit-uat.org/#/builders/20/builds/2584 https://ews-build.webkit-uat.org/#/builders/20/builds/2564 https://ews-build.webkit-uat.org/#/builders/20/builds/2563 https://ews-build.webkit-uat.org/#/builders/20/builds/2554 https://ews-build.webkit-uat.org/#/builders/20/builds/2552 https://ews-build.webkit-uat.org/#/builders/20/builds/2551 https://ews-build.webkit-uat.org/#/builders/20/builds/2544 https://ews-build.webkit-uat.org/#/builders/20/builds/2535 https://ews-build.webkit-uat.org/#/builders/20/builds/2530 https://ews-build.webkit-uat.org/#/builders/20/builds/2521 https://ews-build.webkit-uat.org/#/builders/20/builds/2506 https://ews-build.webkit-uat.org/#/builders/20/builds/2497 https://ews-build.webkit-uat.org/#/builders/20/builds/2496 https://ews-build.webkit-uat.org/#/builders/20/builds/2492 https://ews-build.webkit-uat.org/#/builders/20/builds/2490 https://ews-build.webkit-uat.org/#/builders/20/builds/2480 https://ews-build.webkit-uat.org/#/builders/20/builds/2478 https://ews-build.webkit-uat.org/#/builders/20/builds/2473 https://ews-build.webkit-uat.org/#/builders/20/builds/2450 https://ews-build.webkit-uat.org/#/builders/20/builds/2446 https://ews-build.webkit-uat.org/#/builders/20/builds/2429 https://ews-build.webkit-uat.org/#/builders/20/builds/2426 https://ews-build.webkit-uat.org/#/builders/20/builds/2415 https://ews-build.webkit-uat.org/#/builders/20/builds/2413 https://ews-build.webkit-uat.org/#/builders/20/builds/2407 https://ews-build.webkit-uat.org/#/builders/20/builds/2403 https://ews-build.webkit-uat.org/#/builders/20/builds/2389 https://ews-build.webkit-uat.org/#/builders/20/builds/2385 https://ews-build.webkit-uat.org/#/builders/20/builds/2384 https://ews-build.webkit-uat.org/#/builders/20/builds/2376 https://ews-build.webkit-uat.org/#/builders/20/builds/2374 https://ews-build.webkit-uat.org/#/builders/20/builds/2366 https://ews-build.webkit-uat.org/#/builders/20/builds/2359 https://ews-build.webkit-uat.org/#/builders/20/builds/2335 https://ews-build.webkit-uat.org/#/builders/20/builds/2334 In following builds, this test was flaky in a manner that EWS thought that the patch being tested introduced this test failure (failed twice with patch, passed without patch): https://ews-build.webkit-uat.org/#/builders/20/builds/2498 https://ews-build.webkit-uat.org/#/builders/20/builds/2378 https://ews-build.webkit-uat.org/#/builders/20/builds/2337 https://ews-build.webkit-uat.org/#/builders/20/builds/2329 https://ews-build.webkit-uat.org/#/builders/20/builds/2311 https://ews-build.webkit-uat.org/#/builders/20/builds/2308 https://ews-build.webkit-uat.org/#/builders/20/builds/2256
Radar WebKit Bug Importer
Comment 7 2019-05-06 13:11:21 PDT
<rdar://problem/50509880>
Aakash Jain
Comment 8 2019-05-06 13:33:26 PDT
This test was disabled in https://bugs.webkit.org/show_bug.cgi?id=196630. It would be nice to fix and re-enable it.
Wenson Hsieh
Comment 9 2019-05-17 20:46:57 PDT
I reenabled this in https://trac.webkit.org/changeset/245338/webkit.
Note You need to log in before you can comment on or make changes to this bug.