The following layout test is failing on Mac WK2 + iOS Debug imported/w3c/web-platform-tests/web-animations/interfaces/DocumentTimeline/constructor.html Probable cause: Test was modified in r233667, is 100% failure before that revision and is a flaky failure since it was changed. Reproducible with: run-webkit-tests imported/w3c/web-platform-tests/web-animations/interfaces/DocumentTimeline/constructor.html --iterations 5000 -f --exit-after-n-failures=5 Flakiness Dashboard: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fweb-animations%2Finterfaces%2FDocumentTimeline%2Fconstructor.html Diff: --- /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/web-animations/interfaces/DocumentTimeline/constructor-expected.txt +++ /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/web-animations/interfaces/DocumentTimeline/constructor-actual.txt @@ -1,5 +1,5 @@ -PASS An origin time of zero is used when none is supplied +FAIL An origin time of zero is used when none is supplied assert_approx_equals: expected 30 +/- 0.001 but got 15 PASS A zero origin time produces a document timeline with a current time identical to the default document timeline PASS A positive origin time makes the document timeline's current time lag behind the default document timeline PASS A negative origin time makes the document timeline's current time run ahead of the default document timeline
<rdar://problem/48527231>
Adding additional repro steps: Test was flaky through its entire available history on dashboard and searching through Trac I could not find any recent changes. Started Bisection point at 228049 which was the oldest revision I could spade. Once I started getting 100% failures I switched test parameters to --exit-after-n-failures=25 to limit iterations run. For Debug failures it failed in r233667 and 233636 was the only version I could spade previous to that. Once I found the revision it was starting to fail on, Looking on Trac for r233667 it appears test expectations were modified. Release WK2: 242285 flaky failure 242279 flaky failure 242112 flaky failure 237300 flaky failure 235000 flaky failure 234133 flaky failure 233702 flaky failure 233676 flaky failure 233671 flaky failure 233668 flaky failure 233667 4997 tests ran as expected, 3 didn't failed unexpectedly (text diff) 233666 0 tests ran as expected, 25 didn't 233665 0 tests ran as expected, 25 didn't 233649 0 tests ran as expected, 25 didn't 233595 0 tests ran as expected, 25 didn't 233483 0 tests ran as expected, 25 didn't 233263 0 tests ran as expected, 25 didn't 231525 0 tests ran as expected, 25 didn't 228049 0 tests ran as expected, 25 didn't : Unexpected text-only failures (1) [ Failure ] Debug WK2: 233678 733 tests ran as expected, 5 didn't (4262 didn't run) failed unexpectedly (text diff) 233636 0 tests ran as expected, 25 didn't Unexpected text-only failures (1) [ Failure ] Since initial testing found that iOS Simulator Release WK2, and Mac Debug WK1 are now flaky failures. iOS Simulator: 242640 722 tests ran as expected, 5 didn't (4273 didn't run) Unexpected text-only failures (1) 233667 4997 tests ran as expected, 3 didn't failed unexpectedly (text diff) 233666 0 tests ran as expected, 25 didn't Unexpected text-only failures (1) [ Failure ] Mac Debug WK1: 233678 395 tests ran as expected, 5 didn't (4600 didn't run): failed unexpectedly (text diff) 233636 0 tests ran as expected, 25 didn't (4975 didn't run): failed unexpectedly (text diff)
Correction, This is not a regression to this specific version. The code just changed in r233667. Test was likely a flaky failure before that.
Created attachment 366936 [details] Patch
Committed r244032: <https://trac.webkit.org/changeset/244032>