LayoutTest performance-api/performance-now-api.html is a flaky failure https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r211972%20(3437)/results.html https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=performance-api%2Fperformance-now-api.html --- /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/performance-api/performance-now-api-expected.txt +++ /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/performance-api/performance-now-api-actual.txt @@ -12,7 +12,7 @@ PASS performance instanceof Performance is true PASS typeof performance.now() === "number" is true PASS performance.now() <= performance.now() is true -PASS delta >= 95 && delta <= 120 is true +FAIL delta >= 95 && delta <= 120 should be true. Was false. Starting worker: resources/now-api.js [Worker] Performance
This is a new test that was just added, and of course layout tests can't measure time with anywhere near this precision.
(In reply to comment #1) > This is a new test that was just added, and of course layout tests can't > measure time with anywhere near this precision. Is there any way to write a time based test safely? Marking a test like this as flakey means there is no test coverage for it at all.
I don't know how to make an end to end layout test that expects something to happen in a given time frame, kernel scheduling can easily make a thread stop for several seconds. Injecting test specific code can of course work, but then it's always a question of whether the behavior is likely to be broken in a way that bypasses the test. I think that the best question to ask with regards to a new regression test is whether it tests for something that's likely to break. The easiest way to answer it is by adding tests together with fixes - that way you know that it's fragile code, and you can confirm whether your test catches that specific failure mode.