Bug 217118 – REGRESSION(r267402): [ macOS iOS Release ] imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html is a flaky failure

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Hector Lopez]

imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html

Test is a flaky failure according to history on macOS Release and iOS Release. First occurrence of failure at https://trac.webkit.org/changeset/267402/webkit

History:

https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fuser-timing%2Fmeasure-l3.any.worker.html&platform=ios&platform=mac&limit=50000

Diff:

--- /Volumes/Data/slave/catalina-release-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker-expected.txt
+++ /Volumes/Data/slave/catalina-release-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker-actual.txt
@@ -1,5 +1,5 @@
 
 PASS When the end mark is given and the start is unprovided, the end time of the measure entry should be the end mark's time, the start time should be 0.
-PASS When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now.
+FAIL When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now. assert_greater_than_equal: expected a number greater than or equal to 13.000000000000002 but got 13
 PASS When start and end mark are both given, the start time and end time of the measure entry should be the the marks' time, repectively

Was able to reproduce this with r267769:

run-webkit-tests imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html  --iterations 1000 --exit-after-n-failures 3 -1 -f

running 1 test

Running 6 DumpRenderTrees in parallel.

[35/1000] imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html failed unexpectedly (text diff)
[46/1000] imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html failed unexpectedly (text diff)
[47/1000] imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html failed unexpectedly (text diff)
Exiting early after 3 failures. 42 tests run.


39 tests ran as expected, 3 didn't (958 didn't run):


Regressions: Unexpected text-only failures (1)
  imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html [ Failure ]

Diff:
--- /Volumes/Data/Builds/build-267769/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker-expected.txt
+++ /Volumes/Data/Builds/build-267769/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker-actual.txt
@@ -1,5 +1,5 @@
 
 PASS When the end mark is given and the start is unprovided, the end time of the measure entry should be the end mark's time, the start time should be 0.
-PASS When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now.
+FAIL When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now. assert_greater_than_equal: expected a number greater than or equal to 26.000000000000004 but got 26
 PASS When start and end mark are both given, the start time and end time of the measure entry should be the the marks' time, repectively

[Radar WebKit Bug Importer]

<rdar://problem/69774340>

[Hector Lopez]

Test expectation while investigated:

https://trac.webkit.org/changeset/267778/webkit

[Alex Christensen]

Created attachment 417279 [details]
Patch

[EWS Watchlist]

This patch modifies the imported WPT tests. Please ensure that any changes on the tests (not coming from a WPT import) are exported to WPT. Please see https://trac.webkit.org/wiki/WPTExportProcess

[Tim Horton]

Comment on attachment 417279 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=417279&action=review

> LayoutTests/imported/w3c/web-platform-tests/user-timing/measure-l3.any.js:25
> +  assert_greater_than_equal(Number(endTime(measureEntry).toFixed(2)), endMin);
> +  assert_greater_than_equal(endMax, Number(endTime(measureEntry).toFixed(2)));

Would it be better to do this by subtraction and picking a maximum delta instead?

[Alex Christensen]

Comment on attachment 417279 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=417279&action=review

>> LayoutTests/imported/w3c/web-platform-tests/user-timing/measure-l3.any.js:25
>> +  assert_greater_than_equal(endMax, Number(endTime(measureEntry).toFixed(2)));
> 
> Would it be better to do this by subtraction and picking a maximum delta instead?

Probably, but this is consistent with what is already done elsewhere in this test.

[EWS]

Committed r271320: <https://trac.webkit.org/changeset/271320>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 417279 [details].