Bug 217118 - REGRESSION(r267402): [ macOS iOS Release ] imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html is a flaky failure
Summary: REGRESSION(r267402): [ macOS iOS Release ] imported/w3c/web-platform-tests/us...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Alex Christensen
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-09-29 19:40 PDT by Hector Lopez
Modified: 2021-01-08 13:50 PST (History)
8 users (show)

See Also:


Attachments
Patch (4.95 KB, patch)
2021-01-08 11:00 PST, Alex Christensen
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Hector Lopez 2020-09-29 19:40:02 PDT
imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html

Test is a flaky failure according to history on macOS Release and iOS Release. First occurrence of failure at https://trac.webkit.org/changeset/267402/webkit

History:

https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fuser-timing%2Fmeasure-l3.any.worker.html&platform=ios&platform=mac&limit=50000

Diff:

--- /Volumes/Data/slave/catalina-release-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker-expected.txt
+++ /Volumes/Data/slave/catalina-release-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker-actual.txt
@@ -1,5 +1,5 @@
 
 PASS When the end mark is given and the start is unprovided, the end time of the measure entry should be the end mark's time, the start time should be 0.
-PASS When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now.
+FAIL When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now. assert_greater_than_equal: expected a number greater than or equal to 13.000000000000002 but got 13
 PASS When start and end mark are both given, the start time and end time of the measure entry should be the the marks' time, repectively

Was able to reproduce this with r267769:

run-webkit-tests imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html  --iterations 1000 --exit-after-n-failures 3 -1 -f

running 1 test

Running 6 DumpRenderTrees in parallel.

[35/1000] imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html failed unexpectedly (text diff)
[46/1000] imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html failed unexpectedly (text diff)
[47/1000] imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html failed unexpectedly (text diff)
Exiting early after 3 failures. 42 tests run.


39 tests ran as expected, 3 didn't (958 didn't run):


Regressions: Unexpected text-only failures (1)
  imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker.html [ Failure ]

Diff:
--- /Volumes/Data/Builds/build-267769/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker-expected.txt
+++ /Volumes/Data/Builds/build-267769/layout-test-results/imported/w3c/web-platform-tests/user-timing/measure-l3.any.worker-actual.txt
@@ -1,5 +1,5 @@
 
 PASS When the end mark is given and the start is unprovided, the end time of the measure entry should be the end mark's time, the start time should be 0.
-PASS When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now.
+FAIL When the start mark is given and the end is unprovided, the start time of the measure entry should be the start mark's time, the end should be now. assert_greater_than_equal: expected a number greater than or equal to 26.000000000000004 but got 26
 PASS When start and end mark are both given, the start time and end time of the measure entry should be the the marks' time, repectively
Comment 1 Radar WebKit Bug Importer 2020-09-29 19:40:23 PDT
<rdar://problem/69774340>
Comment 2 Hector Lopez 2020-09-29 19:45:59 PDT
Test expectation while investigated:

https://trac.webkit.org/changeset/267778/webkit
Comment 3 Alex Christensen 2021-01-08 11:00:40 PST
Created attachment 417279 [details]
Patch
Comment 4 EWS Watchlist 2021-01-08 11:01:40 PST
This patch modifies the imported WPT tests. Please ensure that any changes on the tests (not coming from a WPT import) are exported to WPT. Please see https://trac.webkit.org/wiki/WPTExportProcess
Comment 5 Tim Horton 2021-01-08 11:56:35 PST
Comment on attachment 417279 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=417279&action=review

> LayoutTests/imported/w3c/web-platform-tests/user-timing/measure-l3.any.js:25
> +  assert_greater_than_equal(Number(endTime(measureEntry).toFixed(2)), endMin);
> +  assert_greater_than_equal(endMax, Number(endTime(measureEntry).toFixed(2)));

Would it be better to do this by subtraction and picking a maximum delta instead?
Comment 6 Alex Christensen 2021-01-08 11:57:52 PST
Comment on attachment 417279 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=417279&action=review

>> LayoutTests/imported/w3c/web-platform-tests/user-timing/measure-l3.any.js:25
>> +  assert_greater_than_equal(endMax, Number(endTime(measureEntry).toFixed(2)));
> 
> Would it be better to do this by subtraction and picking a maximum delta instead?

Probably, but this is consistent with what is already done elsewhere in this test.
Comment 7 EWS 2021-01-08 13:50:00 PST
Committed r271320: <https://trac.webkit.org/changeset/271320>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 417279 [details].