Bug 199280 - Layout Test fast/parser/parser-yield-timing.html is flaky
Summary: Layout Test fast/parser/parser-yield-timing.html is flaky
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-06-27 11:45 PDT by Russell Epstein
Modified: 2019-06-27 13:58 PDT (History)
7 users (show)

See Also:


Attachments
Update TestExpectations (1.25 KB, patch)
2019-06-27 11:48 PDT, Russell Epstein
no flags Details | Formatted Diff | Diff
Patch (3.24 KB, patch)
2019-06-27 13:23 PDT, Russell Epstein
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Russell Epstein 2019-06-27 11:45:06 PDT
The following layout test is flaky on Mojave Debug WK1

fast/parser/parser-yield-timing.html

Steps to reproduce:

run-webkit-tests --iter 1000 -f -1 fast/parser/parser-yield-timing.html

Regression range appears to be between r244212 and r244236.

Confirmed test passing in r244211, failing r244239. Lack of testable builds is preventing further reduction of the regression range.

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=fast%2Fparser%2Fparser-yield-timing.html

Diff:

--- /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/parser/parser-yield-timing-expected.txt
+++ /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/parser/parser-yield-timing-actual.txt
@@ -4,8 +4,8 @@
 
 
 PASS new Date().getTime() - startTime is within 300 of 1000
-PASS new Date().getTime() - startTime is within 300 of 2000
-PASS new Date().getTime() - startTime is within 300 of 3000
+FAIL new Date().getTime() - startTime should be within 300 of 2000. Was 2307.
+FAIL new Date().getTime() - startTime should be within 300 of 3000. Was 3331.
 PASS successfullyParsed is true
 
 TEST COMPLETE
Comment 1 Russell Epstein 2019-06-27 11:48:56 PDT
Created attachment 373040 [details]
Update TestExpectations
Comment 2 Truitt Savell 2019-06-27 13:00:43 PDT
Comment on attachment 373040 [details]
Update TestExpectations

Clearing flags on attachment: 373040

Committed r246898: <https://trac.webkit.org/changeset/246898>
Comment 3 Radar WebKit Bug Importer 2019-06-27 13:00:53 PDT
<rdar://problem/52278766>
Comment 4 Russell Epstein 2019-06-27 13:23:19 PDT
Created attachment 373048 [details]
Patch
Comment 5 WebKit Commit Bot 2019-06-27 13:57:16 PDT
Comment on attachment 373048 [details]
Patch

Clearing flags on attachment: 373048

Committed r246904: <https://trac.webkit.org/changeset/246904>
Comment 6 WebKit Commit Bot 2019-06-27 13:57:18 PDT
All reviewed patches have been landed.  Closing bug.
Comment 7 Russell Epstein 2019-06-27 13:58:43 PDT
Had an offline discussion with thorton. Discussed that it may be better to increase the test tolerance rather than marking the test as flaky.

Increased test tolerance from ±300ms to ±400ms to reduce noise.
Updated fast/parser/parser-yield-timing-expected.txt to reflect this change.
Removed expected failure from TestExpectations.

https://trac.webkit.org/changeset/246904/webkit

(In reply to WebKit Commit Bot from comment #5)
> Comment on attachment 373048 [details]
> Patch
> 
> Clearing flags on attachment: 373048
> 
> Committed r246904: <https://trac.webkit.org/changeset/246904>