Bug 168005 - LayoutTest performance-api/performance-now-time-origin-in-worker.html is a flaky failure
Summary: LayoutTest performance-api/performance-now-time-origin-in-worker.html is a fl...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-02-08 10:53 PST by Ryan Haddad
Modified: 2017-02-09 13:21 PST (History)
3 users (show)

See Also:


Attachments
[PATCH] Proposed Fix (2.69 KB, patch)
2017-02-08 11:25 PST, Joseph Pecoraro
ap: review-
ap: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryan Haddad 2017-02-08 10:53:32 PST
LayoutTest performance-api/performance-now-time-origin-in-worker.html is a flaky failure

https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK1%20(Tests)/r211875%20(11401)/results.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=performance-api%2Fperformance-now-time-origin-in-worker.html

--- /Volumes/Data/slave/elcapitan-debug-tests-wk1/build/layout-test-results/performance-api/performance-now-time-origin-in-worker-expected.txt
+++ /Volumes/Data/slave/elcapitan-debug-tests-wk1/build/layout-test-results/performance-api/performance-now-time-origin-in-worker-actual.txt
@@ -4,7 +4,7 @@
 
 
 Starting worker: resources/time-origin-in-worker.js
-PASS [Worker] performance.now() < 200 is true
+FAIL [Worker] performance.now() < 200 should be true. Was false.
 PASS successfullyParsed is true
 
 TEST COMPLETE
Comment 1 Joseph Pecoraro 2017-02-08 11:12:37 PST
This implies it takes more than 200ms to start a worker in debug builds. I suppose we can up this time. It may also be good to print out the time in failures so that we can see what the time was, if it blew the budget.
Comment 2 Joseph Pecoraro 2017-02-08 11:25:04 PST
Created attachment 300928 [details]
[PATCH] Proposed Fix
Comment 3 Alexey Proskuryakov 2017-02-08 17:46:59 PST
Comment on attachment 300928 [details]
[PATCH] Proposed Fix

View in context: https://bugs.webkit.org/attachment.cgi?id=300928&action=review

> LayoutTests/ChangeLog:8
> +        Increase the budget from 200ms to 250ms and if it fails, log the time

Any timeout can take up to a couple seconds in a test, so 250 ms can never be right.

What exactly is this test checking for, can it work without precise time measurement?
Comment 4 Joseph Pecoraro 2017-02-08 22:12:08 PST
(In reply to comment #3)
> Comment on attachment 300928 [details]
> [PATCH] Proposed Fix
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=300928&action=review
> 
> > LayoutTests/ChangeLog:8
> > +        Increase the budget from 200ms to 250ms and if it fails, log the time
> 
> Any timeout can take up to a couple seconds in a test, so 250 ms can never
> be right.
> 
> What exactly is this test checking for, can it work without precise time
> measurement?

This is checking that the performance.now() values in a Worker are relative to their creation time and not relative to the Document's creation time.

Ultimately this is checking that the performance.now in the Worker and Page are different. Currently to do this, the test waits 200ms on the Document, spawns a Worker, and checks that performance.now in that Worker is less then 200ms. The patch above bumped this to 250ms and would log the failure time if the test fails to get an idea of how long it took.

PerformanceTiming Level 3 introduces performance.timeOrigin which would be the easier way to verify this. Then we could just send the timeOrigin back to the page and verify the difference. Currently we don't have that luxury, so the best I can come up is ensuring the different by waiting some N milliseconds on the Page before spawning the Worker to verify we can get a time below N in the Worker.
Comment 5 Alexey Proskuryakov 2017-02-08 22:18:47 PST
> Ultimately this is checking that the performance.now in the Worker and Page are different. 

I can't think of a way to test this reliably, we should probably just mark the test as flaky until performance.timeOrigin is available.
Comment 6 Alexey Proskuryakov 2017-02-09 13:05:23 PST
Comment on attachment 300928 [details]
[PATCH] Proposed Fix

Sorry, forgot to mark r-. Changing to 250ms can't help enough.
Comment 7 Ryan Haddad 2017-02-09 13:21:07 PST
Marked test as flaky in https://trac.webkit.org/changeset/211982