The following layout test recently started flaking on Mac: streams/reference-implementation/readable-stream.html See: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=streams%2Freference-implementation%2Freadable-stream.html
Youenn, this looks like a feature you've been working on. Any ideas why this might have recently started flaking?
Tim, thanks for spotting these flakiness. Calvaris, it seems the changes brought on the tests when inverting the order of the promise resolution have some bad effects. Will you have some time to check these?
(In reply to comment #2) > Tim, thanks for spotting these flakiness. > Calvaris, it seems the changes brought on the tests when inverting the order > of the promise resolution have some bad effects. Will you have some time to > check these? Sure, I'm on it now.
Created attachment 259755 [details] Patch It is true that timeout hitting earlier happens a bit more often after the promise changes but it was already happening before aparently. Let's try to increase the timeout and see if the problem disappears.
(In reply to comment #4) > Created attachment 259755 [details] > Patch > > It is true that timeout hitting earlier happens a bit more often after the > promise changes but it was already happening before aparently. Let's try to > increase the timeout and see if the problem disappears. Then, maybe we could rewrite the test cases a bit to reduce the flakiness probability. In particular we could call setTimeout at the latest time. For instance, in test 6, we could move the call to setTimeout within startPromise.then() callback.
Created attachment 259791 [details] Patch Set timeouts later as suggested by Youenn. I thought of that too but I was reluctant because we'd lose the structure of the original reference test, but I decided to add a comment to help with that.
This flakiness seemed important. Can we get this reviewed?
Comment on attachment 259791 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=259791&action=review Sad, but looks like there is no other way to test for this. > LayoutTests/streams/reference-implementation/readable-stream.html:273 > + // In the original test this timeout is outside the promise fulfillment callback, but it makes test flakier so > + // we better move it inside here. This comment is not helpful without a reference to original. Are we modifying a W3C test here? I would just remove the comment (and other copies of it) unless there is a really important reason for it - otherwise anyone interested can just blame the source to see history.
If feasible, one option might be to backport these changes to the reference tests.
Created attachment 260141 [details] Patch for landing
(In reply to comment #9) > If feasible, one option might be to backport these changes to the reference > tests. I'll propose the change, let's see if Domenic accepts it.
Comment on attachment 260141 [details] Patch for landing Clearing flags on attachment: 260141 Committed r189093: <http://trac.webkit.org/changeset/189093>
All reviewed patches have been landed. Closing bug.
(In reply to comment #11) > (In reply to comment #9) > > If feasible, one option might be to backport these changes to the reference > > tests. > > I'll propose the change, let's see if Domenic accepts it. Done: https://github.com/whatwg/streams/pull/392