Bug 171580 – REGRESSION: LayoutTest streams/reference-implementation/readable-stream-templated.html is a flaky failure

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Ryan Haddad]

LayoutTest streams/reference-implementation/readable-stream-templated.html is a flaky failure

https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK1%20(Tests)/r216097%20(935)/results.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=streams%2Freference-implementation%2Freadable-stream-templated.html

--- /Volumes/Data/slave/elcapitan-debug-tests-wk1/build/layout-test-results/streams/reference-implementation/readable-stream-templated-expected.txt
+++ /Volumes/Data/slave/elcapitan-debug-tests-wk1/build/layout-test-results/streams/reference-implementation/readable-stream-templated-actual.txt
@@ -15,7 +15,7 @@
 CONSOLE MESSAGE: Unhandled Promise Rejection: TypeError: cancel() called on a reader owned by no readable stream
 CONSOLE MESSAGE: Unhandled Promise Rejection: TypeError: cancel() called on a reader owned by no readable stream
 
-Harness Error (FAIL), message = cancel() called on a reader owned by no readable stream
+Harness Error (FAIL), message = !!!
 
 PASS Running templatedRSClosed with ReadableStream (closed via call in start) 
 PASS piping to a WritableStream in the writable state should close the writable stream

[Ryan Haddad]

According to the flakiness dashboard, the earliest failure on macOS is 4/28/2017 4:20:02 PM @r215945, but the changes around that time do not appear to be related.

[Alexey Proskuryakov]

The earliest failure I see on internal bots was with r215927, but this failure is not frequent enough to pinpoint the culprit.

Happens on both Mac and iOS.

Looking at the test, I strongly suspect that it became flaky after r215916 (unhandledrejection). Joe, could you please take a look?

[Radar WebKit Bug Importer]

<rdar://problem/32009647>

[Joseph Pecoraro]

I'm unable to reproduce this.

  $ run-webkit-tests --debug --iterations=40 --force streams/reference-implementation
  $ run-webkit-tests --debug --iterations=40 --force streams/reference-implementation -1

So I think the best thing to do is just silence the messages in this test.

[Joseph Pecoraro]

Hmm, the File message is "!!!" instead of the unhandled promise rejection message. Let me try to figure out what could cause that.

[Joseph Pecoraro]

"!!!" is an Error that can be thrown in the tests and is included in promise rejections.

[Joseph Pecoraro]

Created attachment 309194 [details]
[PATCH] Proposed Fix

[Alexey Proskuryakov]

Comment on attachment 309194 [details]
[PATCH] Proposed Fix

rs=me

A couple more things to try when reproducing:

- overload CPUs with many processes (--fully-parallel --child-processes=50 --repeat 1000)
- enable GuardMalloc to change performance characteristics

[Joseph Pecoraro]

<https://trac.webkit.org/changeset/216267/webkit>

[youenn fablet]

The best fix would have been to move from async_test to promise_test.
Landed fix seems good enough to me.

[Joseph Pecoraro]

(In reply to youenn fablet from comment #10)
> The best fix would have been to move from async_test to promise_test.
> Landed fix seems good enough to me.

Yeah, I don't want to modify these tests too much. They are roughly imported, and probably out of date from upstream. I don't know if we should treat these as imported tests and avoid modifications or treat them as our own tests and feel free to modify at will.

[youenn fablet]

Piping is outdated since it ties ReadableStream (up-to-date) to WritableStream (outdated). These tests should probably be removed when improving WritableStream and replaced by WPT existing WritableStream tests