Bug 209989 - [ macOS debug ] REGRESSION (r259463): http/tests/media/clearkey/collect-webkit-media-session.html is failing
Summary: [ macOS debug ] REGRESSION (r259463): http/tests/media/clearkey/collect-webki...
Status: RESOLVED CONFIGURATION CHANGED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
: 210073 (view as bug list)
Depends on:
Blocks:
 
Reported: 2020-04-03 15:03 PDT by Jacob Uphoff
Modified: 2020-05-07 13:04 PDT (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jacob Uphoff 2020-04-03 15:03:27 PDT
http/tests/media/clearkey/collect-webkit-media-session.html

This test is failing on Mojave debug since around commit r259463

History:

https://results.webkit.org/?suite=layout-tests&test=http%2Ftests%2Fmedia%2Fclearkey%2Fcollect-webkit-media-session.html&style=debug

Diff:

--- /Volumes/Data/slave/mojave-debug-tests-wk2/build/layout-test-results/http/tests/media/clearkey/collect-webkit-media-session-expected.txt
+++ /Volumes/Data/slave/mojave-debug-tests-wk2/build/layout-test-results/http/tests/media/clearkey/collect-webkit-media-session-actual.txt
@@ -1,4 +1,4 @@
 
 
-PASS Ensure that the frame's document get collected after being stopped while doing some webkit media session calls 
+FAIL Ensure that the frame's document get collected after being stopped while doing some webkit media session calls promise_test: Unhandled rejection with value: "Test failed"
Comment 1 Radar WebKit Bug Importer 2020-04-03 15:04:36 PDT
<rdar://problem/61277230>
Comment 2 Radar WebKit Bug Importer 2020-04-03 15:05:05 PDT
<rdar://problem/61277243>
Comment 3 Jacob Uphoff 2020-04-03 15:41:25 PDT
Looks like http/tests/media/media-stream/collect-media-devices.https.html is also failing from the same commit.
Comment 4 Jason Lawrence 2020-04-03 17:02:33 PDT
I can reproduce this issue on r259463, but I cannot reproduce this on r259460.
This is the command that I used:
run-webkit-tests --force -f --iterations  90 --exit-after-n-failures 3 -g --debug http/tests/media/clearkey/collect-webkit-media-session.html

r259460
[4/90] http/tests/media/clearkey/collect-webkit-media-session.html failed unexpectedly (text diff)
[5/90] http/tests/media/clearkey/collect-webkit-media-session.html failed unexpectedly (text diff)
[6/90] http/tests/media/clearkey/collect-webkit-media-session.html failed unexpectedly (text diff)
Exiting early after 3 failures. 3 tests run.


0 tests ran as expected, 3 didn't (87 didn't run):

r259460:
All 90 tests ran as expected.
Comment 5 Aakash Jain 2020-04-04 05:41:14 PDT
This is slowing down mac-wk1-debug ews queue, and also causing false positives.

e.g.:
https://ews-build.webkit.org/#/builders/32/builds/6715
https://ews-build.webkit.org/#/builders/32/builds/6714
https://ews-build.webkit.org/#/builders/32/builds/6708

We should consider updating the test expectations if we can't get the root-caused fixed soon.
Comment 6 Alexey Proskuryakov 2020-04-04 12:51:12 PDT
> I can reproduce this issue on r259463, but I cannot reproduce this on r259460.

Since the other changes in between are for IndexedDB, seems almost certain that it's r259463.
Comment 7 Yusuke Suzuki 2020-04-06 08:23:13 PDT
This is testing "something gets collected by GC". But the nature of conservative GC makes is super flaky.
Any memory layout / register allocation / JIT change can easily break this assumption.

The test is not always failing. This implies that we are observing that document is collected in most cases. But sometimes, it is not.
This sounds super likely that conservative GC finds it and keeps it held, and this behavior change is just introduced because the change in JSC causes memory layout / register allocation / JIT changes, which allows conservative GC to find it sometimes.

The most possible fix would be just adjusting the test to make it more stable.
I'll look it later.
Comment 8 Jacob Uphoff 2020-04-06 08:31:00 PDT
Set expectations here: https://trac.webkit.org/changeset/259570/webkit
Comment 9 Jason Lawrence 2020-04-06 14:07:19 PDT
*** Bug 210073 has been marked as a duplicate of this bug. ***
Comment 10 Ryan Haddad 2020-05-07 13:04:54 PDT
Removed flaky expectations in r261329 since these test are consistently passing now.