209989 2020-04-03 15:03:27 -0700 [ macOS debug ] REGRESSION (r259463): http/tests/media/clearkey/collect-webkit-media-session.html is failing 2020-05-07 13:04:54 -0700 1 1 1 Unclassified WebKit Media WebKit Nightly Build Unspecified Unspecified RESOLVED CONFIGURATION CHANGED InRadar P2 Normal --- 1 jacob_uphoff webkit-unassigned aakash_jain ap Lawrence.j webkit-bot-watchers-bugzilla webkit-bug-importer ysuzuki oldest_to_newest 1637537 0 jacob_uphoff 2020-04-03 15:03:27 -0700 http/tests/media/clearkey/collect-webkit-media-session.html This test is failing on Mojave debug since around commit r259463 History: https://results.webkit.org/?suite=layout-tests&test=http%2Ftests%2Fmedia%2Fclearkey%2Fcollect-webkit-media-session.html&style=debug Diff: --- /Volumes/Data/slave/mojave-debug-tests-wk2/build/layout-test-results/http/tests/media/clearkey/collect-webkit-media-session-expected.txt +++ /Volumes/Data/slave/mojave-debug-tests-wk2/build/layout-test-results/http/tests/media/clearkey/collect-webkit-media-session-actual.txt @@ -1,4 +1,4 @@ -PASS Ensure that the frame's document get collected after being stopped while doing some webkit media session calls +FAIL Ensure that the frame's document get collected after being stopped while doing some webkit media session calls promise_test: Unhandled rejection with value: "Test failed" 1637539 1 webkit-bug-importer 2020-04-03 15:04:36 -0700 <rdar://problem/61277230> 1637540 2 webkit-bug-importer 2020-04-03 15:05:05 -0700 <rdar://problem/61277243> 1637548 3 jacob_uphoff 2020-04-03 15:41:25 -0700 Looks like http/tests/media/media-stream/collect-media-devices.https.html is also failing from the same commit. 1637570 4 Lawrence.j 2020-04-03 17:02:33 -0700 I can reproduce this issue on r259463, but I cannot reproduce this on r259460. This is the command that I used: run-webkit-tests --force -f --iterations 90 --exit-after-n-failures 3 -g --debug http/tests/media/clearkey/collect-webkit-media-session.html r259460 [4/90] http/tests/media/clearkey/collect-webkit-media-session.html failed unexpectedly (text diff) [5/90] http/tests/media/clearkey/collect-webkit-media-session.html failed unexpectedly (text diff) [6/90] http/tests/media/clearkey/collect-webkit-media-session.html failed unexpectedly (text diff) Exiting early after 3 failures. 3 tests run. 0 tests ran as expected, 3 didn't (87 didn't run): r259460: All 90 tests ran as expected. 1637670 5 aakash_jain 2020-04-04 05:41:14 -0700 This is slowing down mac-wk1-debug ews queue, and also causing false positives. e.g.: https://ews-build.webkit.org/#/builders/32/builds/6715 https://ews-build.webkit.org/#/builders/32/builds/6714 https://ews-build.webkit.org/#/builders/32/builds/6708 We should consider updating the test expectations if we can't get the root-caused fixed soon. 1637727 6 ap 2020-04-04 12:51:12 -0700 > I can reproduce this issue on r259463, but I cannot reproduce this on r259460. Since the other changes in between are for IndexedDB, seems almost certain that it's r259463. 1638096 7 ysuzuki 2020-04-06 08:23:13 -0700 This is testing "something gets collected by GC". But the nature of conservative GC makes is super flaky. Any memory layout / register allocation / JIT change can easily break this assumption. The test is not always failing. This implies that we are observing that document is collected in most cases. But sometimes, it is not. This sounds super likely that conservative GC finds it and keeps it held, and this behavior change is just introduced because the change in JSC causes memory layout / register allocation / JIT changes, which allows conservative GC to find it sometimes. The most possible fix would be just adjusting the test to make it more stable. I'll look it later. 1638102 8 jacob_uphoff 2020-04-06 08:31:00 -0700 Set expectations here: https://trac.webkit.org/changeset/259570/webkit 1638302 9 Lawrence.j 2020-04-06 14:07:19 -0700 *** Bug 210073 has been marked as a duplicate of this bug. *** 1650314 10 ryanhaddad 2020-05-07 13:04:54 -0700 Removed flaky expectations in r261329 since these test are consistently passing now.