Bug 205299 – REGRESSION: animations/leak-document-with-css-animation.html is very flaky

• Component: Animations
• Status: RESOLVED FIXED

[Alexey Proskuryakov]

animations/leak-document-with-css-animation.html only failed once between 2019-10-10 and 2019-11-14, and now it's failing multiple times each day.

https://results.webkit.org/?suite=layout-tests&test=animations%2Fleak-document-with-css-animation.html

@@ -5,7 +5,7 @@
 
 The iframe has finished loading.
 The iframe has been destroyed.
-PASS internals.numberOfLiveDocuments() is numberOfLiveDocumentsAfterIframeLoaded - 1
+FAIL internals.numberOfLiveDocuments() should be 7. Was 6.
 
 PASS successfullyParsed is true

[Radar WebKit Bug Importer]

<rdar://problem/57984602>

[Alexey Proskuryakov]

Marked as flaky in http://trac.webkit.org/r253575

[Antoine Quint]

I made two changes in Web Animations on November 14: r252455 and r252461.

[Antoine Quint]

Of note, there was a failure with a run with r252460: https://build.webkit.org/results/Apple%20High%20Sierra%20Release%20WK1%20(Tests)/r252460%20(17046)/animations/leak-document-with-css-animation-pretty-diff.html

[Antoine Quint]

Running many iterations of the test itself doesn't reveal any issues. Need to try running tests in the same order of a worker that showed a failure next.

[Antoine Quint]

No luck with the test list from https://build.webkit.org/builders/Apple-Catalina-Release-WK2-Tests/builds/1595.

[Antoine Quint]

Something that puzzles me is that when the test runs in isolation, it has 2 live documents after the iframe has loaded, and 1 after it has been destroyed.

However, when it fails on the bots it shows that the number of live documents is much higher. I suspect that internals.numberOfLiveDocuments() returns the number of live documents across runners.

At any rate, the figures reported shows a lower number of live documents still, which is good. So maybe we ought to rework this test to not check for a hard number, but only a lower number after the iframe has been destroyed.

[Antoine Quint]

Actually, I think we should call `gc()` before we take the reference snapshot of live documents, hopefully this will fix the flakiness by allowing any document pending destruction to be destroyed. If flakiness persists, I think we can then consider using a less-than test.

[Antoine Quint]

Created attachment 385949 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 385949 [details]
Patch

Clearing flags on attachment: 385949

Committed r253679: <https://trac.webkit.org/changeset/253679>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Antoine Quint]

Looking good so far, no failures since this change.

[Antoine Quint]

No failure in 6 days, I think the flakiness is addressed.