Bug 103115 – http/tests/loading/remove-child-triggers-parser.html is flaky

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Jussi Kukkonen (jku)]

The following layout test is flaky on various platforms but especially so on WK2 EFL:

http/tests/loading/remove-child-triggers-parser.html

Testing locally, the first run seems to print correct output, except it's missing the last empty line of output. Running with --iterations shows all the subsequent runs passing.

[Ryan Haddad]

https://build.webkit.org/builders/Apple%20Sierra%20Debug%20WK1%20(Tests)/builds/1034

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Floading%2Fremove-child-triggers-parser.html

--- /Volumes/Data/slave/sierra-debug-tests-wk1/build/layout-test-results/http/tests/loading/remove-child-triggers-parser-expected.txt
+++ /Volumes/Data/slave/sierra-debug-tests-wk1/build/layout-test-results/http/tests/loading/remove-child-triggers-parser-actual.txt
@@ -3,4 +3,3 @@
 main frame - didFinishDocumentLoadForFrame
 main frame - didHandleOnloadEventsForFrame
 main frame - didFinishLoadForFrame
-

[Jonathan Bedard]

Some brief analysis of the failures of this test shows that it has the very odd feature of it's failures sharing a very long list of tests (a few hundred) which were executed before failures and not before successes.  I will run this analysis on a larger set of success/failures, my guess would be that this test isn't flakey, it's order dependent.

[Alexey Proskuryakov]

*** Bug 206628 has been marked as a duplicate of this bug. ***

[Alexey Proskuryakov]

<rdar://problem/58815922>

[Alexey Proskuryakov]

> setTimeout(function() { var child = document.documentElement; child.parentNode.removeChild(child); }, 36);

Certainly smells like a not so great test. Since this is a test for a security bug, it may be important enough to deflake. CC'ing Ryosuke, who reviewed it many years ago.

[Ryosuke Niwa]

Created attachment 388933 [details]
Deflake the test

[Alexey Proskuryakov]

Comment on attachment 388933 [details]
Deflake the test

View in context: https://bugs.webkit.org/attachment.cgi?id=388933&action=review

> LayoutTests/http/tests/loading/remove-child-triggers-parser.html:13
> +}, 36);

36? Can we just do this in onload instead?

[Ryosuke Niwa]

(In reply to Alexey Proskuryakov from comment #7)
> Comment on attachment 388933 [details]
> Deflake the test
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=388933&action=review
> 
> > LayoutTests/http/tests/loading/remove-child-triggers-parser.html:13
> > +}, 36);
> 
> 36? Can we just do this in onload instead?

Sure. I think we still want to wait more than 0s (I think the threshold is 5ms or 10ms). Will delay until load event & then schedule a timer.

[Ryosuke Niwa]

Committed r255223: <https://trac.webkit.org/changeset/255223>

[Alexey Proskuryakov]

I wouldn't expect needing a timer after the load event.

[Aakash Jain]

> Committed r255223: <https://trac.webkit.org/changeset/255223>
This change might have made this test fail consistently on windows, tracked in Bug 206992. Can you please have a look?

[Ryosuke Niwa]

(In reply to Aakash Jain from comment #11)
> > Committed r255223: <https://trac.webkit.org/changeset/255223>
> This change might have made this test fail consistently on windows, tracked
> in Bug 206992. Can you please have a look?

FWIW, I've fixed this, and it looks like the test is no long flaky & not falling on Windows now.