imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/execution-timing/085.html This issue began recently but is very flaky. the diffs are the same between iOS and Mac. Reproduce with: run-webkit-tests --iterations 2000 --exit-after-n-failures 1 --no-retry --no-build -f imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/execution-timing/085.html history: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fexecution-timing%2F085.html Diff: --- /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/execution-timing/085-expected.txt +++ /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/execution-timing/085-actual.txt @@ -1,4 +1,4 @@ FAILED (This TC requires JavaScript enabled) -PASS scheduler: async script and slow-loading defer script +FAIL scheduler: async script and slow-loading defer script assert_array_equals: property 0, expected "external script #2" but got "external script #1"
<rdar://problem/60379091>
This test was last updated in https://trac.webkit.org/changeset/249886/webkit
This test seems to reproduce no matter what revision I run it on.
History shows this started some time after 258250 but no way currently to narrow that down
Marked test as failing while it is investigated: https://trac.webkit.org/changeset/258345/webkit
This quite certainly is a regression from r258268.
(In reply to Alexey Proskuryakov from comment #6) > This quite certainly is a regression from r258268. Flakiness is likely triggered by when the first paint actually happens since the script is delayed until first paint.
(In reply to Chris Dumez from comment #7) > (In reply to Alexey Proskuryakov from comment #6) > > This quite certainly is a regression from r258268. > > Flakiness is likely triggered by when the first paint actually happens since > the script is delayed until first paint. I think the test is inherently flaky. It has a slow loading defer script (1 second to load) and an async script. It expects the async script to always run first.
(In reply to Chris Dumez from comment #8) > (In reply to Chris Dumez from comment #7) > > (In reply to Alexey Proskuryakov from comment #6) > > > This quite certainly is a regression from r258268. > > > > Flakiness is likely triggered by when the first paint actually happens since > > the script is delayed until first paint. > > I think the test is inherently flaky. It has a slow loading defer script (1 > second to load) and an async script. It expects the async script to always > run first. To be clear, there is no expectation that async scripts run before defer scripts. async script run whenever they are done loading, which may happen before or after DOMContentLoaded. defer scripts run after DOMContentLoaded always. This test is using a 1 second delay to try and make sure that the async script finishes loading before DOMContentLoaded but this is bound to be flaky. This is definitely flakier too now that we defer async scripts further in some cases. I think marking this test as flaky is the right thing to do given that this is an upstream test.