217931 2020-10-19 16:45:10 -0700 wpt/**/moving-between-documents/before-prepare-*/move-back-* tests are flaky 2020-12-08 13:58:07 -0800 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=217908 https://bugs.webkit.org/show_bug.cgi?id=214685 https://bugs.webkit.org/show_bug.cgi?id=214734 https://bugs.webkit.org/show_bug.cgi?id=215278 https://bugs.webkit.org/show_bug.cgi?id=215283 https://bugs.webkit.org/show_bug.cgi?id=215706 https://bugs.webkit.org/show_bug.cgi?id=215783 https://bugs.webkit.org/show_bug.cgi?id=218650 https://bugs.webkit.org/show_bug.cgi?id=218651 InRadar P2 Normal --- 219655 1 hector_i_lopez webkit-unassigned ap cdumez dpino gsnedders rackler rniwa ryanhaddad tsavell webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 1699518 0 hector_i_lopez 2020-10-19 16:45:10 -0700 imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-createHTMLDocument-success-empty-src-module.html Test is a flaky failure according to history on macOS Debug. First occurrence of failure is at r265019. History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-createHTMLDocument-success-empty-src-module.html&style=debug&platform=mac&limit=50000 Diff: --- /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-createHTMLDocument-success-empty-src-module-expected.txt +++ /Volumes/Data/slave/catalina-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-createHTMLDocument-success-empty-src-module-actual.txt @@ -1,3 +1,4 @@ +CONSOLE MESSAGE: ReferenceError: Can't find variable: scriptOnError PASS Sanity check around top-level Window 1699519 1 webkit-bug-importer 2020-10-19 16:45:35 -0700 <rdar://problem/70462057> 1699522 2 hector_i_lopez 2020-10-19 16:49:35 -0700 Test expectation while investigated: https://trac.webkit.org/changeset/268698/webkit 1700480 3 ap 2020-10-21 17:18:06 -0700 We are no up to 8 bugs for flaky tests in this one directory, failing with similar messages. 1700482 4 rniwa 2020-10-21 17:21:54 -0700 scriptOnError is referenced in moving-between-documents-iframe.py. Maybe moving-between-documents-helper.js isn't always loaded? 1700509 5 ap 2020-10-21 18:57:57 -0700 I only see 200 OK responses for moving-between-documents-helper.js in WPT server log, https://build.webkit.org/results/Apple-Mojave-Debug-WK2-Tests/r268804%20(12688)/wptwk_process_log.out.txt 1700518 6 rniwa 2020-10-21 19:07:15 -0700 (In reply to Alexey Proskuryakov from comment #5) > I only see 200 OK responses for moving-between-documents-helper.js in WPT > server log, > https://build.webkit.org/results/Apple-Mojave-Debug-WK2-Tests/ > r268804%20(12688)/wptwk_process_log.out.txt Well, if the error happens before that helper JS is loaded, we'd still fail to find the function, right? 1700679 7 ap 2020-10-22 09:27:24 -0700 Maybe? I hope that you found the link useful as a tiny piece of additional information. 1711482 8 gsnedders 2020-12-02 15:11:04 -0800 The identical failures of the various tests in this directory seems to happen on most platforms on debug but not release. They're all timing === "before-prepare" || timing === "move-back" cases in moving-between-documents-iframe.js too. I'm suspicious of the `time.sleep(1)` (i.e., 1s) calls in html/semantics/scripting-1/the-script-element/moving-between-documents/resources/moving-between-documents-iframe.py; I suspect in debug we're just _that slow_ such that the server wins the race and the script runs before window.scriptOnError is assigned. 1711559 9 ap 2020-12-02 18:47:55 -0800 *** Bug 219464 has been marked as a duplicate of this bug. *** 1712898 10 gsnedders 2020-12-08 13:43:28 -0800 *** Bug 214685 has been marked as a duplicate of this bug. *** 1712900 11 gsnedders 2020-12-08 13:43:38 -0800 *** Bug 214734 has been marked as a duplicate of this bug. *** 1712902 12 gsnedders 2020-12-08 13:43:45 -0800 *** Bug 215278 has been marked as a duplicate of this bug. *** 1712904 13 gsnedders 2020-12-08 13:43:51 -0800 *** Bug 215283 has been marked as a duplicate of this bug. *** 1712906 14 gsnedders 2020-12-08 13:43:57 -0800 *** Bug 215706 has been marked as a duplicate of this bug. *** 1712908 15 gsnedders 2020-12-08 13:44:02 -0800 *** Bug 215783 has been marked as a duplicate of this bug. *** 1712910 16 gsnedders 2020-12-08 13:44:08 -0800 *** Bug 218650 has been marked as a duplicate of this bug. *** 1712912 17 gsnedders 2020-12-08 13:44:13 -0800 *** Bug 218651 has been marked as a duplicate of this bug. *** 1712914 18 gsnedders 2020-12-08 13:44:19 -0800 *** Bug 217908 has been marked as a duplicate of this bug. *** 1712921 19 gsnedders 2020-12-08 13:58:07 -0800 filed https://github.com/web-platform-tests/wpt/issues/26808 upstream for this flakiness