Bug 219464

Summary: [macOS Debug] imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-* tests are flaky
Product: WebKit Reporter: Ryan Haddad <ryanhaddad>
Component: New BugsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED DUPLICATE    
Severity: Normal CC: cdumez, webkit-bot-watchers-bugzilla, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   

Description Ryan Haddad 2020-12-02 16:57:29 PST 
The following layout tests are very flaky on Mojave Debug WK1 EWS bots, failing then passing on retry:

imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-createHTMLDocument-parse-error-inline-classic.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-createHTMLDocument-success-empty-src-classic.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-createHTMLDocument-success-inline-module.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-parse-error-inline-classic.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-parse-error-inline-module.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-success-empty-src-classic.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-success-empty-src-module.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-success-inline-classic.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-success-inline-module.html

Recent examples from EWS:
https://ews-build.s3-us-west-2.amazonaws.com/macOS-Mojave-Debug-WK1-Tests-EWS/r415237-23467/results.html
https://ews-build.s3-us-west-2.amazonaws.com/macOS-Mojave-Debug-WK1-Tests-EWS/r415215-23450/results.html

They also appear to be flaky on trunk bots, but they aren't failing nearly as frequently:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-createHTMLDocument-parse-error-inline-classic.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-createHTMLDocument-success-empty-src-classic.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-createHTMLDocument-success-inline-module.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-iframe-parse-error-inline-classic.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-iframe-parse-error-inline-module.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-iframe-success-empty-src-classic.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-iframe-success-empty-src-module.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-iframe-success-inline-classic.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-iframe-success-inline-module.html
Comment 1 Ryan Haddad 2020-12-02 16:59:57 PST 
Appears to be limited to debug bots.
Comment 2 Ryan Haddad 2020-12-02 17:04:01 PST 
The diffs are a collection of unexpected console messages and unexpected subtest PASS messages.
Comment 3 Radar WebKit Bug Importer 2020-12-02 17:04:15 PST 
<rdar://problem/71917013>
Comment 4 Ryan Haddad 2020-12-02 17:07:01 PST 
Unsure if there is a regression point for this or if they have been flaky since resync.

The large number of flakes are contributing to macOS Debug WK1 EWS intermittently exiting early due to 30 failures. 

Marked the tests as flaky for macOS Debug in r270379.
Comment 5 Alexey Proskuryakov 2020-12-02 18:47:55 PST 

*** This bug has been marked as a duplicate of bug 217931 ***