Bug 219464 - [macOS Debug] imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-* tests are flaky
Summary: [macOS Debug] imported/w3c/web-platform-tests/html/semantics/scripting-1/the-...
Status: RESOLVED DUPLICATE of bug 217931
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-12-02 16:57 PST by Ryan Haddad
Modified: 2020-12-02 18:47 PST (History)
3 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Ryan Haddad 2020-12-02 16:57:29 PST
The following layout tests are very flaky on Mojave Debug WK1 EWS bots, failing then passing on retry:

imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-createHTMLDocument-parse-error-inline-classic.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-createHTMLDocument-success-empty-src-classic.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-createHTMLDocument-success-inline-module.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-parse-error-inline-classic.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-parse-error-inline-module.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-success-empty-src-classic.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-success-empty-src-module.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-success-inline-classic.html
imported/w3c/web-platform-tests/html/semantics/scripting-1/the-script-element/moving-between-documents/before-prepare-iframe-success-inline-module.html

Recent examples from EWS:
https://ews-build.s3-us-west-2.amazonaws.com/macOS-Mojave-Debug-WK1-Tests-EWS/r415237-23467/results.html
https://ews-build.s3-us-west-2.amazonaws.com/macOS-Mojave-Debug-WK1-Tests-EWS/r415215-23450/results.html

They also appear to be flaky on trunk bots, but they aren't failing nearly as frequently:
https://results.webkit.org/?suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-createHTMLDocument-parse-error-inline-classic.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-createHTMLDocument-success-empty-src-classic.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-createHTMLDocument-success-inline-module.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-iframe-parse-error-inline-classic.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-iframe-parse-error-inline-module.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-iframe-success-empty-src-classic.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-iframe-success-empty-src-module.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-iframe-success-inline-classic.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fscripting-1%2Fthe-script-element%2Fmoving-between-documents%2Fbefore-prepare-iframe-success-inline-module.html
Comment 1 Ryan Haddad 2020-12-02 16:59:57 PST
Appears to be limited to debug bots.
Comment 2 Ryan Haddad 2020-12-02 17:04:01 PST
The diffs are a collection of unexpected console messages and unexpected subtest PASS messages.
Comment 3 Radar WebKit Bug Importer 2020-12-02 17:04:15 PST
<rdar://problem/71917013>
Comment 4 Ryan Haddad 2020-12-02 17:07:01 PST
Unsure if there is a regression point for this or if they have been flaky since resync.

The large number of flakes are contributing to macOS Debug WK1 EWS intermittently exiting early due to 30 failures. 

Marked the tests as flaky for macOS Debug in r270379.
Comment 5 Alexey Proskuryakov 2020-12-02 18:47:55 PST

*** This bug has been marked as a duplicate of bug 217931 ***