Bug 195466

Summary: (r242595) Layout Tests in imported/w3c/web-platform-tests/html/semantics/embedded-content/* are failing
Product: WebKit Reporter: Truitt Savell <tsavell>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: NEW ---    
Severity: Normal CC: eric.carlson, fred.wang, jer.noble, rwlbuis, sroberts, webkit-bot-watchers-bugzilla
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=194770

Description Truitt Savell 2019-03-08 10:14:04 PST
The following layout tests are failing on Mac

imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/ready-states/autoplay.html
imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/error-codes/error.html

Probable cause:

These two tests were added as part of https://trac.webkit.org/changeset/242595/webkit. They are flakey on introduction. 

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fembedded-content%2Fmedia-elements%2Fready-states%2Fautoplay.html%20imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fembedded-content%2Fmedia-elements%2Ferror-codes%2Ferror.html
Comment 1 Truitt Savell 2019-03-08 10:25:21 PST
marked tests as flakey in:
https://trac.webkit.org/changeset/242644/webkit
Comment 2 Truitt Savell 2019-03-13 12:47:58 PDT
Additional failing test from the above changes:
imported/w3c/web-platform-tests/html/semantics/embedded-content/the-img-element/sizes/parse-a-sizes-attribute-width-1000px.html
Comment 3 Truitt Savell 2019-03-13 13:08:01 PDT
marked flakey in:
https://trac.webkit.org/changeset/242900/webkit
Comment 4 Shawn Roberts 2019-04-03 09:20:19 PDT
Tests are now showing up as a flaky failure on iOS Simulator Release and Debug as well.

Reproduced with:

run-webkit-tests imported/w3c/web-platform-tests/html/semantics/embedded-content/the-img-element/sizes/parse-a-sizes-attribute-width-1000px.html --iterations 500 -f --ios-simulator --exit-after-n-failures=5

Fails 5 within 22 iterations.

Diff:

https://build.webkit.org/results/Apple%20iOS%2012%20Simulator%20Release%20WK2%20(Tests)/r243794%20(3430)/imported/w3c/web-platform-tests/html/semantics/embedded-content/the-img-element/sizes/parse-a-sizes-attribute-width-1000px-diff.txt
Comment 5 Shawn Roberts 2019-04-03 09:30:54 PDT
Updated test expectations for iOS in https://trac.webkit.org/changeset/243802/webkit