imported/w3c/web-platform-tests/html/dom/idlharness.https.html Is a flaky failure on macOS-AppleSilicon-Big-Sur-Debug-WK2-Tests-EWS and macOS-Catalina-Release-WK2-Tests-EWS. The flaky failure is not showing up in the open source directory: https://results.webkit.org/?suite=layout-tests&test=imported/w3c/web-platform-tests/html/dom/idlharness.https.html The test started flaky at https://ews-build.webkit.org/#/builders/60/builds/6499 Diff: --- /Volumes/Data/worker/macOS-AppleSilicon-Big-Sur-Debug-WK2-Tests-EWS/build/layout-test-results/imported/w3c/web-platform-tests/html/dom/idlharness.https-expected.txt +++ /Volumes/Data/worker/macOS-AppleSilicon-Big-Sur-Debug-WK2-Tests-EWS/build/layout-test-results/imported/w3c/web-platform-tests/html/dom/idlharness.https-actual.txt @@ -3916,7 +3916,7 @@ PASS CanvasRenderingContext2D interface: operation drawFocusIfNeeded(Element) PASS CanvasRenderingContext2D interface: operation drawFocusIfNeeded(Path2D, Element) FAIL CanvasRenderingContext2D interface: operation scrollPathIntoView() assert_own_property: interface prototype object missing non-static operation expected property "scrollPathIntoView" missing -FAIL CanvasRenderingContext2D interface: operation scrollPathIntoView(Path2D) assert_own_property: interface prototype object missing non-static operation expected property "scrollPathIntoView" missing +FAIL CanvasRenderingContext2D interface: operation scrollPathIntoView(Path2D) assert_own_property: interface prototype object missing non-static operation expected property "sc|ollPathIntoView" missing PASS CanvasRenderingContext2D interface: operation fillText(DOMString, unrestricted double, unrestricted double, optional unrestricted double) PASS CanvasRenderingContext2D interface: operation strokeText(DOMString, unrestricted double, unrestricted double, optional unrestricted double) PASS CanvasRenderingContext2D interface: operation measureText(DOMString)
<rdar://problem/82694768>
Marked expectations: https://trac.webkit.org/changeset/281957/webkit
- scrollPathIntoView + sc|ollPathIntoView What? This can be hard to investigate, given that we don't even see the issue on post-commit bots, but super important to get to the bottom of. > Marked expectations: https://trac.webkit.org/changeset/281957/webkit This effectively removed the effect of this line, making the test show up as always failing on Mac WK2. Ayumi, please add the DumpJSConsoleLogInStdErr property to Mac-wk2 as well. LayoutTests/TestExpectations:819:imported/w3c/web-platform-tests/html/dom/idlharness.https.html [ DumpJSConsoleLogInStdErr ]
"r" -> "|" is not a single bitflip otherwise I would've suspected a hardware issue. Does this appear only on a particular builder?
> The test started flaky at https://ews-build.webkit.org/#/builders/60/builds/6499 Hmm, I assumed that this was happening currently, but the link is for a three month old build, and I cannot find this failure on macOS-AppleSilicon-Big-Sur-Debug-WK2-Tests-EWS in the last month. What motivated filing this bug? Was this happening somewhere else recently?
I see it fail on macOS-Catalina-Release-WK2-Tests-EWS recently, but with an entirely different failure: +FAIL idl_test setup promise_test: Unhandled rejection with value: object "TypeError: Load failed"
Updated test expectations to include DumpJSConsoleLogInStdErr property: https://trac.webkit.org/changeset/282283/webkit
Tried to reproduce locally on Catalina, but the test timed out instead of failing using run-webkit-tests --iterations 50 --exit-after-n-failures 1 --force imported/w3c/web-platform-tests/html/dom/idlharness.https.html
Test hanged with --no-timeout
After https://trac.webkit.org/changeset/282484/webkit, the test started failing on iOS and macOS wk2 that appears to require rebaselining. Diff: --- /Volumes/Data/worker/ios-simulator-14-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/dom/idlharness.https-expected.txt +++ /Volumes/Data/worker/ios-simulator-14-release-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/dom/idlharness.https-actual.txt @@ -4997,14 +4997,14 @@ PASS Worker interface: attribute onmessage FAIL Worker interface: attribute onmessageerror assert_true: The prototype object must have a property "onmessageerror" expected true got false PASS Worker interface: attribute onerror -FAIL SharedWorker interface: existence and properties of interface object assert_own_property: self does not have own property "SharedWorker" expected property "SharedWorker" missing -FAIL SharedWorker interface object length assert_own_property: self does not have own property "SharedWorker" expected property "SharedWorker" missing -FAIL SharedWorker interface object name assert_own_property: self does not have own property "SharedWorker" expected property "SharedWorker" missing -FAIL SharedWorker interface: existence and properties of interface prototype object assert_own_property: self does not have own property "SharedWorker" expected property "SharedWorker" missing -FAIL SharedWorker interface: existence and properties of interface prototype object's "constructor" property assert_own_property: self does not have own property "SharedWorker" expected property "SharedWorker" missing -FAIL SharedWorker interface: existence and properties of interface prototype object's @@unscopables property assert_own_property: self does not have own property "SharedWorker" expected property "SharedWorker" missing -FAIL SharedWorker interface: attribute port assert_own_property: self does not have own property "SharedWorker" expected property "SharedWorker" missing -FAIL SharedWorker interface: attribute onerror assert_own_property: self does not have own property "SharedWorker" expected property "SharedWorker" missing +PASS SharedWorker interface: existence and properties of interface object +PASS SharedWorker interface object length +PASS SharedWorker interface object name +PASS SharedWorker interface: existence and properties of interface prototype object +PASS SharedWorker interface: existence and properties of interface prototype object's "constructor" property +PASS SharedWorker interface: existence and properties of interface prototype object's @@unscopables property +PASS SharedWorker interface: attribute port +PASS SharedWorker interface: attribute onerror PASS WorkerNavigator interface: existence and properties of interface object PASS WorkerLocation interface: existence and properties of interface object PASS WorkletGlobalScope interface: existence and properties of interface object
Rebaselined here https://trac.webkit.org/changeset/282592/webkit. Also removed the previous test expectations.
Reverted r282592 for reason: Rebaselined wrong on macOS Committed r282618 (241776@main): <https://commits.webkit.org/241776@main>
After rebaselining both iOS and macOS, macOS started failing. (I think that I added a wrong expected.txt) Reverted my change and rebaselined again only iOS. macOS expected.txt and expectations stay same: https://trac.webkit.org/changeset/282619/webkit
Committed r287657 (245754@trunk): <https://commits.webkit.org/245754@trunk>