Bug 231918 – REGRESSION (r283935): [ macOS wk1 ] imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-dialog-element/dialog-autofocus-multiple-times.html is a flaky failure

• Component: New Bugs
• Status: REOPENED

imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-dialog-element/dialog-autofocus-multiple-times.html

Is a flaky failure on macOS wk1 Release.

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Finteractive-elements%2Fthe-dialog-element%2Fdialog-autofocus-multiple-times.html

Result page: https://build.webkit.org/results/Apple-BigSur-Release-WK1-Tests/r284398%20(6218)/results.html

Diff:

--- /Volumes/Data/worker/bigsur-release-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-dialog-element/dialog-autofocus-multiple-times-expected.txt
+++ /Volumes/Data/worker/bigsur-release-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-dialog-element/dialog-autofocus-multiple-times-actual.txt
@@ -1,4 +1,6 @@
 
 
-PASS autofocus is run every time a dialog is opened
+Harness Error (TIMEOUT), message = null
 
+TIMEOUT autofocus is run every time a dialog is opened Test timed out
+


According to the history, the test started being flaky at around r283935.

[Radar WebKit Bug Importer]

<rdar://problem/84393055>

The failure is also seen on wk1 Debug.

I was able to reproduce the failure locally using run-webkit-tests -1 --iterations 50 --exit-after-n-failures 1 imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-dialog-element/dialog-autofocus-multiple-times.html

I was able to reproduce the failure with r283938 and the test passed with r283934. No revision available to test in between.

I think https://trac.webkit.org/changeset/283935/webkit might have caused the regression.

Marked test expectations https://trac.webkit.org/changeset/284414/webkit

*** Bug 231969 has been marked as a duplicate of this bug. ***

imported/w3c/web-platform-tests/html/semantics/selectors/pseudo-classes/focus-autofocus.html is also flaky failing on macOS wk1 Release with the same diff.

Marked expectations https://trac.webkit.org/changeset/284493/webkit

*** Bug 231969 has been marked as a duplicate of this bug. ***

[Chris Dumez]

Created attachment 443548 [details]
Patch

[Tim Nguyen (:ntim)]

Comment on attachment 443548 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=443548&action=review

> LayoutTests/platform/mac-wk1/TestExpectations:788
>  webkit.org/b/231918 imported/w3c/web-platform-tests/html/semantics/selectors/pseudo-classes/focus-autofocus.html [ Pass Failure ]
>  
>  webkit.org/b/232042 imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-dialog-element/dialog-autofocus.html [ Pass Failure ] 

Can you check if those tests stop being flaky as well and remove them if they're not?

They're part of the same series of wk1 autofocus flaky failures

[Tim Nguyen (:ntim)]

There's also: https://webkit-search.igalia.com/webkit/rev/80a3f4df82d5a2a27251e2aa6dd32f94d35c724f/LayoutTests/platform/mac-wk1/TestExpectations#1417,1419

[Chris Dumez]

(In reply to Tim Nguyen (:ntim) from comment #11)
> Comment on attachment 443548 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=443548&action=review
> 
> > LayoutTests/platform/mac-wk1/TestExpectations:788
> >  webkit.org/b/231918 imported/w3c/web-platform-tests/html/semantics/selectors/pseudo-classes/focus-autofocus.html [ Pass Failure ]
> >  
> >  webkit.org/b/232042 imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-dialog-element/dialog-autofocus.html [ Pass Failure ] 
> 
> Can you check if those tests stop being flaky as well and remove them if
> they're not?
> 
> They're part of the same series of wk1 autofocus flaky failures

imported/w3c/web-platform-tests/html/semantics/selectors/pseudo-classes/focus-autofocus.html is still flaky. I haven't checked the others yet.

[Chris Dumez]

imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-dialog-element/dialog-autofocus.html is fixed by this patch too. I'll unskip it.

[Chris Dumez]

*** Bug 232042 has been marked as a duplicate of this bug. ***

[Chris Dumez]

Created attachment 443611 [details]
Patch

[Chris Dumez]

(In reply to Chris Dumez from comment #14)
> imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-
> dialog-element/dialog-autofocus.html is fixed by this patch too. I'll unskip
> it.

imported/w3c/web-platform-tests/html/interaction/focus/the-autofocus-attribute/document-with-fragment-valid.html is still flaky though, so different reason.

[Chris Dumez]

(In reply to Chris Dumez from comment #17)
> (In reply to Chris Dumez from comment #14)
> > imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-
> > dialog-element/dialog-autofocus.html is fixed by this patch too. I'll unskip
> > it.
> 
> imported/w3c/web-platform-tests/html/interaction/focus/the-autofocus-
> attribute/document-with-fragment-valid.html is still flaky though, so
> different reason.

Ditto for imported/w3c/web-platform-tests/html/interaction/focus/the-autofocus-attribute/spin-by-blocking-style-sheet.html. Still flaky timeout so likely caused by something else.

[EWS]

Committed r285449 (243999@main): <https://commits.webkit.org/243999@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 443611 [details].

[Antoine Quint]

I just saw this test being flaky during this test run: https://ews-build.webkit.org/#/builders/56/builds/22257.

[Antoine Quint]

https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Finteractive-elements%2Fthe-dialog-element%2Fdialog-autofocus.html suggests that at least this test, which was unskipped in this patch, is not reliably passing.

Reopening based on this evidence.