Bug 260546 – REGRESSION:267076@main [ macOS ] [ Debug ] imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-details-element/toggleEvent.html is a flaky test failure

• Status: NEW
• Component: DOM

[Dawn Morningstar]

imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-details-element/toggleEvent.html 
is a flaky text failure after 267076@main where toggleEvent was enabled and the failure expectation was removed.
This failure is only on debug, and is nearly constant, but the flakes make rebaslining not the solution.

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Finteractive-elements%2Fthe-details-element%2FtoggleEvent.html&style=debug

DIFF:
--- /Volumes/Data/worker/Apple-Ventura-Debug-WK2-GPUProcess-Tests/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-details-element/toggleEvent-expected.txt
+++ /Volumes/Data/worker/Apple-Ventura-Debug-WK2-GPUProcess-Tests/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-details-element/toggleEvent-actual.txt
@@ -3,7 +3,7 @@
 PASS Removing open from 'details' should fire a toggle event at the 'details' element, with 'oldState: open' and 'newState: closed'
 PASS Adding open to 'details' (display:none) should fire a toggle event at the 'details' element, with 'oldState: closed' and 'newState: open'
 PASS Adding open to 'details' (no children) should fire a toggle event at the 'details' element, with 'oldState: closed' and 'newState: open'
-PASS Calling open twice on 'details' fires only one toggle event, with 'oldState: closed' and 'newState: open'
+FAIL Calling open twice on 'details' fires only one toggle event, with 'oldState: closed' and 'newState: open' assert_true: expected true got false
 PASS Calling setAttribute('open', '') from 'details' should fire a toggle event at the 'details' element, with 'oldState: closed' and 'newState: open'
 PASS Calling removeAttribute('open') from 'details' should fire a toggle event at the 'details' element, with 'oldState: open' and 'newState: closed'
 PASS Setting open=true to opened 'details' element should not fire a toggle event at the 'details' element

DIFF-URL:
https://build.webkit.org/results/Apple-Ventura-Debug-WK2-GPUProcess-Tests/267082@main%20(3372)/imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-details-element/toggleEvent-diff.txt 

This is reproducible at TOT on a debug spade with: rwt --root [buildtoTest] imported/w3c/web-platform-tests/html/semantics/interactive-elements/the-details-element/toggleEvent.html --iterations 100

[Radar WebKit Bug Importer]

<rdar://problem/114281621>

[Dawn Morningstar]

Asking Tim how to proceed since Michael is in a much different time zone.

[sideshowbarker]

Yeah it had been marked [ Pass Failure ] in LayoutTests/TestExpectations but I removed that in my commit — see https://github.com/WebKit/WebKit/commit/74637c383d415af9b4e4ca2ce7fbb3268e6ead85#diff-0b438e462819a7b0d06250189152c7bcfb07b7c39acd73938bdfccc7b25b8058

In hindsight, I probably shouldn’t have done that — because that test is flaky locally as well — specifically, only one particular subtest.

Should I do another PR which adds it back to LayoutTests/TestExpectations as [ Pass Failure ]?

[Dawn Morningstar]

(In reply to Michael[tm] Smith from comment #3)
> Yeah it had been marked [ Pass Failure ] in LayoutTests/TestExpectations but
> I removed that in my commit — see
> https://github.com/WebKit/WebKit/commit/
> 74637c383d415af9b4e4ca2ce7fbb3268e6ead85#diff-
> 0b438e462819a7b0d06250189152c7bcfb07b7c39acd73938bdfccc7b25b8058
> 
> In hindsight, I probably shouldn’t have done that — because that test is
> flaky locally as well — specifically, only one particular subtest.
> 
> Should I do another PR which adds it back to LayoutTests/TestExpectations as
> [ Pass Failure ]?

Michael, No worries! I noticed the expectation was removed, I can handle adding it back, but I was wondering if I should only add it for Debug since it's now passing elsewhere.

[sideshowbarker]

(In reply to Dawn Flores from comment #4)
> Michael, No worries! I noticed the expectation was removed, I can handle
> adding it back, but I was wondering if I should only add it for Debug since
> it's now passing elsewhere.

That part maybe we need for Tim to weigh in on. In my local testing for the patch, I was only building Debug builds — I never tested with any Release builds. So I can’t say that I’ve ever seen the flakiness in a local Release build (though I definitely did with my local Debug builds).

[Tim Nguyen (:ntim)]

Dawn, can we mark it as [ Pass Failure ] on [ Debug ] only for now?

[Dawn Morningstar]

Pull request: https://github.com/WebKit/WebKit/pull/17081

[EWS]

Test gardening commit 267298@main (8df3f2fc4f5d): <https://commits.webkit.org/267298@main>

Reviewed commits have been landed. Closing PR #17081 and removing active labels.