Bug 257624
| Summary: | [GTK][WPE] Flaky tests detected on May of 2023 | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Carlos Alberto Lopez Perez <clopez> |
| Component: | WebKitGTK | Assignee: | Nobody <webkit-unassigned> |
| Status: | NEW | ||
| Severity: | Normal | CC: | bugs-noreply, dpino, klee |
| Priority: | P2 | ||
| Version: | WebKit Nightly Build | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| See Also: | https://bugs.webkit.org/show_bug.cgi?id=257623 | ||
Carlos Alberto Lopez Perez
On 1th of June I ran flakyhunter tool to detect flaky tests on the tree not gardened during May.
This is what got analyzed
GTK:
Analyzing test results for the last 200 runs of the bot gtk-release...
Got 7192 test results for 200 runs in the range [263752@main-264794@main]
WPE:
Analyzing test results for the last 200 runs of the bot wpe-release...
Got 7204 test results for 200 runs in the range [264116@main-264802@main
| Attachments | ||
|---|---|---|
| Add attachment proposed patch, testcase, etc. |
Carlos Alberto Lopez Perez
I uploaded the raw output from the flakyhunter here: https://people.igalia.com/clopez/wkbug/257624/
I processed that and added the new flakies for both GTK and WPE on https://commits.webkit.org/264809@main
If you have doubts about why a test was added there then grep the name of the test on that files uploaded above and it will give you hints.
Diego Pino
fast/mediastream/RTCSessionDescription.html is not crashing anymore since 264659@main, probably fixed by 264664@main. I'll remove it from TestExpectations.
klee
The following tests are still reported as flaky:
fast/mediastream/MediaStream-video-element-enter-background.html [ Failure Pass ]
fast/mediastream/MediaStream-video-element-track-stop.html [ Failure ]
imported/w3c/web-platform-tests/html/infrastructure/urls/resolving-urls/query-encoding/windows-1251.html?include=loading [ Failure Pass ]
imported/w3c/web-platform-tests/media-source/mediasource-play.html [ Failure Pass ]
webgl/1.0.x/conformance/rendering/texture-switch-performance.html [ Failure Pass ]
webgl/2.0.y/conformance/rendering/texture-switch-performance.html [ Failure Pass ]
Based on the results from the last 50,000 commits, the following tests have been consistently passing and can be removed from the flaky test list:
media/video-seek-with-negative-playback.html [ Pass Timeout ]
webaudio/audiocontext-state-interrupted.html [ Pass Timeout ]
webaudio/Panner/panner-cone-gain-nan.html [ Pass Timeout ]
webaudio/suspend-context-while-backgrounded.html [ Pass Timeout ]
webaudio/suspend-context-while-interrupted.html [ Crash Pass ]
Please refer to the following link on results.webkit.org for details:
https://results.webkit.org/?limit=50000&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&suite=layout-tests&test=webaudio%2Fsuspend-context-while-interrupted.html&test=webaudio%2Fsuspend-context-while-backgrounded.html&test=webaudio%2FPanner%2Fpanner-cone-gain-nan.html&test=webaudio%2Faudiocontext-state-interrupted.html&test=media%2Fvideo-seek-with-negative-playback.html&test=fast%2Fmediastream%2FMediaStream-video-element-enter-background.html&test=imported%2Fblink%2Ffast%2Fhidpi%2Fborder-background-align.html&test=http%2Fwpt%2Fservice-workers%2Fcontrolled-sharedworker.https.html&test=http%2Fwpt%2Fcross-origin-opener-policy%2Fheader-parsing-with-report-to.https.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fcss%2Fcss-backgrounds%2Fbackground-attachment-margin-root-001.html&test=imported%2Fblink%2Ffast%2Ftransforms%2Ftransform-update-frame-overflow.html&test=imported%2Fblink%2Ffast%2Fhidpi%2Fborder-background-align.html&test=http%2Fwpt%2Fservice-workers%2Fcontrolled-sharedworker.https.html&test=http%2Ftests%2Fmedia%2Fuser-gesture-preserved-across-xmlhttprequest.html&test=http%2Ftests%2Fmedia%2Fvideo-play-waiting.html&test=gamepad%2Fgamepad-event-handlers.html&test=fast%2Fscrolling%2Fkeyboard-scrolling-distance-pageDown.html&test=fast%2Fhidpi%2Fimage-set-units.html&test=fast%2Fhidpi%2Fimage-set-gradient-multi.html&test=fast%2Fhidpi%2Fimage-set-gradient.html&test=fast%2Fhidpi%2Fhidpi-form-controls-drawing-size.html&test=fast%2Fhidpi%2Ffilters-shadow.html&test=fast%2Fhidpi%2Ffilters-morphology.html&test=fast%2Fhidpi%2Ffilters-component-transfer.html&test=fast%2Fhidpi%2Ffilters-and-image-buffer-resolution.html&test=fast%2Fevents%2Fmouse-events-on-textarea-resize.html&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Fembedded-content%2Fmedia-elements%2Fready-states%2Fautoplay.html&test=perf%2Fshow-hide-table-rows.html&test=js%2Fdom%2Fcreate-lots-of-workers.html&test=http%2Fwpt%2Fcross-origin-opener-policy%2Fheader-parsing-with-report-to.https.html&platform=WPE