REGRESSION (r240644): Layout Test inspector/page/overrideSetting-ICECandidateFilteringEnabled.html is a flaky timeout

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Truitt Savell]

The following layout test is flaky on Mojave

inspector/page/overrideSetting-ICECandidateFilteringEnabled.html

Probable cause:

Test was introduced in https://trac.webkit.org/changeset/240644/webkit and is a flakey timeout. 

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=inspector%2Fpage%2FoverrideSetting-ICECandidateFilteringEnabled.html

[Radar WebKit Bug Importer]

<rdar://problem/48008005>

[Devin Rousso]

Created attachment 363171 [details]
Patch

Not really sure why it's flaky, but I ran it a few hundred times on my machine (WK1 and WK2) and it seemed to work better

[Joseph Pecoraro]

Comment on attachment 363171 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=363171&action=review

rs=me

> LayoutTests/inspector/page/overrideSetting-ICECandidateFilteringEnabled.html:24
> +        delete pc.onicecandidate;

Just set it to null instead of delete?

[WebKit Commit Bot]

Comment on attachment 363171 [details]
Patch

Clearing flags on attachment: 363171

Committed r242231: <https://trac.webkit.org/changeset/242231>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Shawn Roberts]

This test is still timing out on Mojave WK1 about 1 in every 4 runs on the dashboard.
Also is timing out on Mojave Wk2, but not nearly as consistently.

Was able to reproduce locally with :

run-webkit-tests inspector/page/overrideSetting-ICECandidateFilteringEnabled.html --iterations 500 -f -1

Produced 22 failures.

run-webkit-tests inspector/page/overrideSetting-ICECandidateFilteringEnabled.html --iterations 500 -f

Produced 9 failures.

Flakiness Dashboard : 

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=inspector%2Fpage%2FoverrideSetting-ICECandidateFilteringEnabled.html

[Ryan Haddad]

Reopening based on Shawn's comment above.

[Devin Rousso]

Created attachment 367060 [details]
Patch

It seems like the main issue was with the first part of the test.  Removing it made it pass without issue on 1000+ iterations on my machine.  I've also enabled debug logging just in case it still fails so we can hopefully get more information as to why.

[Joseph Pecoraro]

Comment on attachment 367060 [details]
Patch

r=me

[WebKit Commit Bot]

Comment on attachment 367060 [details]
Patch

Clearing flags on attachment: 367060

Committed r244280: <https://trac.webkit.org/changeset/244280>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Devin Rousso]

This is still showing intermittent failures.

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=inspector%2Fpage%2FoverrideSetting-ICECandidateFilteringEnabled.html

[Devin Rousso]

Created attachment 369754 [details]
Patch

It seems like the failure occurs after the first "part" of the test.  Given that the second "part" doesn't really add anything new (e.g. the first "part" goes from filtering enabled to disabled, while the second "part" does the opposite), I think it's fine to remove.

[Joseph Pecoraro]

Comment on attachment 369754 [details]
Patch

rs=me

[WebKit Commit Bot]

Comment on attachment 369754 [details]
Patch

Clearing flags on attachment: 369754

Committed r245363: <https://trac.webkit.org/changeset/245363>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.