Bug 229094 - REGRESSION: [ iOS ] 5 TestWebKitAPI.WebpagePreferences.* api tests are flaky timing out
Summary: REGRESSION: [ iOS ] 5 TestWebKitAPI.WebpagePreferences.* api tests are flaky ...
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKit API (show other bugs)
Version: WebKit Nightly Build
Hardware: iPhone / iPad Unspecified
: P2 Normal
Assignee: Alex Christensen
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-08-13 14:02 PDT by ayumi_kojima
Modified: 2021-09-17 14:14 PDT (History)
7 users (show)

See Also:


Attachments
Patch (3.56 KB, patch)
2021-09-17 11:25 PDT, Alex Christensen
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description ayumi_kojima 2021-08-13 14:02:33 PDT
TestWebKitAPI.WebpagePreferences.WebsitePoliciesAutoplayEnabled
TestWebKitAPI.WebpagePreferences.WebsitePoliciesUserInterferenceWithPlaying
TestWebKitAPI.WebpagePreferences.WebsitePoliciesUpdates
TestWebKitAPI.WebpagePreferences.WebsitePoliciesPlayingWithoutInterference
TestWebKitAPI.WebpagePreferences.WebsitePoliciesPlayingWithUserGesture

Are flaky timing out on iOS 14 E Simulator. 

History: https://results.webkit.org/?suite=api-tests&suite=api-tests&suite=api-tests&suite=api-tests&suite=api-tests&test=TestWebKitAPI.WebpagePreferences.WebsitePoliciesAutoplayEnabled&test=TestWebKitAPI.WebpagePreferences.WebsitePoliciesPlayingWithUserGesture&test=TestWebKitAPI.WebpagePreferences.WebsitePoliciesPlayingWithoutInterference&test=TestWebKitAPI.WebpagePreferences.WebsitePoliciesUpdates&test=TestWebKitAPI.WebpagePreferences.WebsitePoliciesUserInterferenceWithPlaying

stdio: https://build.webkit.org/#/builders/45/builds/1794/steps/17/logs/stdio

Looking at the history, it looks like the flaky timeout has started at https://trac.webkit.org/changeset/279835/webkit because the tests were consistently crashing for a short period of time before they started flaky timing out and this change https://trac.webkit.org/changeset/279679/webkit fixed the crash.
Comment 1 Radar WebKit Bug Importer 2021-08-13 14:03:28 PDT
<rdar://problem/81915299>
Comment 2 Chris Dumez 2021-08-13 15:31:46 PDT
I don't think the regression point is correct.

r279835 got reverted in r279840 and yet the tests were still flaky after that.
Comment 3 ayumi_kojima 2021-08-13 15:48:04 PDT
You are right. It is r279840.
Comment 4 Chris Dumez 2021-08-13 15:50:48 PDT
(In reply to ayumi_kojima from comment #3)
> You are right. It is r279840.

No, this doesn't make sense either since r279840 is a simple revert of r279835.
Comment 5 ayumi_kojima 2021-08-13 16:07:45 PDT
Okay, now I understand it. I will try bisect it and research more. Thanks Chris!
Comment 6 Eric Hutchison 2021-08-13 18:27:59 PDT
Reproduction: 

1) TestWebKitAPI.WebpagePreferences.WebsitePoliciesAutoplayEnabled timeout was reproduced on r279831 using run-api-tests --iOS-simulator --release TestWebKitAPI.WebpagePreferences.WebsitePoliciesAutoplayEnabled. The test passed on r 279866 using the same command. There are no available spades between those revisions to test on.

2) TestWebKitAPI.WebpagePreferences.WebsitePoliciesUserInterferenceWithPlaying
TestWebKitAPI.WebpagePreferences.WebsitePoliciesUpdates
TestWebKitAPI.WebpagePreferences.WebsitePoliciesPlayingWithoutInterference
TestWebKitAPI.WebpagePreferences.WebsitePoliciesPlayingWithUserGesture:
each of these tests passed on r279831 and r279866 with the same command as above. Was unable to reproduce the timeout on these tests.
Comment 7 Eric Hutchison 2021-09-08 11:39:35 PDT
Attempted to reproduce timeouts using run-api-tests --iterations 100 --iOS-simulator --no-build tesname on r281989 and r281920. Unable to reproduce.
Comment 8 Alex Christensen 2021-09-17 11:25:12 PDT
Created attachment 438491 [details]
Patch
Comment 9 EWS 2021-09-17 11:56:43 PDT
Committed r282678 (241819@main): <https://commits.webkit.org/241819@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 438491 [details].
Comment 10 Eric Hutchison 2021-09-17 14:14:02 PDT
Patch applied disables test; issue is not resolved. Reopening bug.