Bug 229094

Summary: REGRESSION: [ iOS ] 5 TestWebKitAPI.WebpagePreferences.* api tests are flaky timing out
Product: WebKit Reporter: ayumi_kojima
Component: WebKit APIAssignee: Alex Christensen <achristensen>
Status: RESOLVED FIXED    
Severity: Normal CC: achristensen, cdumez, darin, ehutchison, ggaren, webkit-bot-watchers-bugzilla, webkit-bug-importer, wenson_hsieh
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: iPhone / iPad   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=229452
https://bugs.webkit.org/show_bug.cgi?id=229258
Attachments:
Description Flags
Patch
none
Patch ews-feeder: commit-queue-

Description ayumi_kojima 2021-08-13 14:02:33 PDT 
TestWebKitAPI.WebpagePreferences.WebsitePoliciesAutoplayEnabled
TestWebKitAPI.WebpagePreferences.WebsitePoliciesUserInterferenceWithPlaying
TestWebKitAPI.WebpagePreferences.WebsitePoliciesUpdates
TestWebKitAPI.WebpagePreferences.WebsitePoliciesPlayingWithoutInterference
TestWebKitAPI.WebpagePreferences.WebsitePoliciesPlayingWithUserGesture

Are flaky timing out on iOS 14 E Simulator. 

History: https://results.webkit.org/?suite=api-tests&suite=api-tests&suite=api-tests&suite=api-tests&suite=api-tests&test=TestWebKitAPI.WebpagePreferences.WebsitePoliciesAutoplayEnabled&test=TestWebKitAPI.WebpagePreferences.WebsitePoliciesPlayingWithUserGesture&test=TestWebKitAPI.WebpagePreferences.WebsitePoliciesPlayingWithoutInterference&test=TestWebKitAPI.WebpagePreferences.WebsitePoliciesUpdates&test=TestWebKitAPI.WebpagePreferences.WebsitePoliciesUserInterferenceWithPlaying

stdio: https://build.webkit.org/#/builders/45/builds/1794/steps/17/logs/stdio

Looking at the history, it looks like the flaky timeout has started at https://trac.webkit.org/changeset/279835/webkit because the tests were consistently crashing for a short period of time before they started flaky timing out and this change https://trac.webkit.org/changeset/279679/webkit fixed the crash.
Comment 1 Radar WebKit Bug Importer 2021-08-13 14:03:28 PDT 
<rdar://problem/81915299>
Comment 2 Chris Dumez 2021-08-13 15:31:46 PDT 
I don't think the regression point is correct.

r279835 got reverted in r279840 and yet the tests were still flaky after that.
Comment 3 ayumi_kojima 2021-08-13 15:48:04 PDT 
You are right. It is r279840.
Comment 4 Chris Dumez 2021-08-13 15:50:48 PDT 
(In reply to ayumi_kojima from comment #3)
> You are right. It is r279840.

No, this doesn't make sense either since r279840 is a simple revert of r279835.
Comment 5 ayumi_kojima 2021-08-13 16:07:45 PDT 
Okay, now I understand it. I will try bisect it and research more. Thanks Chris!
Comment 6 Eric Hutchison 2021-08-13 18:27:59 PDT 
Reproduction: 

1) TestWebKitAPI.WebpagePreferences.WebsitePoliciesAutoplayEnabled timeout was reproduced on r279831 using run-api-tests --iOS-simulator --release TestWebKitAPI.WebpagePreferences.WebsitePoliciesAutoplayEnabled. The test passed on r 279866 using the same command. There are no available spades between those revisions to test on.

2) TestWebKitAPI.WebpagePreferences.WebsitePoliciesUserInterferenceWithPlaying
TestWebKitAPI.WebpagePreferences.WebsitePoliciesUpdates
TestWebKitAPI.WebpagePreferences.WebsitePoliciesPlayingWithoutInterference
TestWebKitAPI.WebpagePreferences.WebsitePoliciesPlayingWithUserGesture:
each of these tests passed on r279831 and r279866 with the same command as above. Was unable to reproduce the timeout on these tests.
Comment 7 Eric Hutchison 2021-09-08 11:39:35 PDT 
Attempted to reproduce timeouts using run-api-tests --iterations 100 --iOS-simulator --no-build tesname on r281989 and r281920. Unable to reproduce.
Comment 8 Alex Christensen 2021-09-17 11:25:12 PDT 
Created attachment 438491 [details]
Patch
Comment 9 EWS 2021-09-17 11:56:43 PDT 
Committed r282678 (241819@main): <https://commits.webkit.org/241819@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 438491 [details].
Comment 10 Eric Hutchison 2021-09-17 14:14:02 PDT 
Patch applied disables test; issue is not resolved. Reopening bug.
Comment 11 Alex Christensen 2021-12-20 18:03:41 PST 
Created attachment 447672 [details]
Patch
Comment 12 EWS 2021-12-20 22:23:54 PST 
Committed r287298 (245450@main): <https://commits.webkit.org/245450@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 447672 [details].