Bug 195425 - [ iOS Sim ] REGRESSION (r242277) Layout Test http/tests/cookies/same-site/lax-samesite-cookie-after-cross-site-history-load.php is a flaky timeout
Summary: [ iOS Sim ] REGRESSION (r242277) Layout Test http/tests/cookies/same-site/lax...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Chris Dumez
URL:
Keywords: InRadar
Depends on:
Blocks: 194023
  Show dependency tree / graph
 
Reported: 2019-03-07 11:09 PST by Shawn Roberts
Modified: 2019-03-27 13:47 PDT (History)
9 users (show)

See Also:

Attachments
Patch (3.47 KB, patch)
2019-03-27 13:10 PDT, Chris Dumez 		no flags Details | Formatted Diff | Diff
View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Shawn Roberts 2019-03-07 11:09:55 PST 
The following layout test a timeout on iOS Simulator

http/tests/cookies/same-site/lax-samesite-cookie-after-cross-site-history-load.php

Probable cause:

Test will timeout when it runs after any layout test in the http/tests/cookies/same-site/* directory
Then pass on the re-try
Prior to r242277 this test case is passing

Reproduced with:

run-webkit-tests http/tests/cookies/same-site/ --child-processes 1 --ios-simulator

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fcookies%2Fsame-site%2Flax-samesite-cookie-after-cross-site-history-load.php
Comment 1 Radar WebKit Bug Importer 2019-03-07 11:10:33 PST 
<rdar://problem/48682403>
Comment 2 Shawn Roberts 2019-03-08 09:58:07 PST 
Additional repro steps:

Test first showed on the dashboard for iOS Sim / iOS Debug around r242280

Was unable to reproduce locally running just the test by itself using: 

run-webkit-tests http/tests/cookies/same-site/lax-samesite-cookie-after-cross-site-history-load.php --iterations 500 - f --ios-simulator

Looked at build log for Apple iOS 12 Simulator Release WK2 (Tests) Build #3027 and pulled the worker test list that included this test using Layout test failure analysis website.

Created a test list file from this worker list, and found test failed when running with this test list, then through bisection removed tests preceding this test until I found when running any test in the "http/tests/cookies/same-site/" caused this test to fail.

Searched on trac and found the test change to this test was r241070 and used this as a bisecting point. 

Tested with the following revisions using this command "run-webkit-tests http/tests/cookies/same-site/ --child-processes 1 --ios-simulator"

242623 Timeout
242435 Timeout
242336 Timeout
242285 Timeout
242280 Timeout
242277 Timeout

242276 Passed
242275 Passed
242260 Passed
242234 Passed
241847 Passed
241070 Passed
Comment 3 Chris Dumez 2019-03-21 15:28:52 PDT 
Seems very unlikely http://trac.webkit.org/changeset/242277/webkit is the cause but ok.
Comment 4 Chris Dumez 2019-03-21 15:44:28 PDT 
Seems to reproduce like so:
run-webkit-tests LayoutTests/http/tests/cookies/same-site/fetch-in-same-origin-worker.html LayoutTests/http/tests/cookies/same-site/lax-samesite-cookie-after-cross-site-history-load.php --child-processes 1 --ios-simulator --no-build
Comment 5 Chris Dumez 2019-03-27 13:10:14 PDT 
Created attachment 366097 [details]
Patch
Comment 6 WebKit Commit Bot 2019-03-27 13:47:58 PDT 
Comment on attachment 366097 [details]
Patch

Clearing flags on attachment: 366097

Committed r243562: <https://trac.webkit.org/changeset/243562>
Comment 7 WebKit Commit Bot 2019-03-27 13:47:59 PDT 
All reviewed patches have been landed.  Closing bug.