Bug 194164 – [ Mac WK2 ] REGRESSION (r234440) Layout Test http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-user-interaction-but-access-from-wrong-frame.html is flaky timeout

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Shawn Roberts]

The following layout test is failing on Mac WK2

http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-user-interaction-but-access-from-wrong-frame.html

Probable cause:

Appears to be a flaky timeout from inception in build 234440 across several builds tested up until current 240850.

Reproducible with:

run-webkit-tests --root t240850 http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-user-interaction-but-access-from-wrong-frame.html --iterations 500 -f

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2FstorageAccess%2Frequest-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-user-interaction-but-access-from-wrong-frame.html

Diff: 

--- /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-user-interaction-but-access-from-wrong-frame-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-user-interaction-but-access-from-wrong-frame-actual.txt
@@ -1,53 +1,5 @@
-Tests that cross-origin iframe storage access is denied if the iframe is sandboxed, has the allow token, the iframe origin is a prevalent resource, the iframe origin has had user interaction, the user opts in, but the frame is not the one with access.
+#PID UNRESPONSIVE - com.apple.WebKit.WebContent.Development (pid 15202)
+FAIL: Timed out waiting for notifyDone to be called

[John Wilander]

Thanks for filing. I think the title is misleading though. The test is flaky timeout, not failing. A test that suddenly starts failing should have a clear point of regression.

[Shawn Roberts]

Sorry about that, updated title.

(In reply to John Wilander from comment #1)
> Thanks for filing. I think the title is misleading though. The test is flaky
> timeout, not failing. A test that suddenly starts failing should have a
> clear point of regression.

[Radar WebKit Bug Importer]

<rdar://problem/48011649>

[Shawn Roberts]

Created attachment 362425 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 362425 [details]
Patch

Rejecting attachment 362425 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-03', 'apply-attachment', '--no-update', '--non-interactive', 362425, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Logging in as [e-mail]...
Fetching: https://bugs.webkit.org/attachment.cgi?id=362425&action=edit
Fetching: https://bugs.webkit.org/show_bug.cgi?id=194164&ctype=xml&excludefield=attachmentdata
Processing 1 patch from 1 bug.
Processing patch 362425 from bug 194164.
Fetching: https://bugs.webkit.org/attachment.cgi?id=362425
Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Parsed 2 diffs from patch file(s).
patching file LayoutTests/ChangeLog
Hunk #1 succeeded at 1 with fuzz 3.
patching file LayoutTests/platform/mac-wk2/TestExpectations
Hunk #2 FAILED at 910.
1 out of 2 hunks FAILED -- saving rejects to file LayoutTests/platform/mac-wk2/TestExpectations.rej

Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Full output: https://webkit-queues.webkit.org/results/11219522

[Shawn Roberts]

Created attachment 362514 [details]
Update TestExpectations

[WebKit Commit Bot]

Comment on attachment 362514 [details]
Update TestExpectations

Rejecting attachment 362514 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-03', 'land-attachment', '--force-clean', '--non-interactive', '--parent-command=commit-queue', 362514, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Logging in as [e-mail]...
Fetching: https://bugs.webkit.org/attachment.cgi?id=362514&action=edit
Fetching: https://bugs.webkit.org/show_bug.cgi?id=194164&ctype=xml&excludefield=attachmentdata
Processing 1 patch from 1 bug.
Updating working directory
Processing patch 362514 from bug 194164.
Fetching: https://bugs.webkit.org/attachment.cgi?id=362514
Failed to run "['git', 'svn', 'dcommit', '--rmdir']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Committing to http://svn.webkit.org/repository/webkit/trunk ...
	M	LayoutTests/ChangeLog

ERROR from SVN:
Item is out of date: File '/trunk/LayoutTests/ChangeLog' is out of date
W: fc16e53f8b7b7bed65c5bdf60e7c07500823779c and refs/remotes/origin/master differ, using rebase:
:040000 040000 edb1adabf42393bd67ce900bce2cdaf7cfa735c2 5c473314cd6b050bdd1a7e2056c007f63bf10d44 M	LayoutTests
Current branch master is up to date.
ERROR: Not all changes have been committed into SVN, however the committed
ones (if any) seem to be successfully integrated into the working tree.
Please see the above messages for details.


Failed to run "['git', 'svn', 'dcommit', '--rmdir']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Committing to http://svn.webkit.org/repository/webkit/trunk ...
	M	LayoutTests/ChangeLog

ERROR from SVN:
Item is out of date: File '/trunk/LayoutTests/ChangeLog' is out of date
W: fc16e53f8b7b7bed65c5bdf60e7c07500823779c and refs/remotes/origin/master differ, using rebase:
:040000 040000 edb1adabf42393bd67ce900bce2cdaf7cfa735c2 5c473314cd6b050bdd1a7e2056c007f63bf10d44 M	LayoutTests
Current branch master is up to date.
ERROR: Not all changes have been committed into SVN, however the committed
ones (if any) seem to be successfully integrated into the working tree.
Please see the above messages for details.


Failed to run "['git', 'svn', 'dcommit', '--rmdir']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit
Updating OpenSource
From https://git.webkit.org/git/WebKit
   30e9cc4a08f..90841d4db41  master     -> origin/master
Partial-rebuilding .git/svn/refs/remotes/origin/master/.rev_map.268f45cc-cd09-0410-ab3c-d52691b4dbfc ...
Currently at 241821 = 30e9cc4a08f6ed87404f55ecd5fde67089944ed6
r241822 = 967d9d29356de32ede1d36fce1c21be0f1ab7740
r241823 = 90841d4db4148abd726079d6a541cf60cfd3a538
Done rebuilding .git/svn/refs/remotes/origin/master/.rev_map.268f45cc-cd09-0410-ab3c-d52691b4dbfc
First, rewinding head to replay your work on top of it...
Fast-forwarded master to refs/remotes/origin/master.
Total errors found: 0 in 1 files

Full output: https://webkit-queues.webkit.org/results/11220603

[Shawn Roberts]

Marked flaky in https://trac.webkit.org/changeset/243827/webkit

[Ryosuke Niwa]

*** Bug 194602 has been marked as a duplicate of this bug. ***

[Ryosuke Niwa]

http/tests/storageAccess/request-and-grant-access-then-detach-should-not-have-access.html is also flaky after the same change.

[Kate Cheney]

Created attachment 382743 [details]
Patch

[Alexey Proskuryakov]

Comment on attachment 382743 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=382743&action=review

> LayoutTests/http/tests/storageAccess/request-and-grant-access-cross-origin-sandboxed-iframe-from-prevalent-domain-with-user-interaction-but-access-from-wrong-frame.html:8
> +<body onload="setTimeout(function () { start() }, 1000);">

A 1 second timeout raises red flags. Regression tests often work under a ot of contention, so timers can be delayed by multiple seconds. Additionally, slowing any test down by a whole second is somewhat costly.

Is there a way to follow the state machine of this test precisely, without timeouts?

[Kate Cheney]

Created attachment 384069 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 384069 [details]
Patch

Clearing flags on attachment: 384069

Committed r252744: <https://trac.webkit.org/changeset/252744>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.