Bug 228089 - [ MacOS & iOS & GTK ] imported/w3c/web-platform-tests/html/semantics/links/links-created-by-a-and-area-elements/htmlanchorelement_noopener.html is flaky failing
Summary: [ MacOS & iOS & GTK ] imported/w3c/web-platform-tests/html/semantics/links/li...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Chris Dumez
URL:
Keywords: InRadar
Depends on: 230181
Blocks:
  Show dependency treegraph
 
Reported: 2021-07-19 14:09 PDT by ayumi_kojima
Modified: 2021-09-16 15:20 PDT (History)
10 users (show)

See Also:


Attachments
Patch (8.02 KB, patch)
2021-09-15 08:55 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff
Patch (8.75 KB, patch)
2021-09-15 10:14 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff
Patch (23.59 KB, patch)
2021-09-15 10:25 PDT, Chris Dumez
ews-feeder: commit-queue-
Details | Formatted Diff | Diff
Patch (8.72 KB, patch)
2021-09-15 13:08 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff
Patch (9.60 KB, patch)
2021-09-15 14:13 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff
Patch (20.35 KB, patch)
2021-09-16 07:34 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description ayumi_kojima 2021-07-19 14:09:13 PDT
imported/w3c/web-platform-tests/html/semantics/links/links-created-by-a-and-area-elements/htmlanchorelement_noopener.html 

Is flaky failing on MacOS wk1 

History: https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fsemantics%2Flinks%2Flinks-created-by-a-and-area-elements%2Fhtmlanchorelement_noopener.html

Diff:

========================

--- /Volumes/Data/worker/bigsur-debug-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/links/links-created-by-a-and-area-elements/htmlanchorelement_noopener-expected.txt
+++ /Volumes/Data/worker/bigsur-debug-tests-wk1/build/layout-test-results/imported/w3c/web-platform-tests/html/semantics/links/links-created-by-a-and-area-elements/htmlanchorelement_noopener-actual.txt
@@ -3,6 +3,6 @@
 PASS Check that rel=noopener with target=_self does a normal load
 PASS Check that rel=noopener with target=_parent does a normal load
 PASS Check that rel=noopener with target=_top does a normal load
-PASS Check that targeting of rel=noopener with a given name reuses an existing window with that name
+FAIL Check that targeting of rel=noopener with a given name reuses an existing window with that name assert_equals: expected object "[object Window]" but got null
 PASS Check that targeting of rel=noopener with a given name reuses an existing subframe with that name

========================

I was able to reproduce the failure on tip-of-tree and on r27971 using run-webkit-tests imported/w3c/web-platform-tests/html/semantics/links/links-created-by-a-and-area-elements/htmlanchorelement_noopener.html -1 -f -iterations 500 --exit-after-n-failures 2 --exit-after-n-crashes-or-timeouts 2
 
On r279969, the test failed, but I received a different diff:

========================

--- /Volumes/Data/Builds/test-279969/layout-test-results/imported/w3c/web-platform-tests/html/semantics/links/links-created-by-a-and-area-elements/htmlanchorelement_noopener-expected.txt
+++ /Volumes/Data/Builds/test-279969/layout-test-results/imported/w3c/web-platform-tests/html/semantics/links/links-created-by-a-and-area-elements/htmlanchorelement_noopener-actual.txt
@@ -1,8 +1,10 @@
+CONSOLE MESSAGE: ReferenceError: Can't find variable: BroadcastChannel
 
+
+Harness Error (FAIL), message = ReferenceError: Can't find variable: BroadcastChannel
 
 PASS Check that rel=noopener with target=_self does a normal load
 PASS Check that rel=noopener with target=_parent does a normal load
 PASS Check that rel=noopener with target=_top does a normal load
-PASS Check that targeting of rel=noopener with a given name reuses an existing window with that name
-PASS Check that targeting of rel=noopener with a given name reuses an existing subframe with that name
+NOTRUN Check that targeting of rel=noopener with a given name reuses an existing window with that name

========================
Comment 1 Radar WebKit Bug Importer 2021-07-19 14:09:50 PDT
<rdar://problem/80801807>
Comment 2 Chris Dumez 2021-07-19 14:22:00 PDT
This test is indeed relying on BroadcastChannel. Prior to my patch that added BroadcastChannel, it was not properly running due to lack of BroadcastChannel support.

When I added support, the test started running and I landed baselines for it. However, it appears the test is flaky.

This is not really an issue with my patch but really an issue with the test (or WebKit not properly supporting what this test is checking).
Comment 3 Chris Dumez 2021-07-19 14:44:17 PDT
(In reply to Chris Dumez from comment #2)
> This test is indeed relying on BroadcastChannel. Prior to my patch that
> added BroadcastChannel, it was not properly running due to lack of
> BroadcastChannel support.
> 
> When I added support, the test started running and I landed baselines for
> it. However, it appears the test is flaky.
> 
> This is not really an issue with my patch but really an issue with the test
> (or WebKit not properly supporting what this test is checking).

w.opener is sometimes null in the message event handler. It is unclear why at this point but this seems to be a navigation with target="foo" issue, not a BroadcastChannel issue.
Comment 4 Chris Dumez 2021-07-19 15:35:59 PDT
Committed r280051 (239786@main): <https://commits.webkit.org/239786@main>
Comment 5 Chris Dumez 2021-07-19 15:43:30 PDT
Marked test as flaky on WK1 via <https://commits.webkit.org/r280051>.

This is not super high priority given that it is not really a regression, merely a test that started running as a recent of us enabling BroadcastChannel, which the test required to run.
Comment 6 Chris Dumez 2021-07-19 15:43:48 PDT
Reopening since the bug wasn't fixed.
Comment 7 ayumi_kojima 2021-09-14 08:19:43 PDT
The test is now failing on iOS and Mac wk2. (Started at around r282378)

Added expectations: https://trac.webkit.org/changeset/282388/webkit
Comment 8 Chris Dumez 2021-09-15 08:55:52 PDT
Created attachment 438248 [details]
Patch
Comment 9 Chris Dumez 2021-09-15 10:14:51 PDT
Created attachment 438252 [details]
Patch
Comment 10 Chris Dumez 2021-09-15 10:25:22 PDT
Created attachment 438256 [details]
Patch
Comment 11 Carlos Alberto Lopez Perez 2021-09-15 11:36:15 PDT Comment hidden (obsolete)
Comment 12 Carlos Alberto Lopez Perez 2021-09-15 11:39:28 PDT
This test is also flaky on GTK, with the same failure:

--- /home/clopez/webkit/webkit-flatpak/layout-test-results/imported/w3c/web-platform-tests/html/semantics/links/links-created-by-a-and-area-elements/htmlanchorelement_noopener-expected.txt
+++ /home/clopez/webkit/webkit-flatpak/layout-test-results/imported/w3c/web-platform-tests/html/semantics/links/links-created-by-a-and-area-elements/htmlanchorelement_noopener-actual.txt
@@ -3,6 +3,6 @@
 PASS Check that rel=noopener with target=_self does a normal load
 PASS Check that rel=noopener with target=_parent does a normal load
 PASS Check that rel=noopener with target=_top does a normal load
-PASS Check that targeting of rel=noopener with a given name reuses an existing window with that name
+FAIL Check that targeting of rel=noopener with a given name reuses an existing window with that name assert_equals: expected object "[object Window]" but got null
 PASS Check that targeting of rel=noopener with a given name reuses an existing subframe with that name
Comment 13 Chris Dumez 2021-09-15 13:08:19 PDT
Created attachment 438281 [details]
Patch
Comment 14 Carlos Alberto Lopez Perez 2021-09-15 14:01:21 PDT
GTK expectations updated in r282471
Comment 15 Chris Dumez 2021-09-15 14:13:26 PDT
Created attachment 438286 [details]
Patch
Comment 16 Darin Adler 2021-09-15 17:13:40 PDT
Comment on attachment 438286 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=438286&action=review

> Source/WebCore/page/DOMWindow.cpp:1062
> +            page->chrome().closeWindowSoon();

Is the "soon" still needed? Seems like the queued task is now taking care of the scheduling. Can we clean up further by just doing closeWindow; at some point?
Comment 17 Chris Dumez 2021-09-15 17:33:08 PDT
(In reply to Darin Adler from comment #16)
> Comment on attachment 438286 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=438286&action=review
> 
> > Source/WebCore/page/DOMWindow.cpp:1062
> > +            page->chrome().closeWindowSoon();
> 
> Is the "soon" still needed? Seems like the queued task is now taking care of
> the scheduling. Can we clean up further by just doing closeWindow; at some
> point?

I considered it but technically WebKit2 ends up doing an async IPC to the UIProcess so the closing does not happen synchronously and the "soon" part is not a lie.
Comment 18 Chris Dumez 2021-09-16 07:34:47 PDT
Created attachment 438349 [details]
Patch
Comment 19 EWS 2021-09-16 15:20:43 PDT
Committed r282606 (241767@main): <https://commits.webkit.org/241767@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 438349 [details].