Bug 237552 - [iOS Mac ] imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/same-hash.html is a flaky text failure
Summary: [iOS Mac ] imported/w3c/web-platform-tests/html/browsers/history/the-location...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Karl Rackler
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2022-03-07 13:29 PST by Dawn Morningstar
Modified: 2022-05-04 13:37 PDT (History)
7 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dawn Morningstar 2022-03-07 13:29:21 PST
imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/same-hash.html 
Is a flaky text failure on iOS and wk1 queues since the test was introduced.

HISTORY:
https://results.webkit.org/?suite=layout-tests&test=imported%2Fw3c%2Fweb-platform-tests%2Fhtml%2Fbrowsers%2Fhistory%2Fthe-location-interface%2Fsame-hash.html

DIFF:
--- /Volumes/Data/worker/ios-simulator-15-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/same-hash-expected.txt
+++ /Volumes/Data/worker/ios-simulator-15-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/same-hash-actual.txt
@@ -1,6 +1,6 @@
 
 
-PASS Using location.hash = "#te<st" must not reset scroll position
+FAIL Using location.hash = "#te<st" must not reset scroll position assert_greater_than: First hash assignment scrolls the iframe expected a number greater than 150 but got 0
 PASS Using location.hash = "te<st" must not reset scroll position
 PASS Using location.hash = "#te%3Cst" must not reset scroll position
 PASS Using location.hash = "te%3Cst" must not reset scroll position

DIFF-URL:
https://build.webkit.org/results/Apple-iOS-15-Simulator-Debug-WK2-Tests/r290885%20(1859)/imported/w3c/web-platform-tests/html/browsers/history/the-location-interface/same-hash-diff.txt
Comment 1 Radar WebKit Bug Importer 2022-03-07 13:30:34 PST
<rdar://problem/89926580>
Comment 2 Dawn Morningstar 2022-03-15 12:33:36 PDT
Correction, this is not solely happening on iOS queues.
Comment 3 Dawn Morningstar 2022-03-15 12:42:07 PDT
Marking expectations until further investigation can occur.
Comment 4 Chris Dumez 2022-03-15 12:44:43 PDT
Not truly a regression, just a WPT test update from upstream.

Looking at the output, this likely has to do with some scrolling happening asynchronously when the test expects it not too. As a result, when the test checks the scroll position, sometimes it has scrolled and sometimes it hasn't.
Comment 5 Dawn Morningstar 2022-03-15 12:53:30 PDT
(In reply to Chris Dumez from comment #4)
> Not truly a regression, just a WPT test update from upstream.
> 
> Looking at the output, this likely has to do with some scrolling happening
> asynchronously when the test expects it not too. As a result, when the test
> checks the scroll position, sometimes it has scrolled and sometimes it
> hasn't.

Ah I see, incorrect starting position, this would mostly be a test order issue then? Or is modifying the test to reset starting position possible?
Comment 6 Chris Dumez 2022-03-15 12:57:31 PDT
(In reply to Matteo Flores from comment #5)
> (In reply to Chris Dumez from comment #4)
> > Not truly a regression, just a WPT test update from upstream.
> > 
> > Looking at the output, this likely has to do with some scrolling happening
> > asynchronously when the test expects it not too. As a result, when the test
> > checks the scroll position, sometimes it has scrolled and sometimes it
> > hasn't.
> 
> Ah I see, incorrect starting position, this would mostly be a test order
> issue then? Or is modifying the test to reset starting position possible?

Either the test needs to be modified to not wait a little bit for the scrolling position to get updated (which would mean an upstream PR since this is an imported W3C/WPT test, not a WebKit test). Or we really do have a bug and the scroll position is supposed to get updated synchronously here (which would be a WebKit bug but likely not a recent regression).
Comment 7 Karl Rackler 2022-05-04 13:30:38 PDT
I have marked this test as a flaky failure while this issue is investigated.
Comment 8 Karl Rackler 2022-05-04 13:34:32 PDT
Pull request: https://github.com/WebKit/WebKit/pull/501
Comment 9 EWS 2022-05-04 13:37:28 PDT
Test gardening commit r293787 (250266@main): <https://commits.webkit.org/250266@main>

Reviewed commits have been landed. Closing PR #501 and removing active labels.