Bug 196495 - [ iOS Sim ] Layout Test fast/viewport/ios/viewport-shrink-to-fit-on-new-navigation.html is a flaky failure
Summary: [ iOS Sim ] Layout Test fast/viewport/ios/viewport-shrink-to-fit-on-new-navig...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2019-04-02 08:44 PDT by Shawn Roberts
Modified: 2020-04-06 21:10 PDT (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Shawn Roberts 2019-04-02 08:44:03 PDT
The following layout test is failing on iOS Simulator Release

fast/viewport/ios/viewport-shrink-to-fit-on-new-navigation.html

Probable cause:

Test was added in https://trac.webkit.org/changeset/242069/webkit and is a flaky failure on its entire available history in dashboard. When testing locally it will either fail with a timeout, or a timeout and a text diff. Will get about 5 iterations in and start failing when testing locally.

Test failing is also affecting EWS

Reproduced with: 

run-webkit-tests fast/viewport/ios/viewport-shrink-to-fit-on-new-navigation.html --iterations 100 -f --ios-simulator --exit-after-n-failures=5

Flakiness Dashboard:

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=fast%2Fviewport%2Fios%2Fviewport-shrink-to-fit-on-new-navigation.html

Diff:

--- /Volumes/Data/slave/ios-simulator-12-release-tests-wk2/build/layout-test-results/fast/viewport/ios/viewport-shrink-to-fit-on-new-navigation-expected.txt
+++ /Volumes/Data/slave/ios-simulator-12-release-tests-wk2/build/layout-test-results/fast/viewport/ios/viewport-shrink-to-fit-on-new-navigation-actual.txt
@@ -1,6 +1,6 @@
 ALERT: pageshow - not from cache
 ALERT: pageshow - from cache
-window size: [1008, 1726]
+window size: [320, 548]
 zoom scale: 0.32
Comment 1 Shawn Roberts 2019-04-02 08:50:48 PDT
Marked flaky in https://trac.webkit.org/changeset/243742/webkit while waiting for a fix and to help EWS.
Comment 2 Radar WebKit Bug Importer 2019-04-02 13:45:13 PDT
<rdar://problem/49532510>
Comment 3 Shawn Roberts 2019-04-25 17:02:24 PDT
Test had a recent time out on iOS Simulator. While investigating realized that after marking test as a failure, changes in https://trac.webkit.org/changeset/243961/webkit have made it a constant failure.

It appears that it just needs to be re baselined because of the constant failure. Thought I don't know if r243961 would have done anything for the flakiness.
Comment 4 Russell Epstein 2019-07-08 17:10:56 PDT
This test is no longer a flaky failure, but a consistent one instead.



Diff:
--- /Volumes/Data/slave/ios-simulator-12-debug-tests-wk2/build/layout-test-results/fast/viewport/ios/viewport-shrink-to-fit-on-new-navigation-expected.txt
+++ /Volumes/Data/slave/ios-simulator-12-debug-tests-wk2/build/layout-test-results/fast/viewport/ios/viewport-shrink-to-fit-on-new-navigation-actual.txt
@@ -1,6 +1,6 @@
 ALERT: pageshow - not from cache
 ALERT: pageshow - from cache
 window size: [1008, 1726]
-zoom scale: 0.32
+zoom scale: 1.00
Comment 5 Ryan Haddad 2020-04-06 21:10:11 PDT
New results history link that shows the test is still consistently failing:
https://results.webkit.org/?suite=layout-tests&test=fast%2Fviewport%2Fios%2Fviewport-shrink-to-fit-on-new-navigation.html