Bug 211470 - REGRESSION: (r261113): [ Mac ] http/tests/frame-throttling/raf-throttle-in-cross-origin-subframe.html is flaky failing.
Summary: REGRESSION: (r261113): [ Mac ] http/tests/frame-throttling/raf-throttle-in-cr...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Mac macOS 10.14
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-05-05 14:32 PDT by Jason Lawrence
Modified: 2020-05-05 15:59 PDT (History)
2 users (show)

See Also:


Attachments
Test list (51.05 KB, text/plain)
2020-05-05 15:23 PDT, Jason Lawrence
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Jason Lawrence 2020-05-05 14:32:54 PDT
http/tests/frame-throttling/raf-throttle-in-cross-origin-subframe.html

Description:
This test is flaky failing on Mac. The first failure appears on 05/04/2020.

History:
https://results.webkit.org/?suite=layout-tests&test=http%2Ftests%2Fframe-throttling%2Fraf-throttle-in-cross-origin-subframe.html&platform=mac&platform=ios&limit=50000

Diff:
--- /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/http/tests/frame-throttling/raf-throttle-in-cross-origin-subframe-expected.txt
+++ /Volumes/Data/slave/catalina-release-tests-wk2/build/layout-test-results/http/tests/frame-throttling/raf-throttle-in-cross-origin-subframe-actual.txt
@@ -6,16 +6,16 @@
 Received message: frameload
 Received message: frameload
 Checking that requestAnimationFrame is throttled in cross origin frame
-Received message: throttled[cross]: NonInteractiveCrossOriginFrame
-Received message: throttled[same]: [Unthrottled]
-PASS throttledState['cross'] is "NonInteractiveCrossOriginFrame"
-PASS throttledState['same'] is "[Unthrottled]"
+Received message: throttled[cross]: VisuallyIdleNonInteractiveCrossOriginFrame
+Received message: throttled[same]: VisuallyIdle
+FAIL throttledState['cross'] should be NonInteractiveCrossOriginFrame. Was VisuallyIdleNonInteractiveCrossOriginFrame.
+FAIL throttledState['same'] should be [Unthrottled]. Was VisuallyIdle.
 Interacted with cross-origin frame
 Interacted with same-origin frame
-Received message: throttled[cross]: [Unthrottled]
-Received message: throttled[same]: [Unthrottled]
-PASS throttledState['cross'] is "[Unthrottled]"
-PASS throttledState['same'] is "[Unthrottled]"
+Received message: throttled[cross]: VisuallyIdle
+Received message: throttled[same]: VisuallyIdle
+FAIL throttledState['cross'] should be [Unthrottled]. Was VisuallyIdle.
+FAIL throttledState['same'] should be [Unthrottled]. Was VisuallyIdle.
 PASS successfullyParsed is true
 
 TEST COMPLETE
Comment 1 Radar WebKit Bug Importer 2020-05-05 14:33:30 PDT
<rdar://problem/62903762>
Comment 2 Jason Lawrence 2020-05-05 15:23:51 PDT
Created attachment 398558 [details]
Test list
Comment 3 Jason Lawrence 2020-05-05 15:24:19 PDT
I can reproduce this issue with a test list on r261169 and r261113, but not r261112.
run-webkit-tests --child-processes 1 --test-list Test\ list.txt

r261169:
[1026/1026] http/tests/frame-throttling/raf-throttle-in-cross-origin-subframe.html failed unexpectedly (text diff)
1025 tests ran as expected, 1 didn't:

r261113
[685/1026] fast/shapes/shape-outside-floats/shape-outside-floats-circle-003.html failed unexpectedly (reference mismatch)
[1026/1026] http/tests/frame-throttling/raf-throttle-in-cross-origin-subframe.html failed unexpectedly (text diff)
1024 tests ran as expected, 2 didn't:

r261112:
[569/1026] fast/multicol/newmulticol/positioned-with-constrained-height.html failed unexpectedly (reference mismatch)
[1026/1026] http/tests/frame-throttling/raf-throttle-in-cross-origin-subframe.html failed unexpectedly (test timed out, text diff)
Retrying 2 unexpected failures ...
1024 tests ran as expected, 2 didn't:
Comment 4 Jason Lawrence 2020-05-05 15:59:17 PDT
I have marked this test as flaky failing while this issue is investigated.
https://trac.webkit.org/changeset/261204/webkit