Bug 212836 – REGRESSION: macOS WK1 tests intermittently exiting early after 500 failures

• Component: New Bugs
• Status: RESOLVED FIXED

[Ryan Haddad]

macOS WK1 tests are intermittently exiting early after 500 failures

https://build.webkit.org/builders/Apple-Catalina-Debug-WK1-Tests/builds/5060
https://build.webkit.org/builders/Apple-Catalina-Debug-WK1-Tests/builds/5053
https://build.webkit.org/builders/Apple-Catalina-Debug-WK1-Tests/builds/5038
https://build.webkit.org/builders/Apple-Catalina-Debug-WK1-Tests/builds/5035

https://build.webkit.org/builders/Apple%20Mojave%20Release%20WK1%20%28Tests%29/builds/15054
https://build.webkit.org/builders/Apple%20Mojave%20Release%20WK1%20%28Tests%29/builds/15037
https://build.webkit.org/builders/Apple%20Mojave%20Release%20WK1%20%28Tests%29/builds/15035
https://build.webkit.org/builders/Apple%20Mojave%20Release%20WK1%20%28Tests%29/builds/15029

[Radar WebKit Bug Importer]

<rdar://problem/64033778>

[Ryan Haddad]

In at least a few cases that I checked, it looks like the expected output from one test is showing up in the test that follows it, causing them all to fail:

--- /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/repaint/align-self-overflow-change-expected.txt
+++ /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/repaint/align-self-overflow-change-actual.txt
@@ -1,11 +1,9 @@
-Tests invalidation on align-self style change (just overflow). Passes if there is no red.
+Tests invalidation on align-self style change. Passes if there is no red.
 
 (repaint rects
-  (rect 0 2 200 200)
-  (rect 0 2 200 200)
-  (rect 0 52 200 200)
-  (rect 0 252 200 100)
-  (rect 0 2 200 50)
-  (rect 0 2 800 14)
+  (rect 0 302 100 300)
+  (rect 0 302 100 50)
+  (rect 0 52 100 300)
+  (rect 100 52 100 300)
 )
 

--- /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/repaint/align-self-change-expected.txt
+++ /Volumes/Data/slave/mojave-debug-tests-wk1/build/layout-test-results/fast/repaint/align-self-change-actual.txt
@@ -1,9 +1,8 @@
-Tests invalidation on align-self style change. Passes if there is no red.
+Tests invalidation on align-items style change (just overflow). Passes if there is no red.
 
 (repaint rects
-  (rect 0 302 100 300)
-  (rect 0 302 100 50)
-  (rect 0 52 100 300)
-  (rect 100 52 100 300)
+  (rect 0 2 200 350)
+  (rect 0 52 200 300)
+  (rect 0 2 800 14)
 )

[Jonathan Bedard]

Looking through history, this is definitely a recent regression:
https://results.webkit.org/suites?flavor=wk1&platform=mac

The regression is before r262527, and https://trac.webkit.org/changeset/262527/webkit is actually my best guess. If that is the commit to blame, it means that commit has simply exposed an underlying issue, not caused it, since that commit adds a bunch of new tests.

[Jonathan Bedard]

(In reply to Jonathan Bedard from comment #3)
> Looking through history, this is definitely a recent regression:
> https://results.webkit.org/suites?flavor=wk1&platform=mac
> 
> The regression is before r262527, and
> https://trac.webkit.org/changeset/262527/webkit is actually my best guess.
> If that is the commit to blame, it means that commit has simply exposed an
> underlying issue, not caused it, since that commit adds a bunch of new tests.

Although in a discussion with bot watchers, it was pointed out to me that the test added in https://trac.webkit.org/changeset/262524/webkit seems to be the last test ran in any given worker before all tests in that worker start failing. So it seems that reverting https://trac.webkit.org/changeset/262524/webkit is the best course of action.

[Jason Lawrence]

In an effort to resolve this issue, I have reverted r262524.
https://trac.webkit.org/changeset/262651/webkit

[Alexey Proskuryakov]

Hasn't happened since then, marking resolved.