Bug 316712 – REGRESSION(314731@main): [GTK][WPE] New test failures when using js-test.js instead of waitUntilDone

• Status: RESOLVED
• Resolution: FIXED
• Component: WPE WebKit

[Carlos Alberto Lopez Perez]

314731@main changed the sync event to notify the tooling that the test has finished from testRunner.waitUntilDone() to js-test.js / js-test-pre.js.

This has caused this tests to fail on platforms GTK and WPE.

Regressions: Unexpected text-only failures (5)
  fast/dom/Window/open-window-min-size.html [ Failure ]
  fast/events/touch/gesture/gesture-click.html [ Failure ]
  fast/events/touch/gesture/gesture-dblclick.html [ Failure ]
  fast/events/touch/gesture/gesture-scroll.html [ Failure ]
  fast/forms/validation-message-clone.html [ Failure ]

Regressions: Unexpected timeouts (1)
  fast/events/touch/gesture/touch-gesture-scroll-sideways.html [ Timeout ]

The diffs:

--- /sdk/webkit/layout-test-results/fast/forms/validation-message-clone-expected.txt
+++ /sdk/webkit/layout-test-results/fast/forms/validation-message-clone-actual.txt
@@ -3,6 +3,10 @@
 
 
 
-PASS internals.userAgentPart(lastInOriginalShadow) == "-webkit-validation-bubble" is true
+FAIL internals.userAgentPart(lastInOriginalShadow) == "-webkit-validation-bubble" should be true. Was false.
 PASS internals.userAgentPart(lastInClonedShadow) == "-webkit-validation-bubble" is false
+PASS successfullyParsed is true
+Some tests failed.
 
+TEST COMPLETE
+
--- /sdk/webkit/layout-test-results/fast/events/touch/gesture/gesture-click-expected.txt
+++ /sdk/webkit/layout-test-results/fast/events/touch/gesture/gesture-click-actual.txt
@@ -1,10 +1,12 @@
+CONSOLE MESSAGE: TypeError: eventSender.gestureTap is not a function. (In 'eventSender.gestureTap(10, 12)', 'eventSender.gestureTap' is undefined)
 This tests basic gesture callbacks.
 
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
-Gesture manager not implemented on this platform.
-PASS successfullyParsed is true
+FAIL Unexpected error: TypeError: eventSender.gestureTap is not a function. (In 'eventSender.gestureTap(10, 12)', 'eventSender.gestureTap' is undefined)
+FAIL successfullyParsed should be true. Was false.
+Some tests failed.
 
 TEST COMPLETE
 
--- /sdk/webkit/layout-test-results/fast/events/touch/gesture/touch-gesture-scroll-sideways-expected.txt
+++ /sdk/webkit/layout-test-results/fast/events/touch/gesture/touch-gesture-scroll-sideways-actual.txt
@@ -1,19 +1,11 @@
+FAIL: Timed out waiting for notifyDone to be called
+
 This tests that precise scrolls on a horizontal scroll bar move vertically.
 
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
-first wheel event
-wheel event 0+> [object HTMLDivElement]
-PASS movingdiv.scrollTop is 19
-PASS movingdiv.scrollLeft is 0
-PASS wheelEventsOccurred is 1
-second wheel event
-wheel event 1+> [object HTMLDivElement]
-without precise deltas, scrollLeft: 0 scrollTop: 44
-PASS wheelEventsOccurred is 2
-scroll event 0+> [object HTMLDivElement]
-PASS scrollEventsOccurred is 1
+gesture events not implemented on this platform or gesture event scrolling of a document is broken
 PASS successfullyParsed is true
 
 TEST COMPLETE
--- /sdk/webkit/layout-test-results/fast/events/touch/gesture/gesture-dblclick-expected.txt
+++ /sdk/webkit/layout-test-results/fast/events/touch/gesture/gesture-dblclick-actual.txt
@@ -1,10 +1,12 @@
-This tests gesture callbacks for the double tap gesture.
+CONSOLE MESSAGE: TypeError: eventSender.gestureTap is not a function. (In 'eventSender.gestureTap(10, 12)', 'eventSender.gestureTap' is undefined)
+This tests gesture callbacks for a double tap sequence of gestures.
 
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
-Gesture manager not implemented on this platform.
-PASS successfullyParsed is true
+FAIL Unexpected error: TypeError: eventSender.gestureTap is not a function. (In 'eventSender.gestureTap(10, 12)', 'eventSender.gestureTap' is undefined)
+FAIL successfullyParsed should be true. Was false.
+Some tests failed.
 
 TEST COMPLETE
 
--- /sdk/webkit/layout-test-results/fast/events/touch/gesture/gesture-scroll-expected.txt
+++ /sdk/webkit/layout-test-results/fast/events/touch/gesture/gesture-scroll-actual.txt
@@ -1,10 +1,13 @@
+CONSOLE MESSAGE: TypeError: eventSender.gestureScrollBegin is not a function. (In 'eventSender.gestureScrollBegin(95, 12)', 'eventSender.gestureScrollBegin' is undefined)
 This tests scroll gesture events. Square is (mostly) green on pass
 
 On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
 
 
-Gesture events not implemented on this platform or broken
-PASS successfullyParsed is true
+first gesture
+FAIL Unexpected error: TypeError: eventSender.gestureScrollBegin is not a function. (In 'eventSender.gestureScrollBegin(95, 12)', 'eventSender.gestureScrollBegin' is undefined)
+FAIL successfullyParsed should be true. Was false.
+Some tests failed.
 
 TEST COMPLETE
 
--- /sdk/webkit/layout-test-results/fast/dom/Window/open-window-min-size-expected.txt
+++ /sdk/webkit/layout-test-results/fast/dom/Window/open-window-min-size-actual.txt
@@ -1,10 +1,9 @@
 CONSOLE MESSAGE: Window 'showModalDialog' function is deprecated and will be removed soon.
 CONSOLE MESSAGE: showModalDialog() is deprecated and will be removed. Please use the <dialog> element instead.
-PASS successfullyParsed is true
+CONSOLE MESSAGE: TypeError: undefined is not an object (evaluating 'msg["width"]')
+FAIL Unexpected error: TypeError: undefined is not an object (evaluating 'msg["width"]')
+FAIL successfullyParsed should be true. Was false.
+Some tests failed.
 
 TEST COMPLETE
-PASS 100 is >= 100
-PASS 100 is >= 100
-PASS 100 is >= 100
-PASS 100 is >= 100

[Alexey Proskuryakov]

Sorry! I made certain that EWSes were green.

[Carlos Alberto Lopez Perez]

(In reply to Alexey Proskuryakov from comment #1)
> Sorry! I made certain that EWSes were green.

I think in this case this failures weren't caught by the EWS because the EWS runs with "--skip-failing-tests", so that means that if a test is marked as expected to fail is not even going to be tried.

And 314731@main made some tests to change expectation (for example: fast/dom/Window/open-window-min-size.html instead of giving Timeout now gives Failure)

So that seems a weak point of our EWS: it is unable to flag patches that make a failing test to keep failing, but with different expected failure.

[Carlos Alberto Lopez Perez]

Note: fast/forms/validation-message-clone.html is skipped on GTK because "These tests test the Shadow DOM based HTML form validation UI but GTK WK2 is using native dialogs instead."
See Skip line at platform/gtk/TestExpectations that overrides the expectation line from platform/glib/TestExpectations

[Alexey Proskuryakov]

Cf. bug 248821 for EWS behavior.

[Alexey Proskuryakov]

I've posted a synthetic PR for touch and gesture events in bug 316697, it may help some of these.

[Carlos Alberto Lopez Perez]

Expectations updated at 314852@main

[Alexey Proskuryakov]

Pull request: https://github.com/WebKit/WebKit/pull/66885

[EWS]

Committed 315221@main (b0711e240419): <https://commits.webkit.org/315221@main>

Reviewed commits have been landed. Closing PR #66885 and removing active labels.