Bug 301508 – [WebXR][GLib] TestWebKitWebXR/permission-request is flaky

• Component: WebKitGTK
• Status: RESOLVED FIXED

[Fujii Hironori]

[WebXR][GLib] TestWebKitWebXR/permission-request is flaky

https://build.webkit.org/#/builders/57/builds/21883

>   /webkit/WebKitWebXR/permission-request:                             TIMEOUT
> ERROR:../../../Tools/TestWebKitAPI/Tests/WebKitGLib/TestWebKitWebXR.cpp:246:void testWebKitXRPermissionRequest(WebXRTest*, gconstpointer): assertion failed (data.result.title.utf8().data() == "pass"): ("" == "pass")

[Fujii Hironori]

another failure. line#257 failed.

https://s3-us-west-2.amazonaws.com/ews-archives.webkit.org/wpe-x86_64-release-re-run-api-tests/fd5138dc-78208.txt

>   /webkit/WebKitWebXR/permission-request:                             TIMEOUT
> ERROR:../../../Tools/TestWebKitAPI/Tests/WebKitGLib/TestWebKitWebXR.cpp:257:void testWebKitXRPermissionRequest(WebXRTest*, gconstpointer): assertion failed (data.result.title.utf8().data() == "fail"): ("" == "fail")

[Simon Pena]

For API tests, do we have a history view to see how often something is flaky? 

At the moment, we stop the monado-service at the end of the tests, but I am not sure we do anything if it we find it already gone by the time we stop it. We could start by updating the test runner, and checking whether it's still running, otherwise printing an error. It would help narrow things down.

[Fujii Hironori]

GTK and WPE are using run-gtk-tests and run-wpe-tests scripts which don't have the reporing feature.
I don't know what's going on in the bots. But, I suppose we will see the log like the following if the monado service isn't running.

> XR_ERROR_RUNTIME_FAILURE in xrCreateInstance: Failed to create instance '-1'

[Fujii Hironori]

Ah. stderr is redirected to os.devnull.
https://github.com/WebKit/WebKit/blob/ea52e1d0df0f41d9cce8b9d08e6f0efaf95d2d52/Tools/Scripts/webkitpy/port/monadodriver.py#L68-L69

[Claudio Saavedra]

*** Bug 301571 has been marked as a duplicate of this bug. ***

[Simon Pena]

I've been debugging this locally. While the bots are flaky and pass, file or time out, when I run this locally I get it failing consistently. 

In my setup, building from tip of the tree, attempting to request a session with unbounded required features fails with `NotSupportedError: The operation is not supported.`. In that case, our permission request callback doesn't get called. I am updating my SDK, because my monado-service was the package provided, and want to see what happens when I use the jhbuild one.

[Simon Pena]

Indeed, with the latest webkit-container-sdk and jhbuild-provided monado-service, the test passes locally.

[Fujii Hironori]

This is reproducible on my PC by limiting CPU affinity with taskset. 

> WITH_OPENXR_RUNTIME=y taskset 1 ./Tools/Scripts/run-gtk-tests --release WebKitGTK/TestWebKitWebXR

But, not 100%. 1/20

[Fujii Hironori]

Pull request: https://github.com/WebKit/WebKit/pull/53123

[EWS]

Committed 302292@main (4eef88235757): <https://commits.webkit.org/302292@main>

Reviewed commits have been landed. Closing PR #53123 and removing active labels.