Bug 251107 - [GTK][WPE] WebGL2 failures for GTK and WPE ports
Summary: [GTK][WPE] WebGL2 failures for GTK and WPE ports
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKitGTK (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
: 252905 252906 (view as bug list)
Depends on: 251080 251106 252905 252906 252909
Blocks:
  Show dependency treegraph
 
Reported: 2023-01-24 13:07 PST by Alejandro G. Castro
Modified: 2023-06-01 15:24 PDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Alejandro G. Castro 2023-01-24 13:07:24 PST
We use this bug as a reference to keep track of the failures and flakies we have marked when we enabled WebGL2 in bug 251106 and bug 251080.
Comment 1 Carlos Alberto Lopez Perez 2023-02-24 10:33:59 PST
*** Bug 252906 has been marked as a duplicate of this bug. ***
Comment 2 Carlos Alberto Lopez Perez 2023-02-24 10:34:22 PST
*** Bug 252905 has been marked as a duplicate of this bug. ***
Comment 3 Carlos Alberto Lopez Perez 2023-02-24 10:56:21 PST
On 260799@main I have updated the list of WebGL2 flaky tests based on what the post-commit bots reported since this tests were unskipped.

However, I tried to run locally the WebGL2 tests and I still get lots of unexpected failures and crashes that the bots don't report (using flatpak).

With GTK happens more than with WPE.

I'm not marking that now as I don't have more time to keep investigating this for today. Just leaving a note here about this issues.

On the GTK EWS this tests caused issues related to "persistent flakiness" where you could run the same test 11 times and the 11 times it would fail and then you rebuild and then it passes the 11 times.

So the GTK EWS WK2 bot was reporting false positives on random pull-request of unexpected failures on WebGL2 tests :(

Check the links to the EWS on bug 252905 for examples of that
Comment 4 Carlos Alberto Lopez Perez 2023-06-01 14:42:53 PDT
We continue to have issues regarding this.

Some WebGL2 tests exhibit a flakiness behavior that make the EWS give false positives.

The EWS does this:

1. Run layout tests with patch and annotate the failures 
2. Repeat each failure from 1. 10 times and annotate the ones that failed the 10 times, so not even once they passed-
3. Build WebKit without patch and run 10 times the ones that failed on 2. the 10 times.
4. Report as new failures the tests that passed 10 times without patch but failed 10 times with it.

So it is highly unlikely that a flaky tests can be a "false positive". Assuming a randomness factor of x% the odds of hitting 10 hits and 10 fails in a round are pretty low.

Yet this WebGL tests manage to do that.

I suspect it can be related to something on the graphic driver state that once activated makes it fail continuously.

Example of the issue, see the GTK bot runs on: https://github.com/WebKit/WebKit/pull/14386
Comment 5 Carlos Alberto Lopez Perez 2023-06-01 15:24:33 PDT
More WebGL2 tests were gardened on 264809@main