284916 2024-12-18 16:32:13 -0800 [Tools][GTK][WPE] MVT test suite: integrate run-mvt-tests with webkitpy facilities and support test expectations 2025-01-16 12:35:07 -0800 1 1 1 Unclassified WebKit WPE WebKit WebKit Nightly Build Unspecified Unspecified RESOLVED FIXED https://bugs.webkit.org/show_bug.cgi?id=273679 https://bugs.webkit.org/show_bug.cgi?id=286089 P2 Normal --- 1 clopez clopez bugs-noreply dpino lmoura oldest_to_newest 2082837 0 clopez 2024-12-18 16:32:13 -0800 I have been evaluating the script run-mvt-tests in order to add a new step to the tests bots and run it. However I found several issues with it that I would like to fix before adding the step: - It does not support running with the system libraries (only flatpak) and the JHBuild support is broken, it is not working for me. This is a requirement for running it on the downstream CI and also for the upcoming Flatpak SDK - It is manually defining environment variables like WEBKIT_EXEC_PATH or WEBKIT_INJECTED_BUNDLE_PATH instead of relying on webkitpy for that. - It does not support WebKitGTK (not a goal, but seems a nice perk to compare results between ports) - It does not support running with the WPE MiniBrowser instead of Cog - It does not support running the complete MVT tests in a round (all the test suites). You have to manually specify which test-suite of MVT you want to run - It does not support test expectations. So it is not useful to run on the bots as it is because it will be failing always if we don't have the ability to mark expected failures - The logging it does is a bit confusing. It is not clear exactly which tests are failing because it prints the verbose log rather than the name of the tests that fail. 2082875 1 clopez 2024-12-18 18:33:16 -0800 Pull request: https://github.com/WebKit/WebKit/pull/38180 2087039 2 ews-feeder 2025-01-14 06:06:24 -0800 Committed 288842@main (566fb370f6b3): <https://commits.webkit.org/288842@main> Reviewed commits have been landed. Closing PR #38180 and removing active labels.