[webkitpy] Allow passing subtests argument to glib testrunner API
Created attachment 410818 [details] Patch
Created attachment 410824 [details] Patch
For instance, now it's possible to run a GLIB API subtest like this: ``` $ python Tools/Scripts/run-gtk-tests --debug WebKitBuild/GTK/Debug/bin/TestWebKitAPI/WebKit2Gtk/TestUIClient -p /webkit/WebKitWebView/mouse-target Could not start accessibility bus, so disabling TestWebKitAccessibility TEST: WebKitBuild/GTK/Debug/bin/TestWebKitAPI/WebKit2Gtk/TestUIClient... /webkit/WebKitWebView/mouse-target: FAIL GLib-GIO-DEBUG: _g_io_module_get_default: Found default implementation local (GLocalVfs) for ‘gio-vfs’ ERROR:../../Tools/TestWebKitAPI/Tests/WebKitGLib/TestUIClient.cpp:817:void testWebViewMouseTarget(UIClientTest*, gconstpointer): 'webkit_hit_test_result_context_is_media(hitTestResult)' should be TRUE Ran 1 tests of 1 with 1 successful ```
Comment on attachment 410824 [details] Patch But subtests are specific to a test program, and they are passed to all test programs to be run, no?
(In reply to Carlos Garcia Campos from comment #4) > Comment on attachment 410824 [details] > Patch > > But subtests are specific to a test program, and they are passed to all test > programs to be run, no? Yes, so in the case of launching the runner with a command like: ``` $ python Tools/Scripts/run-gtk-tests --debug -p /webkit/WebKitWebView/mouse-target ``` The runner iterates over all the tests running only the subtests passed. So if a test doesn't feature a subtest, nothing is run. For instance: ``` ... TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestDOMNode... TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestWebKitWebContext... TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestResources... GLib-GObject-FATAL-CRITICAL: g_object_unref: assertion 'G_IS_OBJECT (object)' failed TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestAuthentication... TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestWebKitSettings... TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestAutocleanups... TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestAutomationSession... GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used after threads are created GLib-DEBUG: unsetenv() is not thread-safe and should not be used after threads are created TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestUIClient... /webkit/WebKitWebView/mouse-target: FAIL GLib-GIO-DEBUG: _g_io_module_get_default: Found default implementation local (GLocalVfs) for ‘gio-vfs’ ERROR:../../Tools/TestWebKitAPI/Tests/WebKitGLib/TestUIClient.cpp:817:void testWebViewMouseTarget(UIClientTest*, gconstpointer): 'webkit_hit_test_result_context_is_media(hitTestResult)' should be TRUE TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestLoaderClient... TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestDOMElement... ... ``` For me, this is still better than not being able to run subtests at all from the runner. Is there's any convenient way to figure out whether a subtest belongs to a test? (so I could skip running the test in case not).
Perhaps an easier alternative is to require a test when one or more subtests are passed.
I think we already collect the subtests for the skipped tests.
Created attachment 411102 [details] Patch
Created attachment 411103 [details] Patch
Committed r268447: <https://trac.webkit.org/changeset/268447> All reviewed patches have been landed. Closing bug and clearing flags on attachment 411103 [details].
<rdar://problem/70285212>