Bug 217466 – [webkitpy] Allow passing subtests to GLIB API testrunner

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Diego Pino]

[webkitpy] Allow passing subtests argument to glib testrunner API

[Diego Pino]

Created attachment 410818 [details]
Patch

[Diego Pino]

Created attachment 410824 [details]
Patch

[Diego Pino]

For instance, now it's possible to run a GLIB API subtest like this:

```
$ python Tools/Scripts/run-gtk-tests --debug WebKitBuild/GTK/Debug/bin/TestWebKitAPI/WebKit2Gtk/TestUIClient -p /webkit/WebKitWebView/mouse-target
Could not start accessibility bus, so disabling TestWebKitAccessibility
TEST: WebKitBuild/GTK/Debug/bin/TestWebKitAPI/WebKit2Gtk/TestUIClient...
  /webkit/WebKitWebView/mouse-target:                                 FAIL
GLib-GIO-DEBUG: _g_io_module_get_default: Found default implementation local (GLocalVfs) for ‘gio-vfs’
ERROR:../../Tools/TestWebKitAPI/Tests/WebKitGLib/TestUIClient.cpp:817:void testWebViewMouseTarget(UIClientTest*, gconstpointer): 'webkit_hit_test_result_context_is_media(hitTestResult)' should be TRUE

Ran 1 tests of 1 with 1 successful
```

[Carlos Garcia Campos]

Comment on attachment 410824 [details]
Patch

But subtests are specific to a test program, and they are passed to all test programs to be run, no?

[Diego Pino]

(In reply to Carlos Garcia Campos from comment #4)
> Comment on attachment 410824 [details]
> Patch
> 
> But subtests are specific to a test program, and they are passed to all test
> programs to be run, no?

Yes, so in the case of launching the runner with a command like:

```
$ python Tools/Scripts/run-gtk-tests --debug -p /webkit/WebKitWebView/mouse-target
```

The runner iterates over all the tests running only the subtests passed. So if a test doesn't feature a subtest, nothing is run. For instance:

```
...
TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestDOMNode...
TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestWebKitWebContext...
TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestResources...
GLib-GObject-FATAL-CRITICAL: g_object_unref: assertion 'G_IS_OBJECT (object)' failed
TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestAuthentication...
TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestWebKitSettings...
TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestAutocleanups...
TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestAutomationSession...
GLib-DEBUG: setenv()/putenv() are not thread-safe and should not be used after threads are created
GLib-DEBUG: unsetenv() is not thread-safe and should not be used after threads are created
TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestUIClient...
  /webkit/WebKitWebView/mouse-target:                                 FAIL
GLib-GIO-DEBUG: _g_io_module_get_default: Found default implementation local (GLocalVfs) for ‘gio-vfs’
ERROR:../../Tools/TestWebKitAPI/Tests/WebKitGLib/TestUIClient.cpp:817:void testWebViewMouseTarget(UIClientTest*, gconstpointer): 'webkit_hit_test_result_context_is_media(hitTestResult)' should be TRUE
TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestLoaderClient...
TEST: /app/webkit/Tools/glib/../../WebKitBuild/Release/bin/TestWebKitAPI/WebKit2Gtk/TestDOMElement...
...
```

For me, this is still better than not being able to run subtests at all from the runner. Is there's any convenient way to figure out whether a subtest belongs to a test? (so I could skip running the test in case not).

[Diego Pino]

Perhaps an easier alternative is to require a test when one or more subtests are passed.

[Carlos Garcia Campos]

I think we already collect the subtests for the skipped tests.

[Diego Pino]

Created attachment 411102 [details]
Patch

[Diego Pino]

Created attachment 411103 [details]
Patch

[EWS]

Committed r268447: <https://trac.webkit.org/changeset/268447>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 411103 [details].

[Radar WebKit Bug Importer]

<rdar://problem/70285212>