Bug 187353 – [webkitpy] run-web-platform-tests should allow specifying custom WPT metadata directories

• Component: New Bugs
• Status: RESOLVED FIXED

[Zan Dobersek]

[webkitpy] run-web-platform-tests should allow specifying custom WPT metadata directories

[Zan Dobersek]

Created attachment 344351 [details]
Patch

[Zan Dobersek]

Comment on attachment 344351 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=344351&action=review

> Tools/ChangeLog:12
> +        When using run-web-platform-tests, allow specifying custom WPT metadata
> +        directory. This will avoid generating such metadata-filled directory
> +        from the port-specific JSON expectations file, and instead will search
> +        for the metadata files and the include manifest inside the specified
> +        directory. The MANIFEST.json file will also be generated under there.

For reference, this is the metadata repository equivalent to the current JSON expectations file for the GTK+ port:
https://github.com/zdobersek/webkit-wpt-metadata

[EWS Watchlist]

Comment on attachment 344351 [details]
Patch

Attachment 344351 [details] did not pass win-ews (win):
Output: https://webkit-queues.webkit.org/results/8448173

New failing tests:
http/tests/preload/onload_event.html
http/tests/security/video-poster-cross-origin-crash2.html

[EWS Watchlist]

Created attachment 344363 [details]
Archive of layout-test-results from ews206 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews206  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit

[Zan Dobersek]

Ping, reviewers?

[youenn fablet]

Comment on attachment 344351 [details]
Patch

LGTM.
Generating this manifest.json takes time IIRC so we might think of the best way to cache/checkin/update this file.

View in context: https://bugs.webkit.org/attachment.cgi?id=344351&action=review

> Tools/Scripts/webkitpy/w3c/wpt_runner.py:200
> +                _log.error("Existent wpt metadata directory has to be specified")

s/Existent/Existing/
This test could probably be done in WPTRunner __init__

> Tools/Scripts/webkitpy/w3c/wpt_runner.py:203
> +            include_manifest_path = self._host.filesystem.join(metadata_path, self._port.name(), "TestManifest.ini")

Maybe add a helper routine to create the manifest path from a root metatada path?

[Zan Dobersek]

Comment on attachment 344351 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=344351&action=review

>> Tools/Scripts/webkitpy/w3c/wpt_runner.py:200
>> +                _log.error("Existent wpt metadata directory has to be specified")
> 
> s/Existent/Existing/
> This test could probably be done in WPTRunner __init__

It could be done, but it would still have to be replicated here just to enforce the early False return.

>> Tools/Scripts/webkitpy/w3c/wpt_runner.py:203
>> +            include_manifest_path = self._host.filesystem.join(metadata_path, self._port.name(), "TestManifest.ini")
> 
> Maybe add a helper routine to create the manifest path from a root metatada path?

OK.

[Zan Dobersek]

(In reply to youenn fablet from comment #6)
> Comment on attachment 344351 [details]
> Patch
> 
> LGTM.
> Generating this manifest.json takes time IIRC so we might think of the best
> way to cache/checkin/update this file.
> 

It's time-expensive the first time, but it only gets updated afterwards, which is fast enough IMO.

[Zan Dobersek]

Created attachment 345080 [details]
Patch for landing

[Zan Dobersek]

Comment on attachment 345080 [details]
Patch for landing

Clearing flags on attachment: 345080

Committed r233843: <https://trac.webkit.org/changeset/233843>

[Zan Dobersek]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/42228944>