Bug 256581 – WPT variant parsing seems wrong

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Anne van Kesteren]

E.g., websockets WPT has a bunch of

    <!-- META: variant= -->
    <!-- META: variant=?wss -->

ending up as single test.

I think that's because https://github.com/WebKit/WebKit/commit/9833f67e85a7c743cd0b8fe64666f62fc07b628f didn't consider the empty string to be a valid variant.

We'd mainly end up running more tests so I'm not sure we want to fix this, but we should probably consider it at some point and document the outcome.

[Radar WebKit Bug Importer]

<rdar://problem/109450384>

[Chris Dumez]

If I look at imported/w3c/web-platform-tests/websockets/constructor/019.html for example, I see:
```
<meta name="variant" content="">
<meta name="variant" content="?wss">
<meta name="variant" content="?wpt_flags=h2">
```

When I run the tests, I see we run:
```
[235/366] imported/w3c/web-platform-tests/websockets/constructor/019.html passed
[236/366] imported/w3c/web-platform-tests/websockets/constructor/019.html?wss passed
```

So we seem to run the first 2 variants just fine. The third variant doesn't run but it is only because it is skipped:
```
LayoutTests/TestExpectations:imported/w3c/web-platform-tests/websockets/constructor/019.html?wpt_flags=h2 [ Skip ]
```

Could you clarify what the problem is? Which test are you seeing a problem with?

[Anne van Kesteren]

Sorry, I should have been more clear. The problem is with the any.js tests. E.g., let's take Close-1000-reason.any.js as an example. If you look at Close-1000-reason.any.html (which the import script ends up generating) it'll have

    <!-- META: variant= -->
    <!-- META: variant=?wss -->
    <!-- META: variant=?wpt_flags=h2 -->

and as you note we skip the last one. But we only end up running and creating an expectation file for the ?wss variant. The same is true for Close-1000-reason.any.worker.html (which the import script also generates).

[Anne van Kesteren]

Perhaps the import script should generate <meta> elements instead?

[Sam Sneddon [:gsnedders]]

This is just gonna be Tools/Scripts/webkitpy/layout_tests/controllers/layout_test_finder_legacy.py:148:

   results = re.match(r'<!--\s*META:\s*variant=([?|#]\S+)\s*-->', line)

Which clearly doesn't match the empty string.

[Sam Sneddon [:gsnedders]]

Pull request: https://github.com/WebKit/WebKit/pull/14081

[EWS]

Committed 264289@main (e17c2153ab90): <https://commits.webkit.org/264289@main>

Reviewed commits have been landed. Closing PR #14081 and removing active labels.