Bug 209735 – Import css/CSS2 from WPT repository

• Status: NEW
• Component: CSS

[Antti Koivisto]

Import css/CSS2 from WPT repository

[Sam Sneddon [:gsnedders]]

Doing this would add ~7k new tests. Some people may have concerns about additions of this scale.

[Sam Sneddon [:gsnedders]]

Let's use the See Also to track regressions we've missed as a result of _not_ having these imported, because that hopefully makes it easier to justify the value of importing so many extra tests.

[Sam Sneddon [:gsnedders]]

(In reply to Sam Sneddon [:gsnedders] from comment #1)
> Doing this would add ~7k new tests. Some people may have concerns about
> additions of this scale.

My ballpark estimate nine months ago was ~10–15% slowdown of overall test run time on the bots.

Clearly de-duplicating our old CSS2 imports would reduce the amount this is a regression, but we currently have only ~1k from these, so it'll still be a net-regression.

Additionally, our old imports are largely stored as *-expected.txt render trees, and only some of the CSS2 test suite has been converted to reftests since then — and we currently don't run visual tests for WPT tests (i.e., those we'd presumably store render trees for), so we'd potentially lose some coverage here, while gaining coverage from the more recent tests.

One possible option, though it would add thousands of new lines to TestExpectations would be to import the CSS2 test suite and simply skip the (largely low value, written for the sake of getting CSS 2.1 to REC) Microsoft-authored tests.

[Radar WebKit Bug Importer]

<rdar://problem/123388507>

[Sam Sneddon [:gsnedders]]

(In reply to Sam Sneddon [:gsnedders] from comment #3)
> One possible option, though it would add thousands of new lines to
> TestExpectations would be to import the CSS2 test suite and simply skip the
> (largely low value, written for the sake of getting CSS 2.1 to REC)
> Microsoft-authored tests.

For reference, this would import currently mean running 1947 new tests, and skipping 5181 tests. The thought of adding >5k lines to TestExpectations (currently 7.5k lines) does not fill me with joy, however.

[Sam Sneddon [:gsnedders]]

https://github.com/web-platform-tests/wpt/pull/44732 gets rid of ~1k reftests (replacing them with a single testharness.js test which takes tenths of a second to run).

Bug 269985 would probably also significantly decrease the test runtime for importing everything.

[Karl Dubost]

Sam’s PR has been merged that’s cool.