Bug 209735 - Import css/CSS2 from WPT repository
Summary: Import css/CSS2 from WPT repository
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: CSS (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2020-03-30 05:23 PDT by Antti Koivisto
Modified: 2024-02-23 05:38 PST (History)
8 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Antti Koivisto 2020-03-30 05:23:44 PDT
Import css/CSS2 from WPT repository
Comment 1 Sam Sneddon [:gsnedders] 2023-06-06 17:35:08 PDT
Doing this would add ~7k new tests. Some people may have concerns about additions of this scale.
Comment 2 Sam Sneddon [:gsnedders] 2023-10-30 09:56:28 PDT
Let's use the See Also to track regressions we've missed as a result of _not_ having these imported, because that hopefully makes it easier to justify the value of importing so many extra tests.
Comment 3 Sam Sneddon [:gsnedders] 2024-02-21 10:15:24 PST
(In reply to Sam Sneddon [:gsnedders] from comment #1)
> Doing this would add ~7k new tests. Some people may have concerns about
> additions of this scale.

My ballpark estimate nine months ago was ~10–15% slowdown of overall test run time on the bots.

Clearly de-duplicating our old CSS2 imports would reduce the amount this is a regression, but we currently have only ~1k from these, so it'll still be a net-regression.

Additionally, our old imports are largely stored as *-expected.txt render trees, and only some of the CSS2 test suite has been converted to reftests since then — and we currently don't run visual tests for WPT tests (i.e., those we'd presumably store render trees for), so we'd potentially lose some coverage here, while gaining coverage from the more recent tests.

One possible option, though it would add thousands of new lines to TestExpectations would be to import the CSS2 test suite and simply skip the (largely low value, written for the sake of getting CSS 2.1 to REC) Microsoft-authored tests.
Comment 4 Radar WebKit Bug Importer 2024-02-21 10:15:49 PST
<rdar://problem/123388507>
Comment 5 Sam Sneddon [:gsnedders] 2024-02-21 12:12:46 PST
(In reply to Sam Sneddon [:gsnedders] from comment #3)
> One possible option, though it would add thousands of new lines to
> TestExpectations would be to import the CSS2 test suite and simply skip the
> (largely low value, written for the sake of getting CSS 2.1 to REC)
> Microsoft-authored tests.

For reference, this would import currently mean running 1947 new tests, and skipping 5181 tests. The thought of adding >5k lines to TestExpectations (currently 7.5k lines) does not fill me with joy, however.
Comment 6 Sam Sneddon [:gsnedders] 2024-02-23 05:38:36 PST
https://github.com/web-platform-tests/wpt/pull/44732 gets rid of ~1k reftests (replacing them with a single testharness.js test which takes tenths of a second to run).

Bug 269985 would probably also significantly decrease the test runtime for importing everything.