Bug 87991 - Computed style tests are a maintenance burden on the project
Summary: Computed style tests are a maintenance burden on the project
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Tony Chang
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-05-31 10:54 PDT by Tony Chang
Modified: 2012-06-20 15:15 PDT (History)
8 users (show)

See Also:


Attachments
Patch (919.40 KB, patch)
2012-06-13 16:03 PDT, Tony Chang
no flags Details | Formatted Diff | Diff
Patch for landing (907.51 KB, patch)
2012-06-20 13:39 PDT, Tony Chang
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tony Chang 2012-05-31 10:54:01 PDT
The computed style tests enumerate the computed CSS properties.  The tests in particular are these:

fast/css/getComputedStyle/computed-style-without-renderer.html (6 results)
fast/css/getComputedStyle/computed-style.html (7 results)
svg/css/getComputedStyle-basic.xhtml (12 results)

The following seem to be problems:
1) The 3 tests are similar and require similar -expected.txt changes.
2) These tests aren't able to share results across platforms since different platforms have different features enabled.  The property-names.js file tries to work around this, but it doesn't seem to be getting much use.
3) svg/css/getComputedStyle-basic.xhtml exposes the CSS OM, which isn't well spec'ed.  It makes the test output more verbose and harder to update.

Some possible solutions:
1) Maybe we can merge computed-style-without-renderer.html and computed-style.html into a single test?  It would reduce the number of -expected files to update, although would require 2 changes to each one.  I think that's a small improvement.
2) We could do a pass over all the platforms and update property-names.js's propertiesToSkip list.  Maybe once we're down to a single expectations, people will start using the property-names.js file.  The down side is that the propertiesToSkip list would likely be large.
3) We could convert svg/css/getComputedStyle-basic.xhtml to just list property names and initial values like the fast/css tests.
Comment 1 Tony Chang 2012-06-13 16:03:30 PDT
Created attachment 147438 [details]
Patch
Comment 2 Tony Chang 2012-06-13 16:05:27 PDT
I decided to use a whitelist like we did in bug 85430 and bug 85526.
Comment 3 Ryosuke Niwa 2012-06-14 16:04:28 PDT
Comment on attachment 147438 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=147438&action=review

> LayoutTests/fast/css/getComputedStyle/resources/property-names.js:1
> +// We only test properties that are exposed in all ports.

Doesn't this mean that people need to remember to add values to this test when they add new properties? That seems undesirable. I don't think most people would remember to do that.
Comment 4 Tony Chang 2012-06-14 16:16:15 PDT
(In reply to comment #3)
> (From update of attachment 147438 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=147438&action=review
> 
> > LayoutTests/fast/css/getComputedStyle/resources/property-names.js:1
> > +// We only test properties that are exposed in all ports.
> 
> Doesn't this mean that people need to remember to add values to this test when they add new properties? That seems undesirable. I don't think most people would remember to do that.

I think anyone adding a new property will be adding a new layout test that tests the new property.  They don't have to update this test.
Comment 5 Ryosuke Niwa 2012-06-14 16:19:28 PDT
(In reply to comment #4)
> I think anyone adding a new property will be adding a new layout test that tests the new property.  They don't have to update this test.

But then when are we ever going to add a new property to this test? Never? If so, why are we arbitrarily picking a set of properties to test now?  I think this test is similar to binding tests in the sense that it just gives us the state of the world.  If we believe that all these properties are adequately tested elsewhere, maybe we should just remove it altogether.
Comment 6 Tony Chang 2012-06-14 16:31:28 PDT
(In reply to comment #5)
> (In reply to comment #4)
> > I think anyone adding a new property will be adding a new layout test that tests the new property.  They don't have to update this test.
> 
> But then when are we ever going to add a new property to this test? Never?

You can either add a new property to this test or you can write a new test. Both seem acceptable to WebKit testing practices.

> If so, why are we arbitrarily picking a set of properties to test now?  I think this test is similar to binding tests in the sense that it just gives us the state of the world. If we believe that all these properties are adequately tested elsewhere, maybe we should just remove it altogether.

Some of it is redundant with existing tests, some of it isn't. I suspect properties that were recently added don't need to be tested, but I don't know what properties those are. This maintains existing coverage, while not requiring edits to this test going forward.
Comment 7 Eric Seidel (no email) 2012-06-20 13:29:46 PDT
Comment on attachment 147438 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=147438&action=review

> LayoutTests/fast/css/getComputedStyle/computed-style-expected.txt:-45
> -font-family: Times;

Why are a few of these going away?
Comment 8 Tony Chang 2012-06-20 13:31:58 PDT
Comment on attachment 147438 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=147438&action=review

>> LayoutTests/fast/css/getComputedStyle/computed-style-expected.txt:-45
>> -font-family: Times;
> 
> Why are a few of these going away?

font-family varies on different platforms.  The other values are for features that are disabled on at least one platform.  I'm sure there are other places where we test the properties that are going away.
Comment 9 Eric Seidel (no email) 2012-06-20 13:37:50 PDT
Comment on attachment 147438 [details]
Patch

OK.  We do lose some coverage here.  But the maintinence win is likely worth it.
Comment 10 Tony Chang 2012-06-20 13:39:50 PDT
Created attachment 148645 [details]
Patch for landing
Comment 11 WebKit Review Bot 2012-06-20 15:15:33 PDT
Comment on attachment 148645 [details]
Patch for landing

Clearing flags on attachment: 148645

Committed r120870: <http://trac.webkit.org/changeset/120870>
Comment 12 WebKit Review Bot 2012-06-20 15:15:40 PDT
All reviewed patches have been landed.  Closing bug.