Bug 31311 – Fix the wrong css2.1/t040103-ident-03-c.html test

• Status: NEW
• Component: Tools / Tests

[Victor Wang]

LayoutTests/css2.1/t040103-ident-03-c.html was wrong, the test and its baselines need to be fixed.

[Victor Wang]

Created attachment 42892 [details]
Proposed Patch

[Alexey Proskuryakov]

> LayoutTests/css2.1/t040103-ident-03-c.html was wrong

Why? Please document the reasons here for later reference.

This is an official test from W3C, has a fix been made upstream? Or is there a bug filed in W3C bugzilla?

[Victor Wang]

Could you point me where this test is copied from in W3C? I don't know this test is from W3C so not sure whether it is fixed upstream. I think this test is wrong so it should be fixed.

[Alexey Proskuryakov]

Googling for "t040103-ident-03-c" gives a number of hits, including a discussion of its correctness in <http://lists.w3.org/Archives/Public/www-style/2007Aug/0128.html>. 

A version of this test in <http://www.w3.org/Style/CSS/Test/Print/1.0/20070802/t040103-ident-03-c.htm> seems to have the change incorporated, but I don't know if it's current. <http://www.w3.org/Style/CSS/Test/CSS2.1/current/> links to a seriously refactored version of the test suite, where this test evolved into <http://www.w3.org/Style/CSS/Test/CSS2.1/current/html4/ident-003.htm>.

You need to check with someone involved with the CSS 2.1 working group for the exact status of their test suite. Maybe we should just replace the whole css2.1 directory with a more current version.

[Eric Seidel (no email)]

Comment on attachment 42892 [details]
Proposed Patch

We've historically not modified these files directly.  Given ap's comments above, marking this as r-.  I agree, we should probably just update our copy of css2.1

[Victor Wang]

Good to know, Thanks ap and eric for your info!

[Shinichiro Hamaji]

Are someone working to update our copy of CSS2.1 test suite? If not, I'm happy to look into this.

[Victor Wang]

Feel free to take this if you have a chance.

[Shinichiro Hamaji]

It seems W3C released a new alpha version of test suite.

http://www.w3.org/Style/CSS/Test/CSS2.1/20100127/html4/index.htm

I'm not sure if we want to just update css2.1 directory because the number of test cases significantly increased (from ~400 to ~8200).

If we want to avoid increasing the test running time, one idea I came up with is

- update existing tests with the new release,
- add new tests and their expected results into somewhere different directory, and
- modify run-webkit-tests not to run the new tests by default

Any suggestions will be appreciated, thanks!

[Joseph Pecoraro]

*** Bug 23269 has been marked as a duplicate of this bug. ***

[Joseph Pecoraro]

Hmm, for a few of the LayoutTests/css2.1 tests where I've seen issues
I've opened up the source and sees a comment that says why the test
is wrong [1]. Could we such a comment to the test? We could open a
separate bug for updating the entire suite.

There is also test.html.disabled if we wanted to skip it, but I'm
all for leaving failing "apparent" failing results checked in.

[1]: http://trac.webkit.org/browser/trunk/LayoutTests/css2.1/t1504-c543-txt-decor-00-d-g.html#L28

[Csaba Osztrogonác]

This test is still wrong and expected png is committed for all 
ports. It is annoying a little bit that I see "This should be 
green" in red as expected. :)

Is there any will to fix it in the future?