Bug 135921 – [CSS Shapes] Import the CSS Shapes W3C test suite

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Rebecca Hauck]

The CSS Shapes Level 1 Specification has a complete set of W3C tests that should be imported into WebKit and run regularly.

https://github.com/w3c/csswg-test/tree/master/css-shapes-1

They can be imported using Tools/Scripts/import-w3c-tests

[Rebecca Hauck]

Created attachment 236623 [details]
Patch for the bug

[Rebecca Hauck]

I have a patch submitted to run-webkit-tests in bug 135910 that needs to land before this one does. some of the tests here use svg files as support images and without that patch, they'll be treated as tests.

[Rebecca Hauck]

Created attachment 236627 [details]
updated patch with binary diff

[Bem Jones-Bey]

Comment on attachment 236627 [details]
updated patch with binary diff

You should use Failure instead of Skip in TestExpectations, since these tests are expected to fail.

You have a bunch of -expected.txt files that contain nothing but FAIL. I don't like having fail in an -expected.txt. Given that it's a pain to construct that many -expected.txt files by hand, I don't think I have a better solution, though.

[Rebecca Hauck]

Created attachment 236641 [details]
updated patch with bem's feedback

[Rebecca Hauck]

Ok, I changed [ Skip ] to [ Failure ].  I also changed the FAIL in the expected results to just PASS. They'll still fail that way, but will need be updated for the real output once they're all actually passing again. I also caught one more test that was failing that I hadn't logged a bug for - values/shape-margin-000.html. I'd accidentally lumped it with another bug, so that's updated now, too.

[Rebecca Hauck]

Created attachment 236786 [details]
fixed some test failures and re-rolled patch

I fixed the failures listed in bug 135922, 135924 and 135966 in the W3C repo, re-imported them, updated TestExpectations and re-rolled the patch.

[Rebecca Hauck]

*** Bug 135922 has been marked as a duplicate of this bug. ***

[Rebecca Hauck]

*** Bug 135924 has been marked as a duplicate of this bug. ***

[Rebecca Hauck]

*** Bug 135966 has been marked as a duplicate of this bug. ***

[Bem Jones-Bey]

Comment on attachment 236786 [details]
fixed some test failures and re-rolled patch

View in context: https://bugs.webkit.org/attachment.cgi?id=236786&action=review

r=me, once LayoutTests/css3/shapes/shape-outside-invalid-ellipse-004.html is fixed to pass.

> LayoutTests/ChangeLog:8
> +        This patch contains the imported CSS Shapes W3C test suite

Nit: s/imported//

[Rebecca Hauck]

Created attachment 236789 [details]
updated expectations with ImageFailure, fixed invalid test, fixed description

[WebKit Commit Bot]

Comment on attachment 236789 [details]
updated expectations with ImageFailure, fixed invalid test, fixed description

Clearing flags on attachment: 236789

Committed r172733: <http://trac.webkit.org/changeset/172733>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Benjamin Poulain]

The test css3/shapes/shape-outside/values/shape-image-threshold-001.html has been continuously crashing in debug due to an assertion.

[Bem Jones-Bey]

(In reply to comment #15)
> The test css3/shapes/shape-outside/values/shape-image-threshold-001.html has been continuously crashing in debug due to an assertion.

My bad for not ensuring that we ran the tests in Debug mode before landing. I'm looking into it now, it's bug 135926.