Bug 135921

Summary: [CSS Shapes] Import the CSS Shapes W3C test suite
Product: WebKit Reporter: Rebecca Hauck <rhauck>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: benjamin, bjonesbe, commit-queue
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on: 135910, 136221    
Bug Blocks: 136075    
Attachments:
Description Flags
Patch for the bug
none
updated patch with binary diff
none
updated patch with bem's feedback
none
fixed some test failures and re-rolled patch
bjonesbe: review+, bjonesbe: commit-queue-
updated expectations with ImageFailure, fixed invalid test, fixed description none

Description Rebecca Hauck 2014-08-13 18:55:24 PDT
The CSS Shapes Level 1 Specification has a complete set of W3C tests that should be imported into WebKit and run regularly.

https://github.com/w3c/csswg-test/tree/master/css-shapes-1

They can be imported using Tools/Scripts/import-w3c-tests
Comment 1 Rebecca Hauck 2014-08-14 14:55:57 PDT
Created attachment 236623 [details]
Patch for the bug
Comment 2 Rebecca Hauck 2014-08-14 15:11:09 PDT
I have a patch submitted to run-webkit-tests in bug 135910 that needs to land before this one does. some of the tests here use svg files as support images and without that patch, they'll be treated as tests.
Comment 3 Rebecca Hauck 2014-08-14 15:21:33 PDT
Created attachment 236627 [details]
updated patch with binary diff
Comment 4 Bem Jones-Bey 2014-08-14 15:47:00 PDT
Comment on attachment 236627 [details]
updated patch with binary diff

You should use Failure instead of Skip in TestExpectations, since these tests are expected to fail.

You have a bunch of -expected.txt files that contain nothing but FAIL. I don't like having fail in an -expected.txt. Given that it's a pain to construct that many -expected.txt files by hand, I don't think I have a better solution, though.
Comment 5 Rebecca Hauck 2014-08-14 18:39:38 PDT
Created attachment 236641 [details]
updated patch with bem's feedback
Comment 6 Rebecca Hauck 2014-08-14 18:41:53 PDT
Ok, I changed [ Skip ] to [ Failure ].  I also changed the FAIL in the expected results to just PASS. They'll still fail that way, but will need be updated for the real output once they're all actually passing again. I also caught one more test that was failing that I hadn't logged a bug for - values/shape-margin-000.html. I'd accidentally lumped it with another bug, so that's updated now, too.
Comment 7 Rebecca Hauck 2014-08-18 14:37:10 PDT
Created attachment 236786 [details]
fixed some test failures and re-rolled patch

I fixed the failures listed in bug 135922, 135924 and 135966 in the W3C repo, re-imported them, updated TestExpectations and re-rolled the patch.
Comment 8 Rebecca Hauck 2014-08-18 14:38:44 PDT
*** Bug 135922 has been marked as a duplicate of this bug. ***
Comment 9 Rebecca Hauck 2014-08-18 14:39:12 PDT
*** Bug 135924 has been marked as a duplicate of this bug. ***
Comment 10 Rebecca Hauck 2014-08-18 14:40:15 PDT
*** Bug 135966 has been marked as a duplicate of this bug. ***
Comment 11 Bem Jones-Bey 2014-08-18 14:49:21 PDT
Comment on attachment 236786 [details]
fixed some test failures and re-rolled patch

View in context: https://bugs.webkit.org/attachment.cgi?id=236786&action=review

r=me, once LayoutTests/css3/shapes/shape-outside-invalid-ellipse-004.html is fixed to pass.

> LayoutTests/ChangeLog:8
> +        This patch contains the imported CSS Shapes W3C test suite

Nit: s/imported//
Comment 12 Rebecca Hauck 2014-08-18 15:19:33 PDT
Created attachment 236789 [details]
updated expectations with ImageFailure, fixed invalid test, fixed description
Comment 13 WebKit Commit Bot 2014-08-18 16:10:26 PDT
Comment on attachment 236789 [details]
updated expectations with ImageFailure, fixed invalid test, fixed description

Clearing flags on attachment: 236789

Committed r172733: <http://trac.webkit.org/changeset/172733>
Comment 14 WebKit Commit Bot 2014-08-18 16:10:30 PDT
All reviewed patches have been landed.  Closing bug.
Comment 15 Benjamin Poulain 2014-08-18 19:41:40 PDT
The test css3/shapes/shape-outside/values/shape-image-threshold-001.html has been continuously crashing in debug due to an assertion.
Comment 16 Bem Jones-Bey 2014-08-19 09:46:36 PDT
(In reply to comment #15)
> The test css3/shapes/shape-outside/values/shape-image-threshold-001.html has been continuously crashing in debug due to an assertion.

My bad for not ensuring that we ran the tests in Debug mode before landing. I'm looking into it now, it's bug 135926.