Bug 169196 - [css-grid] Import W3C Test Suite
Summary: [css-grid] Import W3C Test Suite
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Manuel Rego Casasnovas
URL:
Keywords:
Depends on:
Blocks: 60731
  Show dependency treegraph
 
Reported: 2017-03-06 05:03 PST by Manuel Rego Casasnovas
Modified: 2017-03-06 11:09 PST (History)
6 users (show)

See Also:


Attachments
Patch (676.20 KB, patch)
2017-03-06 07:48 PST, Manuel Rego Casasnovas
no flags Details | Formatted Diff | Diff
Current vs expected output for css-grid-1/grid-items/grid-items-sizing-alignment-001.html (39.84 KB, image/png)
2017-03-06 07:55 PST, Manuel Rego Casasnovas
no flags Details
Patch for landing (676.91 KB, patch)
2017-03-06 08:56 PST, Manuel Rego Casasnovas
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Manuel Rego Casasnovas 2017-03-06 05:03:13 PST
Blink has already imported the W3C Test Suite and it'd be nice to do the same for WebKit.
That way we could very easily share tests between both implementations.

The suite is at csswg-test repository: https://github.com/w3c/csswg-test/tree/master/css-grid-1
Comment 1 Manuel Rego Casasnovas 2017-03-06 05:03:27 PST
I've started to work on this, but first I need to do a small change on the test suite:
https://github.com/w3c/csswg-test/pull/1229
Comment 2 Manuel Rego Casasnovas 2017-03-06 07:48:40 PST
Created attachment 303512 [details]
Patch
Comment 3 Manuel Rego Casasnovas 2017-03-06 07:51:35 PST
So I've imported the test suite and uploaded the patch.

Most of the tests are passing but there are a few failures for different reasons:
* CSSOM serialization of grid shorthands (bug #149891)
* min-content and max-content prefixes (bug #169195)
* "normal" => "start" for replaced grid items (bug #169198)
* Transferred size to calculate minimum size of grid items (bug #169197)

As you can see there are bugs reported for each of them,
so once they're solved the tests can be enabled on LayoutTestExpectations.
Comment 4 Manuel Rego Casasnovas 2017-03-06 07:55:49 PST
Created attachment 303513 [details]
Current vs expected output for css-grid-1/grid-items/grid-items-sizing-alignment-001.html

Note that some tests return the following result (at least on Linux):
  imported/w3c/csswg-test/css-grid-1/grid-items/grid-items-sizing-alignment-001.html -> ref test hashes didn't match but diff passed 

This is wrong as you can see in the attached image, the test is clearly failing.
That's the reason why I'm marking them as Failures on the TestExpectation file.
Comment 5 youenn fablet 2017-03-06 08:34:35 PST
Comment on attachment 303512 [details]
Patch

Great to see new tests being imported!

Would you also consider updating LayoutTests/imported/w3c/resources/import-expectations.json?
Basically change "csswg-test/css-grid-1"from "skip"to "import"
Comment 6 Manuel Rego Casasnovas 2017-03-06 08:56:23 PST
Created attachment 303515 [details]
Patch for landing
Comment 7 Manuel Rego Casasnovas 2017-03-06 09:02:15 PST
(In reply to comment #5)
> Comment on attachment 303512 [details]
> Patch
> 
> Great to see new tests being imported!
> 
> Would you also consider updating
> LayoutTests/imported/w3c/resources/import-expectations.json?
> Basically change "csswg-test/css-grid-1"from "skip"to "import"

Sure, I didn't realize about that file.

Thanks for the review.
Comment 8 WebKit Commit Bot 2017-03-06 09:58:50 PST
The commit-queue encountered the following flaky tests while processing attachment 303515 [details]:

media/modern-media-controls/tracks-support/tracks-support-show-panel-after-dragging-controls.html bug 169158 (authors: graouts@apple.com and ryanhaddad@apple.com)
The commit-queue is continuing to process your patch.
Comment 9 WebKit Commit Bot 2017-03-06 10:04:48 PST
Comment on attachment 303515 [details]
Patch for landing

Rejecting attachment 303515 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-01', 'land-attachment', '--force-clean', '--non-interactive', '--parent-command=commit-queue', 303515, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Last 500 characters of output:
ilding .git/svn/refs/remotes/origin/master/.rev_map.268f45cc-cd09-0410-ab3c-d52691b4dbfc ...
Currently at 213452 = 953b0c214c392fbaebbe95de990f8e6884beab67
r213453 = 205886d5a99e99fb1e82259935b3d4ba39bf49c7
r213454 = 067c23f34156a76e5b625b9696c146943bc19252
Done rebuilding .git/svn/refs/remotes/origin/master/.rev_map.268f45cc-cd09-0410-ab3c-d52691b4dbfc
First, rewinding head to replay your work on top of it...
Fast-forwarded master to refs/remotes/origin/master.
Total errors found: 0 in 1 files

Full output: http://webkit-queues.webkit.org/results/3253311
Comment 10 WebKit Commit Bot 2017-03-06 11:09:50 PST
Comment on attachment 303515 [details]
Patch for landing

Clearing flags on attachment: 303515

Committed r213457: <http://trac.webkit.org/changeset/213457>
Comment 11 WebKit Commit Bot 2017-03-06 11:09:55 PST
All reviewed patches have been landed.  Closing bug.