Bug 193273 - Import css-lists testcases from web platform tests
Summary: Import css-lists testcases from web platform tests
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: CSS (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks: 192980
  Show dependency treegraph
 
Reported: 2019-01-08 23:44 PST by cathiechen
Modified: 2019-01-10 02:03 PST (History)
7 users (show)

See Also:


Attachments
Patch (75.88 KB, patch)
2019-01-08 23:57 PST, cathiechen
no flags Details | Formatted Diff | Diff
Patch (74.20 KB, patch)
2019-01-09 01:22 PST, cathiechen
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews102 for mac-sierra (2.65 MB, application/zip)
2019-01-09 02:25 PST, Build Bot
no flags Details
Archive of layout-test-results from ews106 for mac-sierra-wk2 (3.21 MB, application/zip)
2019-01-09 02:38 PST, Build Bot
no flags Details
Archive of layout-test-results from ews115 for mac-sierra (2.36 MB, application/zip)
2019-01-09 02:55 PST, Build Bot
no flags Details
Archive of layout-test-results from ews122 for ios-simulator-wk2 (3.90 MB, application/zip)
2019-01-09 03:35 PST, Build Bot
no flags Details
Patch (79.12 KB, patch)
2019-01-09 04:53 PST, cathiechen
no flags Details | Formatted Diff | Diff
Patch (79.02 KB, patch)
2019-01-09 05:10 PST, cathiechen
no flags Details | Formatted Diff | Diff
Patch (79.04 KB, patch)
2019-01-09 05:44 PST, cathiechen
no flags Details | Formatted Diff | Diff
Patch (80.73 KB, patch)
2019-01-09 09:10 PST, cathiechen
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews125 for ios-simulator-wk2 (3.58 MB, application/zip)
2019-01-09 11:05 PST, Build Bot
no flags Details
Patch (80.81 KB, patch)
2019-01-09 20:24 PST, cathiechen
no flags Details | Formatted Diff | Diff
Patch (81.54 KB, patch)
2019-01-09 23:29 PST, cathiechen
no flags Details | Formatted Diff | Diff
Patch (81.54 KB, patch)
2019-01-09 23:32 PST, cathiechen
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description cathiechen 2019-01-08 23:44:47 PST
Import css-lists testcases from web platform tests.
Comment 1 cathiechen 2019-01-08 23:57:54 PST
Created attachment 358677 [details]
Patch
Comment 2 Manuel Rego Casasnovas 2019-01-09 00:07:33 PST
Comment on attachment 358677 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=358677&action=review

> LayoutTests/TestExpectations:2943
> +# Import css-lists testcases
> +imported/w3c/web-platform-tests/css/css-lists/content-property [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/counter-increment-inside-display-contents.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/counter-invalid.htm [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/counter-reset-increment-display-contents.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/counter-reset-increment-display-none.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/counter-reset-inside-display-contents.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/inheritance.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-style-type-armenian-002.xht [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-style-type-armenian-003.xht [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/add-inline-child-after-marker-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-and-block-textarea-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-and-flex-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-and-grid-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-and-margin-collapse-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-marker-with-lineheight-and-overflow-hidden-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-with-image-display-changed-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/li-with-height-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/li-with-overflow-hidden-change-list-style-position-001.html [ Skip ]

Better than skipping the tests is to mark them as failure or have -expected.txt with FAIL messages (depending on the kind of test).
Comment 3 cathiechen 2019-01-09 01:22:40 PST
Created attachment 358681 [details]
Patch
Comment 4 Build Bot 2019-01-09 02:25:17 PST
Comment on attachment 358681 [details]
Patch

Attachment 358681 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/10682201

New failing tests:
imported/w3c/web-platform-tests/css/css-lists/inheritance.html
imported/w3c/web-platform-tests/css/css-lists/counter-increment-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-and-block-textarea-001.html
imported/w3c/web-platform-tests/css/css-lists/list-with-image-display-changed-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-circle.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-latin.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-disc.html
imported/w3c/web-platform-tests/css/css-lists/counter-reset-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-marker-with-lineheight-and-overflow-hidden-001.html
imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-greek.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-square.html
imported/w3c/web-platform-tests/css/css-lists/list-and-margin-collapse-001.html
Comment 5 Build Bot 2019-01-09 02:25:18 PST
Created attachment 358684 [details]
Archive of layout-test-results from ews102 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-sierra  Platform: Mac OS X 10.12.6
Comment 6 Build Bot 2019-01-09 02:38:14 PST
Comment on attachment 358681 [details]
Patch

Attachment 358681 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/10682219

New failing tests:
imported/w3c/web-platform-tests/css/css-lists/inheritance.html
imported/w3c/web-platform-tests/css/css-lists/counter-increment-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-and-block-textarea-001.html
imported/w3c/web-platform-tests/css/css-lists/list-with-image-display-changed-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-circle.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-latin.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-disc.html
imported/w3c/web-platform-tests/css/css-lists/counter-reset-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-marker-with-lineheight-and-overflow-hidden-001.html
imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-greek.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-square.html
imported/w3c/web-platform-tests/css/css-lists/list-and-margin-collapse-001.html
Comment 7 Build Bot 2019-01-09 02:38:15 PST
Created attachment 358685 [details]
Archive of layout-test-results from ews106 for mac-sierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews106  Port: mac-sierra-wk2  Platform: Mac OS X 10.12.6
Comment 8 Build Bot 2019-01-09 02:55:43 PST
Comment on attachment 358681 [details]
Patch

Attachment 358681 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/10682130

New failing tests:
imported/w3c/web-platform-tests/css/css-lists/inheritance.html
imported/w3c/web-platform-tests/css/css-lists/counter-increment-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-and-block-textarea-001.html
imported/w3c/web-platform-tests/css/css-lists/list-with-image-display-changed-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-circle.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-latin.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-disc.html
imported/w3c/web-platform-tests/css/css-lists/counter-reset-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-marker-with-lineheight-and-overflow-hidden-001.html
imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-greek.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-square.html
imported/w3c/web-platform-tests/css/css-lists/list-and-margin-collapse-001.html
Comment 9 Build Bot 2019-01-09 02:55:45 PST
Created attachment 358687 [details]
Archive of layout-test-results from ews115 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews115  Port: mac-sierra  Platform: Mac OS X 10.12.6
Comment 10 Frédéric Wang (:fredw) 2019-01-09 03:01:00 PST
(In reply to Manuel Rego Casasnovas from comment #2)
> Better than skipping the tests is to mark them as failure or have
> -expected.txt with FAIL messages (depending on the kind of test).

Yes, so basically in the present case you can probably use ImageOnlyFailure in TestExpectations for reftests and -expected.txt with FAIL messages for the testharness.js tests. See the EWS output:

Regressions: Unexpected image-only failures (9)
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-circle.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-disc.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-greek.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-latin.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-square.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/counter-increment-inside-display-contents.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/counter-reset-inside-display-contents.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/list-marker-with-lineheight-and-overflow-hidden-001.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/list-with-image-display-changed-001.html [ ImageOnlyFailure ]

Regressions: Unexpected missing results (4)
  imported/w3c/web-platform-tests/css/css-lists/inheritance.html [ Missing ]
  imported/w3c/web-platform-tests/css/css-lists/list-and-block-textarea-001.html [ Missing ]
  imported/w3c/web-platform-tests/css/css-lists/list-and-margin-collapse-001.html [ Missing ]
  imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html [ Missing ]
Comment 11 Build Bot 2019-01-09 03:35:25 PST
Comment on attachment 358681 [details]
Patch

Attachment 358681 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/10682280

New failing tests:
imported/w3c/web-platform-tests/css/css-lists/inheritance.html
imported/w3c/web-platform-tests/css/css-lists/counter-increment-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-latin.html
imported/w3c/web-platform-tests/css/css-lists/list-and-block-textarea-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-circle.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-armenian.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-georgian.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-disc.html
imported/w3c/web-platform-tests/css/css-lists/counter-reset-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-marker-with-lineheight-and-overflow-hidden-001.html
imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-square.html
imported/w3c/web-platform-tests/css/css-lists/list-with-image-display-changed-001.html
imported/w3c/web-platform-tests/css/css-lists/list-and-margin-collapse-001.html
Comment 12 Build Bot 2019-01-09 03:35:27 PST
Created attachment 358690 [details]
Archive of layout-test-results from ews122 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews122  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6
Comment 13 cathiechen 2019-01-09 04:53:45 PST
Created attachment 358694 [details]
Patch
Comment 14 cathiechen 2019-01-09 05:10:06 PST
Created attachment 358695 [details]
Patch
Comment 15 Frédéric Wang (:fredw) 2019-01-09 05:10:46 PST
Comment on attachment 358694 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=358694&action=review

> LayoutTests/TestExpectations:2924
> +# Import css-lists testcases

I would say
# css-lists reftests that are failing

> LayoutTests/TestExpectations:2934
> +imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html [ Pass Failure ]

I don't think you need this one if you have list-and-writing-mode-001-expected. "Pass Failure" is for flaky tests.

> LayoutTests/platform/ios-simulator-wk2/TestExpectations:48
> +# Import css-lists testcases.

Same here. And I would explain why these tests have different behavior on iOS, if possible.
Comment 16 cathiechen 2019-01-09 05:43:04 PST
Comment on attachment 358694 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=358694&action=review

>> LayoutTests/TestExpectations:2924
>> +# Import css-lists testcases
> 
> I would say
> # css-lists reftests that are failing

Done

>> LayoutTests/TestExpectations:2934
>> +imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html [ Pass Failure ]
> 
> I don't think you need this one if you have list-and-writing-mode-001-expected. "Pass Failure" is for flaky tests.

Done! Thanks!

>> LayoutTests/platform/ios-simulator-wk2/TestExpectations:48
>> +# Import css-lists testcases.
> 
> Same here. And I would explain why these tests have different behavior on iOS, if possible.

Done with comment.
These cases aren't related to zero-height. I couldn't give the precise explanation, so I'd better leave them.
Comment 17 cathiechen 2019-01-09 05:44:52 PST
Created attachment 358698 [details]
Patch
Comment 18 cathiechen 2019-01-09 09:10:53 PST
Created attachment 358708 [details]
Patch
Comment 19 Build Bot 2019-01-09 11:05:39 PST
Comment on attachment 358708 [details]
Patch

Attachment 358708 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/10685737

New failing tests:
imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html
Comment 20 Build Bot 2019-01-09 11:05:42 PST
Created attachment 358716 [details]
Archive of layout-test-results from ews125 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews125  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6
Comment 21 cathiechen 2019-01-09 20:24:40 PST
Created attachment 358775 [details]
Patch
Comment 22 Frédéric Wang (:fredw) 2019-01-09 22:33:01 PST
Comment on attachment 358775 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=358775&action=review

> LayoutTests/ChangeLog:6
> +        Reviewed by  Daniel Bates.

nit: there should be only one space between "by" and "Daniel"

> LayoutTests/ChangeLog:10
> +        * platform/ios-simulator-wk2/TestExpectations:

I wonder if we actually want ios-simulator-wk2 rather than ios-simulator but I guess that's ok for now ; someone will rebaseline the tests if necessary.

> LayoutTests/platform/gtk/TestExpectations:3660
> +# css-lists reftests that are failing

thinking again, maybe you can just remove this comment as it does not bring anything new (sorry for my previous suggestion)

> LayoutTests/platform/ios-simulator-wk2/TestExpectations:48
> +# css-lists reftests that are failing

lower-greek passes so I guess you can just remove this comment.

> LayoutTests/platform/ios-simulator-wk2/TestExpectations:52
> +imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html [ Failure ]

The diff we get from comment 20 is just

-FAIL list and writing-mode assert_equals: the height of li should be 45px, and no extra line generated expected 45 but got 63
+FAIL list and writing-mode assert_equals: the height of li should be 45px, and no extra line generated expected 45 but got 65

so I would just add a specific list-and-writing-mode-001-expected.txt file for this platform.
Comment 23 cathiechen 2019-01-09 23:29:52 PST
Created attachment 358781 [details]
Patch
Comment 24 cathiechen 2019-01-09 23:31:12 PST
Comment on attachment 358775 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=358775&action=review

Hi Fred,

Thanks for the review! :)

>> LayoutTests/ChangeLog:6
>> +        Reviewed by  Daniel Bates.
> 
> nit: there should be only one space between "by" and "Daniel"

Done

>> LayoutTests/ChangeLog:10
>> +        * platform/ios-simulator-wk2/TestExpectations:
> 
> I wonder if we actually want ios-simulator-wk2 rather than ios-simulator but I guess that's ok for now ; someone will rebaseline the tests if necessary.

Done. Moved it to ios-simulator/TestExpectations.

>> LayoutTests/platform/gtk/TestExpectations:3660
>> +# css-lists reftests that are failing
> 
> thinking again, maybe you can just remove this comment as it does not bring anything new (sorry for my previous suggestion)

Done

>> LayoutTests/platform/ios-simulator-wk2/TestExpectations:48
>> +# css-lists reftests that are failing
> 
> lower-greek passes so I guess you can just remove this comment.

Done

>> LayoutTests/platform/ios-simulator-wk2/TestExpectations:52
>> +imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html [ Failure ]
> 
> The diff we get from comment 20 is just
> 
> -FAIL list and writing-mode assert_equals: the height of li should be 45px, and no extra line generated expected 45 but got 63
> +FAIL list and writing-mode assert_equals: the height of li should be 45px, and no extra line generated expected 45 but got 65
> 
> so I would just add a specific list-and-writing-mode-001-expected.txt file for this platform.

Done
Comment 25 cathiechen 2019-01-09 23:32:43 PST
Created attachment 358782 [details]
Patch
Comment 26 Frédéric Wang (:fredw) 2019-01-10 01:31:20 PST
Comment on attachment 358782 [details]
Patch

Thanks!
Comment 27 WebKit Commit Bot 2019-01-10 02:01:23 PST
Comment on attachment 358782 [details]
Patch

Clearing flags on attachment: 358782

Committed r239821: <https://trac.webkit.org/changeset/239821>
Comment 28 WebKit Commit Bot 2019-01-10 02:01:25 PST
All reviewed patches have been landed.  Closing bug.
Comment 29 Radar WebKit Bug Importer 2019-01-10 02:03:43 PST
<rdar://problem/47173334>