Bug 193273 – Import css-lists testcases from web platform tests

• Status: RESOLVED
• Resolution: FIXED
• Component: CSS

[cathiechen]

Import css-lists testcases from web platform tests.

[cathiechen]

Created attachment 358677 [details]
Patch

[Manuel Rego Casasnovas]

Comment on attachment 358677 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=358677&action=review

> LayoutTests/TestExpectations:2943
> +# Import css-lists testcases
> +imported/w3c/web-platform-tests/css/css-lists/content-property [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/counter-increment-inside-display-contents.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/counter-invalid.htm [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/counter-reset-increment-display-contents.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/counter-reset-increment-display-none.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/counter-reset-inside-display-contents.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/inheritance.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-style-type-armenian-002.xht [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-style-type-armenian-003.xht [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/add-inline-child-after-marker-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-and-block-textarea-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-and-flex-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-and-grid-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-and-margin-collapse-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-marker-with-lineheight-and-overflow-hidden-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/list-with-image-display-changed-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/li-with-height-001.html [ Skip ]
> +imported/w3c/web-platform-tests/css/css-lists/li-with-overflow-hidden-change-list-style-position-001.html [ Skip ]

Better than skipping the tests is to mark them as failure or have -expected.txt with FAIL messages (depending on the kind of test).

[cathiechen]

Created attachment 358681 [details]
Patch

[EWS Watchlist]

Comment on attachment 358681 [details]
Patch

Attachment 358681 [details] did not pass mac-ews (mac):
Output: https://webkit-queues.webkit.org/results/10682201

New failing tests:
imported/w3c/web-platform-tests/css/css-lists/inheritance.html
imported/w3c/web-platform-tests/css/css-lists/counter-increment-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-and-block-textarea-001.html
imported/w3c/web-platform-tests/css/css-lists/list-with-image-display-changed-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-circle.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-latin.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-disc.html
imported/w3c/web-platform-tests/css/css-lists/counter-reset-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-marker-with-lineheight-and-overflow-hidden-001.html
imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-greek.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-square.html
imported/w3c/web-platform-tests/css/css-lists/list-and-margin-collapse-001.html

[EWS Watchlist]

Created attachment 358684 [details]
Archive of layout-test-results from ews102 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102  Port: mac-sierra  Platform: Mac OS X 10.12.6

[EWS Watchlist]

Comment on attachment 358681 [details]
Patch

Attachment 358681 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/10682219

New failing tests:
imported/w3c/web-platform-tests/css/css-lists/inheritance.html
imported/w3c/web-platform-tests/css/css-lists/counter-increment-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-and-block-textarea-001.html
imported/w3c/web-platform-tests/css/css-lists/list-with-image-display-changed-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-circle.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-latin.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-disc.html
imported/w3c/web-platform-tests/css/css-lists/counter-reset-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-marker-with-lineheight-and-overflow-hidden-001.html
imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-greek.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-square.html
imported/w3c/web-platform-tests/css/css-lists/list-and-margin-collapse-001.html

[EWS Watchlist]

Created attachment 358685 [details]
Archive of layout-test-results from ews106 for mac-sierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews106  Port: mac-sierra-wk2  Platform: Mac OS X 10.12.6

[EWS Watchlist]

Comment on attachment 358681 [details]
Patch

Attachment 358681 [details] did not pass mac-debug-ews (mac):
Output: https://webkit-queues.webkit.org/results/10682130

New failing tests:
imported/w3c/web-platform-tests/css/css-lists/inheritance.html
imported/w3c/web-platform-tests/css/css-lists/counter-increment-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-and-block-textarea-001.html
imported/w3c/web-platform-tests/css/css-lists/list-with-image-display-changed-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-circle.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-latin.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-disc.html
imported/w3c/web-platform-tests/css/css-lists/counter-reset-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-marker-with-lineheight-and-overflow-hidden-001.html
imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-greek.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-square.html
imported/w3c/web-platform-tests/css/css-lists/list-and-margin-collapse-001.html

[EWS Watchlist]

Created attachment 358687 [details]
Archive of layout-test-results from ews115 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews115  Port: mac-sierra  Platform: Mac OS X 10.12.6

[Frédéric Wang (:fredw)]

(In reply to Manuel Rego Casasnovas from comment #2)
> Better than skipping the tests is to mark them as failure or have
> -expected.txt with FAIL messages (depending on the kind of test).

Yes, so basically in the present case you can probably use ImageOnlyFailure in TestExpectations for reftests and -expected.txt with FAIL messages for the testharness.js tests. See the EWS output:

Regressions: Unexpected image-only failures (9)
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-circle.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-disc.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-greek.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-latin.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-square.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/counter-increment-inside-display-contents.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/counter-reset-inside-display-contents.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/list-marker-with-lineheight-and-overflow-hidden-001.html [ ImageOnlyFailure ]
  imported/w3c/web-platform-tests/css/css-lists/list-with-image-display-changed-001.html [ ImageOnlyFailure ]

Regressions: Unexpected missing results (4)
  imported/w3c/web-platform-tests/css/css-lists/inheritance.html [ Missing ]
  imported/w3c/web-platform-tests/css/css-lists/list-and-block-textarea-001.html [ Missing ]
  imported/w3c/web-platform-tests/css/css-lists/list-and-margin-collapse-001.html [ Missing ]
  imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html [ Missing ]

[EWS Watchlist]

Comment on attachment 358681 [details]
Patch

Attachment 358681 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/10682280

New failing tests:
imported/w3c/web-platform-tests/css/css-lists/inheritance.html
imported/w3c/web-platform-tests/css/css-lists/counter-increment-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-lower-latin.html
imported/w3c/web-platform-tests/css/css-lists/list-and-block-textarea-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-circle.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-armenian.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-georgian.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-disc.html
imported/w3c/web-platform-tests/css/css-lists/counter-reset-inside-display-contents.html
imported/w3c/web-platform-tests/css/css-lists/list-marker-with-lineheight-and-overflow-hidden-001.html
imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html
imported/w3c/web-platform-tests/css/css-lists/content-property/marker-text-matches-square.html
imported/w3c/web-platform-tests/css/css-lists/list-with-image-display-changed-001.html
imported/w3c/web-platform-tests/css/css-lists/list-and-margin-collapse-001.html

[EWS Watchlist]

Created attachment 358690 [details]
Archive of layout-test-results from ews122 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews122  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6

[cathiechen]

Created attachment 358694 [details]
Patch

[cathiechen]

Created attachment 358695 [details]
Patch

[Frédéric Wang (:fredw)]

Comment on attachment 358694 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=358694&action=review

> LayoutTests/TestExpectations:2924
> +# Import css-lists testcases

I would say
# css-lists reftests that are failing

> LayoutTests/TestExpectations:2934
> +imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html [ Pass Failure ]

I don't think you need this one if you have list-and-writing-mode-001-expected. "Pass Failure" is for flaky tests.

> LayoutTests/platform/ios-simulator-wk2/TestExpectations:48
> +# Import css-lists testcases.

Same here. And I would explain why these tests have different behavior on iOS, if possible.

[cathiechen]

Comment on attachment 358694 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=358694&action=review

>> LayoutTests/TestExpectations:2924
>> +# Import css-lists testcases
> 
> I would say
> # css-lists reftests that are failing

Done

>> LayoutTests/TestExpectations:2934
>> +imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html [ Pass Failure ]
> 
> I don't think you need this one if you have list-and-writing-mode-001-expected. "Pass Failure" is for flaky tests.

Done! Thanks!

>> LayoutTests/platform/ios-simulator-wk2/TestExpectations:48
>> +# Import css-lists testcases.
> 
> Same here. And I would explain why these tests have different behavior on iOS, if possible.

Done with comment.
These cases aren't related to zero-height. I couldn't give the precise explanation, so I'd better leave them.

[cathiechen]

Created attachment 358698 [details]
Patch

[cathiechen]

Created attachment 358708 [details]
Patch

[EWS Watchlist]

Comment on attachment 358708 [details]
Patch

Attachment 358708 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/10685737

New failing tests:
imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html

[EWS Watchlist]

Created attachment 358716 [details]
Archive of layout-test-results from ews125 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews125  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6

[cathiechen]

Created attachment 358775 [details]
Patch

[Frédéric Wang (:fredw)]

Comment on attachment 358775 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=358775&action=review

> LayoutTests/ChangeLog:6
> +        Reviewed by  Daniel Bates.

nit: there should be only one space between "by" and "Daniel"

> LayoutTests/ChangeLog:10
> +        * platform/ios-simulator-wk2/TestExpectations:

I wonder if we actually want ios-simulator-wk2 rather than ios-simulator but I guess that's ok for now ; someone will rebaseline the tests if necessary.

> LayoutTests/platform/gtk/TestExpectations:3660
> +# css-lists reftests that are failing

thinking again, maybe you can just remove this comment as it does not bring anything new (sorry for my previous suggestion)

> LayoutTests/platform/ios-simulator-wk2/TestExpectations:48
> +# css-lists reftests that are failing

lower-greek passes so I guess you can just remove this comment.

> LayoutTests/platform/ios-simulator-wk2/TestExpectations:52
> +imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html [ Failure ]

The diff we get from comment 20 is just

-FAIL list and writing-mode assert_equals: the height of li should be 45px, and no extra line generated expected 45 but got 63
+FAIL list and writing-mode assert_equals: the height of li should be 45px, and no extra line generated expected 45 but got 65

so I would just add a specific list-and-writing-mode-001-expected.txt file for this platform.

[cathiechen]

Created attachment 358781 [details]
Patch

[cathiechen]

Comment on attachment 358775 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=358775&action=review

Hi Fred,

Thanks for the review! :)

>> LayoutTests/ChangeLog:6
>> +        Reviewed by  Daniel Bates.
> 
> nit: there should be only one space between "by" and "Daniel"

Done

>> LayoutTests/ChangeLog:10
>> +        * platform/ios-simulator-wk2/TestExpectations:
> 
> I wonder if we actually want ios-simulator-wk2 rather than ios-simulator but I guess that's ok for now ; someone will rebaseline the tests if necessary.

Done. Moved it to ios-simulator/TestExpectations.

>> LayoutTests/platform/gtk/TestExpectations:3660
>> +# css-lists reftests that are failing
> 
> thinking again, maybe you can just remove this comment as it does not bring anything new (sorry for my previous suggestion)

Done

>> LayoutTests/platform/ios-simulator-wk2/TestExpectations:48
>> +# css-lists reftests that are failing
> 
> lower-greek passes so I guess you can just remove this comment.

Done

>> LayoutTests/platform/ios-simulator-wk2/TestExpectations:52
>> +imported/w3c/web-platform-tests/css/css-lists/list-and-writing-mode-001.html [ Failure ]
> 
> The diff we get from comment 20 is just
> 
> -FAIL list and writing-mode assert_equals: the height of li should be 45px, and no extra line generated expected 45 but got 63
> +FAIL list and writing-mode assert_equals: the height of li should be 45px, and no extra line generated expected 45 but got 65
> 
> so I would just add a specific list-and-writing-mode-001-expected.txt file for this platform.

Done

[cathiechen]

Created attachment 358782 [details]
Patch

[Frédéric Wang (:fredw)]

Comment on attachment 358782 [details]
Patch

Thanks!

[WebKit Commit Bot]

Comment on attachment 358782 [details]
Patch

Clearing flags on attachment: 358782

Committed r239821: <https://trac.webkit.org/changeset/239821>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Radar WebKit Bug Importer]

<rdar://problem/47173334>