Bug 179108 - Import Web Platform Tests for WOFF2
Summary: Import Web Platform Tests for WOFF2
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Text (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Frédéric Wang (:fredw)
URL:
Keywords: InRadar
Depends on:
Blocks: 185938 185939
  Show dependency treegraph
 
Reported: 2017-11-01 00:39 PDT by Frédéric Wang (:fredw)
Modified: 2018-05-24 22:02 PDT (History)
8 users (show)

See Also:


Attachments
WIP Patch (2.38 MB, patch)
2017-11-01 00:40 PDT, Frédéric Wang (:fredw)
buildbot: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews101 for mac-elcapitan (2.56 MB, application/zip)
2017-11-03 00:48 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews104 for mac-elcapitan-wk2 (2.30 MB, application/zip)
2017-11-03 01:08 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews116 for mac-elcapitan (2.37 MB, application/zip)
2017-11-03 01:15 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews123 for ios-simulator-wk2 (4.91 MB, application/zip)
2017-11-03 01:22 PDT, Build Bot
no flags Details
WIP Patch (2.41 MB, patch)
2017-11-28 02:17 PST, Frédéric Wang (:fredw)
no flags Details | Formatted Diff | Diff
Archive of layout-test-results from ews103 for mac-elcapitan (3.85 MB, application/zip)
2017-11-28 03:16 PST, EWS Watchlist
no flags Details
Archive of layout-test-results from ews106 for mac-elcapitan-wk2 (3.72 MB, application/zip)
2017-11-28 03:25 PST, EWS Watchlist
no flags Details
Archive of layout-test-results from ews112 for mac-elcapitan (3.68 MB, application/zip)
2017-11-28 03:34 PST, EWS Watchlist
no flags Details
Archive of layout-test-results from ews122 for ios-simulator-wk2 (7.56 MB, application/zip)
2017-11-28 03:57 PST, EWS Watchlist
no flags Details
Patch (2.23 MB, patch)
2018-05-24 01:21 PDT, Frédéric Wang (:fredw)
youennf: review+
ews-watchlist: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews201 for win-future (12.76 MB, application/zip)
2018-05-24 12:06 PDT, EWS Watchlist
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Description Frédéric Wang (:fredw) 2017-11-01 00:39:02 PDT
See https://github.com/w3c/web-platform-tests/tree/master/css/WOFF2
Comment 1 Frédéric Wang (:fredw) 2017-11-01 00:40:47 PDT
Created attachment 325553 [details]
WIP Patch
Comment 2 Myles C. Maxfield 2017-11-02 16:44:53 PDT
This is great. I'm very interested to see the pass rate :)
Comment 3 Frédéric Wang (:fredw) 2017-11-03 00:07:48 PDT
(In reply to Myles C. Maxfield from comment #2)
> This is great. I'm very interested to see the pass rate :)

OK, I'm submitting it to EWS just to check mac and iOS. However, when I tried on GTK all the test fails so I was wondering whether something was wrong with the test configuration (while I know that https://www.w3.org/TR/WOFF2/#conform-mustNotRejectIncorrectTotalSize should definitely pass). Maybe something funny with the *.xht extensions, don't know. I did not have time to debug further unfortunately.
Comment 4 Build Bot 2017-11-03 00:48:21 PDT
Comment on attachment 325553 [details]
WIP Patch

Attachment 325553 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/5087236

Number of test failures exceeded the failure limit.
Comment 5 Build Bot 2017-11-03 00:48:22 PDT
Created attachment 325870 [details]
Archive of layout-test-results from ews101 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews101  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 6 Build Bot 2017-11-03 01:08:05 PDT
Comment on attachment 325553 [details]
WIP Patch

Attachment 325553 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/5087370

Number of test failures exceeded the failure limit.
Comment 7 Build Bot 2017-11-03 01:08:07 PDT
Created attachment 325873 [details]
Archive of layout-test-results from ews104 for mac-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-elcapitan-wk2  Platform: Mac OS X 10.11.6
Comment 8 Build Bot 2017-11-03 01:15:04 PDT
Comment on attachment 325553 [details]
WIP Patch

Attachment 325553 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/5087369

Number of test failures exceeded the failure limit.
Comment 9 Build Bot 2017-11-03 01:15:06 PDT
Created attachment 325875 [details]
Archive of layout-test-results from ews116 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews116  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 10 Build Bot 2017-11-03 01:22:24 PDT
Comment on attachment 325553 [details]
WIP Patch

Attachment 325553 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/5087377

Number of test failures exceeded the failure limit.
Comment 11 Build Bot 2017-11-03 01:22:25 PDT
Created attachment 325876 [details]
Archive of layout-test-results from ews123 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews123  Port: ios-simulator-wk2  Platform: Mac OS X 10.12.6
Comment 12 Frédéric Wang (:fredw) 2017-11-03 02:33:13 PDT
(In reply to Frédéric Wang (:fredw) from comment #3)
> (In reply to Myles C. Maxfield from comment #2)
> > This is great. I'm very interested to see the pass rate :)
> 
> OK, I'm submitting it to EWS just to check mac and iOS. However, when I
> tried on GTK all the test fails so I was wondering whether something was
> wrong with the test configuration (while I know that
> https://www.w3.org/TR/WOFF2/#conform-mustNotRejectIncorrectTotalSize should
> definitely pass). Maybe something funny with the *.xht extensions, don't
> know. I did not have time to debug further unfortunately.

Tests seem to fail on macOS and iOS too.

Mozilla has them in https://hg.mozilla.org/mozilla-central/file/tip/testing/web-platform/tests/css/WOFF2
And I opened https://bugs.chromium.org/p/chromium/issues/detail?id=781154 for chromium.
Comment 13 Frédéric Wang (:fredw) 2017-11-26 04:52:23 PST
OK, the problem was just that the tests require the fonts in web-platform-tests/css/WOFF2/support/FontsToInstall to be installed. When I copy them into webkitgtk-test-fonts/, I get the following results on WebKitGTK:

297 tests ran as expected, 4 didn't:
imported/w3c/web-platform-tests/css/WOFF2/tabledata-glyf-bbox-001.xht
imported/w3c/web-platform-tests/css/WOFF2/tabledata-glyf-origlength-001.xht
imported/w3c/web-platform-tests/css/WOFF2/tabledata-glyf-origlength-002.xht
imported/w3c/web-platform-tests/css/WOFF2/tabledata-recontruct-loca-001.xht

(these tests only fail because of some spacing issue, but the "PASS" message is displayed)

IIRC, on macOS/iOS one has to put the fonts in another place in Tools/ and "whitelist" them for the tests.

I wonder whether it would be better to address that directly at the web-platform-tests repo and I've opened https://github.com/w3c/web-platform-tests/issues/8440
Comment 14 Myles C. Maxfield 2017-11-27 14:01:34 PST
(In reply to Frédéric Wang (:fredw) from comment #13)
> OK, the problem was just that the tests require the fonts in
> web-platform-tests/css/WOFF2/support/FontsToInstall to be installed. When I
> copy them into webkitgtk-test-fonts/, I get the following results on
> WebKitGTK:
> 
> 297 tests ran as expected, 4 didn't:
> imported/w3c/web-platform-tests/css/WOFF2/tabledata-glyf-bbox-001.xht
> imported/w3c/web-platform-tests/css/WOFF2/tabledata-glyf-origlength-001.xht
> imported/w3c/web-platform-tests/css/WOFF2/tabledata-glyf-origlength-002.xht
> imported/w3c/web-platform-tests/css/WOFF2/tabledata-recontruct-loca-001.xht
> 
> (these tests only fail because of some spacing issue, but the "PASS" message
> is displayed)
> 
> IIRC, on macOS/iOS one has to put the fonts in another place in Tools/ and
> "whitelist" them for the tests.
> 
> I wonder whether it would be better to address that directly at the
> web-platform-tests repo and I've opened
> https://github.com/w3c/web-platform-tests/issues/8440

Yeah, using web fonts definitely makes the tests easier to run and integrate into our testing infrastructure.
Comment 15 Frédéric Wang (:fredw) 2017-11-28 02:17:37 PST
Created attachment 327737 [details]
WIP Patch

This version includes https://github.com/w3c/web-platform-tests/commit/3ffcc989b15d7bd671c3ddd22f3c2b90152165de

I have about 90% of success on macOS.
Comment 16 EWS Watchlist 2017-11-28 03:16:49 PST
Comment on attachment 327737 [details]
WIP Patch

Attachment 327737 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/5387239

Number of test failures exceeded the failure limit.
Comment 17 EWS Watchlist 2017-11-28 03:16:50 PST
Created attachment 327740 [details]
Archive of layout-test-results from ews103 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 18 EWS Watchlist 2017-11-28 03:25:08 PST
Comment on attachment 327737 [details]
WIP Patch

Attachment 327737 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/5387260

Number of test failures exceeded the failure limit.
Comment 19 EWS Watchlist 2017-11-28 03:25:10 PST
Created attachment 327741 [details]
Archive of layout-test-results from ews106 for mac-elcapitan-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews106  Port: mac-elcapitan-wk2  Platform: Mac OS X 10.11.6
Comment 20 EWS Watchlist 2017-11-28 03:34:47 PST
Comment on attachment 327737 [details]
WIP Patch

Attachment 327737 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/5387255

Number of test failures exceeded the failure limit.
Comment 21 EWS Watchlist 2017-11-28 03:34:48 PST
Created attachment 327742 [details]
Archive of layout-test-results from ews112 for mac-elcapitan

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews112  Port: mac-elcapitan  Platform: Mac OS X 10.11.6
Comment 22 EWS Watchlist 2017-11-28 03:57:53 PST
Comment on attachment 327737 [details]
WIP Patch

Attachment 327737 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/5387320

New failing tests:
imported/w3c/web-platform-tests/css/WOFF2/tabledata-bad-origlength-loca-002.xht
imported/w3c/web-platform-tests/css/WOFF2/tabledata-bad-origlength-loca-001.xht
imported/w3c/web-platform-tests/css/WOFF2/tabledata-transform-hmtx-003.xht
imported/w3c/web-platform-tests/css/WOFF2/directory-mismatched-tables-001.xht
imported/w3c/web-platform-tests/css/WOFF2/tabledata-glyf-origlength-002.xht
imported/w3c/web-platform-tests/css/WOFF2/tabledata-glyf-bbox-003.xht
imported/w3c/web-platform-tests/css/WOFF2/tabledata-glyf-bbox-001.xht
imported/w3c/web-platform-tests/css/WOFF2/tabledata-glyf-origlength-001.xht
imported/w3c/web-platform-tests/css/WOFF2/blocks-extraneous-data-002.xht
imported/w3c/web-platform-tests/css/WOFF2/datatypes-invalid-base128-003.xht
Comment 23 EWS Watchlist 2017-11-28 03:57:54 PST
Created attachment 327743 [details]
Archive of layout-test-results from ews122 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews122  Port: ios-simulator-wk2  Platform: Mac OS X 10.12.6
Comment 24 Frédéric Wang (:fredw) 2018-05-24 01:21:53 PDT
Created attachment 341179 [details]
Patch
Comment 25 Frédéric Wang (:fredw) 2018-05-24 01:24:50 PDT
@Youenn: Can you please review this?

Given the discussions with new WPT tests (in particular bug 178277) and since macOS/iOS ports fail many WOFF2 tests, for now I'm skipping the WOFF2 tests on all platforms by default and only enabling them on the GTK port. I opened bug 185939 for follow-up work.
Comment 26 EWS Watchlist 2018-05-24 12:06:38 PDT
Comment on attachment 341179 [details]
Patch

Attachment 341179 [details] did not pass win-ews (win):
Output: http://webkit-queues.webkit.org/results/7790481

New failing tests:
http/tests/security/video-poster-cross-origin-crash2.html
Comment 27 EWS Watchlist 2018-05-24 12:06:49 PDT
Created attachment 341214 [details]
Archive of layout-test-results from ews201 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews201  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit
Comment 28 Frédéric Wang (:fredw) 2018-05-24 21:59:21 PDT
Committed r232183: <https://trac.webkit.org/changeset/232183>
Comment 29 Radar WebKit Bug Importer 2018-05-24 22:02:10 PDT
<rdar://problem/40546115>