Attachment 341651 Details for Bug 183356 – Patch

[patch] Patch

bug-183356-20180531102955.patch (text/plain), 70.12 KB, created by Zan Dobersek on 2018-05-31 01:29:57 PDT

(hide)

Description:

Filename:

MIME Type:

Creator: Zan Dobersek

Created: 2018-05-31 01:29:57 PDT

Size: 70.12 KB

patch

obsolete

>Subversion Revision: 232336
>diff --git a/Tools/ChangeLog b/Tools/ChangeLog
>index a108e0cbbfff11844f90be40c9ed57efab420103..ed8582883174dbc2dd507527cc7d7d35de1b7aef 100644
>--- a/Tools/ChangeLog
>+++ b/Tools/ChangeLog
>@@ -1,3 +1,86 @@
>+2018-05-31  Zan Dobersek  <zdobersek@igalia.com>
>+
>+        Add run-web-platform-tests script
>+        https://bugs.webkit.org/show_bug.cgi?id=183356
>+
>+        Reviewed by NOBODY (OOPS!).
>+
>+        Add the run-web-platform-tests script that allows running WebKit builds
>+        against the external web-platform-tests project.
>+
>+        The script sets up all the port-specific information, including the
>+        display driver. If no web-platform-tests repository location is
>+        specified, the TestDownloader utility is used to clone the repository,
>+        and path to that clone is leveraged instead.
>+
>+        Port-specific TestManifest.ini and TestExpectations.json file paths are
>+        then constructed, bailing if any of the files is missing. The JSON file
>+        is then used to construct the metadata file tree, creating an .ini file
>+        for each test that is disabled, has a certain expectation, or has
>+        subtests that themselves have certain expectations. This metadata tree
>+        is built under the build directory, and is purged and reconstructed for
>+        each separate run.
>+
>+        Lastly, the WPT python module is imported from the web-platform-tests
>+        source location, and wpt.main() is invoked along with all the necessary
>+        arguments that set up a web-platform-tests suite run using the WebKit
>+        product.
>+
>+        * Scripts/run-web-platform-tests: Added.
>+        * Scripts/webkitpy/common/config/ports_mock.py:
>+        (MockPort.run_bindings_tests_command):
>+        (MockPort):
>+        (MockPort.wpt_metadata_directory):
>+        (MockPort.wpt_manifest_file):
>+        * Scripts/webkitpy/port/base.py:
>+        (Port.wpt_metadata_directory):
>+        (Port):
>+        (Port.wpt_manifest_file):
>+        * Scripts/webkitpy/w3c/common.py:
>+        (is_file_exportable):
>+        (WPTPaths):
>+        (WPTPaths.checkout_directory):
>+        (WPTPaths.wpt_checkout_path):
>+        * Scripts/webkitpy/w3c/test_downloader.py:
>+        (TestDownloader.clone_tests):
>+        (TestDownloader):
>+        (TestDownloader.download_tests):
>+        * Scripts/webkitpy/w3c/test_exporter.py:
>+        (WebPlatformTestExporter.__init__):
>+        * Scripts/webkitpy/w3c/wpt_runner.py: Added.
>+        (main):
>+        (parse_args):
>+        (create_webdriver):
>+        (spawn_wpt):
>+        (WPTRunner):
>+        (WPTRunner.__init__):
>+        (WPTRunner._prepare_wpt_checkout):
>+        (WPTRunner._generate_metadata_directory):
>+        (WPTRunner.run):
>+        * Scripts/webkitpy/w3c/wpt_runner_unittest.py: Added.
>+        (WPTRunnerTest):
>+        (WPTRunnerTest.MockTestDownloader):
>+        (WPTRunnerTest.MockTestDownloader.default_options):
>+        (WPTRunnerTest.MockTestDownloader.__init__):
>+        (WPTRunnerTest.MockTestDownloader.clone_tests):
>+        (WPTRunnerTest.MockWebDriver):
>+        (WPTRunnerTest.MockWebDriver.create):
>+        (WPTRunnerTest.MockWebDriver.binary_path):
>+        (WPTRunnerTest.MockWebDriver.browser_path):
>+        (WPTRunnerTest.MockWebDriver.browser_args):
>+        (WPTRunnerTest.MockSpawnWPT):
>+        (WPTRunnerTest.MockSpawnWPT.__init__):
>+        (WPTRunnerTest.MockSpawnWPT.__call__):
>+        (WPTRunnerTest.TestInstance):
>+        (WPTRunnerTest.TestInstance.__init__):
>+        (WPTRunnerTest.TestInstance.prepare_mock_files_for_run):
>+        (WPTRunnerTest.test_prepare_wpt_checkout):
>+        (WPTRunnerTest.test_prepare_wpt_checkout_specified_path):
>+        (WPTRunnerTest.test_generate_metadata_directory):
>+        (WPTRunnerTest.test_run):
>+        (WPTRunnerTest.test_run_with_specified_options):
>+        (WPTRunnerTest.test_run_with_args):
>+
> 2018-05-30  Fujii Hironori  <Hironori.Fujii@sony.com>
> 
>         [Win][MiniBrowser] Move MainWindow related code from Common.cpp to MainWindow.cpp
>diff --git a/Tools/Scripts/run-web-platform-tests b/Tools/Scripts/run-web-platform-tests
>new file mode 100755
>index 0000000000000000000000000000000000000000..dcd25159ee87a4ca3f4811c8d1e2078394350505
>--- /dev/null
>+++ b/Tools/Scripts/run-web-platform-tests
>@@ -0,0 +1,30 @@
>+#!/usr/bin/env python
>+
>+# Copyright (C) 2018 Igalia S.L.
>+#
>+# Redistribution and use in source and binary forms, with or without
>+# modification, are permitted provided that the following conditions
>+# are met:
>+# 1.  Redistributions of source code must retain the above copyright
>+#     notice, this list of conditions and the following disclaimer.
>+# 2.  Redistributions in binary form must reproduce the above copyright
>+#     notice, this list of conditions and the following disclaimer in the
>+#     documentation and/or other materials provided with the distribution.
>+#
>+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' AND ANY
>+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
>+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
>+# DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS BE LIABLE FOR ANY
>+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
>+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
>+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
>+# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
>+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>+
>+import sys
>+
>+from webkitpy.w3c import wpt_runner
>+
>+
>+wpt_runner.main(sys.argv[0], sys.argv[1:])
>diff --git a/Tools/Scripts/webkitpy/common/config/ports_mock.py b/Tools/Scripts/webkitpy/common/config/ports_mock.py
>index ca1b18a31068423bdce07cdeb7d7d7d199f400ab..cb83b555ccc95b35b53628233fd190b930d2c30b 100644
>--- a/Tools/Scripts/webkitpy/common/config/ports_mock.py
>+++ b/Tools/Scripts/webkitpy/common/config/ports_mock.py
>@@ -58,3 +58,9 @@ class MockPort(object):
> 
>     def run_bindings_tests_command(self):
>         return ['mock-run-bindings-tests']
>+
>+    def wpt_metadata_directory(self):
>+        return '/mock-path/mock-wpt-tests-metadata'
>+
>+    def wpt_manifest_file(self):
>+        return '/mock-path/mock-wpt-manifest.json'
>diff --git a/Tools/Scripts/webkitpy/port/base.py b/Tools/Scripts/webkitpy/port/base.py
>index f83fc420efb121c494f61a0a7c983ec04499e411..dd8cd6560e45b1c2a6d280ae16582bad50bb77fc 100644
>--- a/Tools/Scripts/webkitpy/port/base.py
>+++ b/Tools/Scripts/webkitpy/port/base.py
>@@ -871,6 +871,12 @@ class Port(object):
>         # to have multiple copies of webkit checked out and built.
>         return self._build_path('layout-test-results')
> 
>+    def wpt_metadata_directory(self):
>+        return self._build_path('web-platform-tests-metadata')
>+
>+    def wpt_manifest_file(self):
>+        return self._build_path('web-platform-tests-manifest.json')
>+
>     def setup_test_run(self, device_class=None):
>         """Perform port-specific work at the beginning of a test run."""
>         pass
>diff --git a/Tools/Scripts/webkitpy/w3c/common.py b/Tools/Scripts/webkitpy/w3c/common.py
>index 66fa34f2a7e9af17376b643cb4c1a64a9888b20f..d4f9f20f88397fd4462e7f907e829df9a418aa8a 100644
>--- a/Tools/Scripts/webkitpy/w3c/common.py
>+++ b/Tools/Scripts/webkitpy/w3c/common.py
>@@ -98,3 +98,16 @@ def is_file_exportable(path):
>     assert path.startswith(CHROMIUM_WPT_DIR)
>     basename = path[path.rfind('/') + 1:]
>     return not is_basename_skipped(basename)
>+
>+
>+class WPTPaths:
>+    CHECKOUT_DIRECTORY = ["WebKitBuild", "w3c-tests"]
>+    WPT_CHECKOUT_PATH = CHECKOUT_DIRECTORY + ["web-platform-tests"]
>+
>+    @staticmethod
>+    def checkout_directory(finder):
>+        return finder.path_from_webkit_base(*WPTPaths.CHECKOUT_DIRECTORY)
>+
>+    @staticmethod
>+    def wpt_checkout_path(finder):
>+        return finder.path_from_webkit_base(*WPTPaths.WPT_CHECKOUT_PATH)
>diff --git a/Tools/Scripts/webkitpy/w3c/test_downloader.py b/Tools/Scripts/webkitpy/w3c/test_downloader.py
>index d17fa84635ebe002c04ec776cfb51d3b790212d5..49d78f9a106aba373339bf363cd87a2e401052e4 100644
>--- a/Tools/Scripts/webkitpy/w3c/test_downloader.py
>+++ b/Tools/Scripts/webkitpy/w3c/test_downloader.py
>@@ -219,7 +219,10 @@ class TestDownloader(object):
>             rules.append('/'.join(path[:-1]) + '/.' + path[-1] + '.url')
>         self._filesystem.write_text_file(self._filesystem.join(destination_directory, test_repository['name'], '.gitignore'), '\n'.join(rules))
> 
>-    def download_tests(self, destination_directory, test_paths=[], use_tip_of_tree=False):
>+    def clone_tests(self, use_tip_of_tree=False):
>         for test_repository in self.test_repositories:
>             self.checkout_test_repository(test_repository['revision'] if not use_tip_of_tree else 'origin/master', test_repository['url'], self._filesystem.join(self.repository_directory, test_repository['name']))
>+
>+    def download_tests(self, destination_directory, test_paths=[], use_tip_of_tree=False):
>+        self.clone_tests(use_tip_of_tree)
>         self.copy_tests(destination_directory, test_paths)
>diff --git a/Tools/Scripts/webkitpy/w3c/test_exporter.py b/Tools/Scripts/webkitpy/w3c/test_exporter.py
>index f762de4647c472fbe60cfe7474cf68569ffb71c4..1cb57df5ac87705daf64b07ac8a57d837b495bc9 100644
>--- a/Tools/Scripts/webkitpy/w3c/test_exporter.py
>+++ b/Tools/Scripts/webkitpy/w3c/test_exporter.py
>@@ -37,7 +37,7 @@ from webkitpy.common.net.bugzilla import Bugzilla
> from webkitpy.common.webkit_finder import WebKitFinder
> from webkitpy.w3c.wpt_github import WPTGitHub
> from webkitpy.w3c.wpt_linter import WPTLinter
>-from webkitpy.w3c.common import WPT_GH_ORG
>+from webkitpy.w3c.common import WPT_GH_ORG, WPTPaths
> from webkitpy.common.memoized import memoized
> 
> _log = logging.getLogger(__name__)
>@@ -69,7 +69,7 @@ class WebPlatformTestExporter(object):
> 
>         if not self._options.repository_directory:
>             webkit_finder = WebKitFinder(self._filesystem)
>-            self._options.repository_directory = webkit_finder.path_from_webkit_base('WebKitBuild', 'w3c-tests', 'web-platform-tests')
>+            self._options.repository_directory = WPTPaths.wpt_checkout_path(webkit_finder)
> 
>         self._linter = WPTLinterClass(self._options.repository_directory, host.filesystem)
> 
>diff --git a/Tools/Scripts/webkitpy/w3c/wpt_runner.py b/Tools/Scripts/webkitpy/w3c/wpt_runner.py
>new file mode 100644
>index 0000000000000000000000000000000000000000..22a528a4aaa63c725cfdada20e4b7216d779d3a5
>--- /dev/null
>+++ b/Tools/Scripts/webkitpy/w3c/wpt_runner.py
>@@ -0,0 +1,201 @@
>+# Copyright (C) 2018 Igalia S.L.
>+#
>+# Redistribution and use in source and binary forms, with or without
>+# modification, are permitted provided that the following conditions
>+# are met:
>+# 1.  Redistributions of source code must retain the above copyright
>+#     notice, this list of conditions and the following disclaimer.
>+# 2.  Redistributions in binary form must reproduce the above copyright
>+#     notice, this list of conditions and the following disclaimer in the
>+#     documentation and/or other materials provided with the distribution.
>+#
>+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' AND ANY
>+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
>+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
>+# DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS BE LIABLE FOR ANY
>+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
>+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
>+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
>+# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
>+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>+
>+import argparse
>+import json
>+import logging
>+import optparse
>+import os
>+import sys
>+
>+from webkitpy.common.host import Host
>+from webkitpy.common.system.logutils import configure_logging
>+from webkitpy.common.webkit_finder import WebKitFinder
>+from webkitpy.w3c.common import WPTPaths
>+from webkitpy.w3c.test_downloader import TestDownloader
>+from webkitpy.webdriver_tests.webdriver_driver import create_driver
>+
>+_log = logging.getLogger(__name__)
>+
>+
>+def main(script_name, argv):
>+    options, args = parse_args(argv)
>+
>+    configure_logging(logging.DEBUG if options.verbose else logging.INFO)
>+
>+    # Determine Port for the specified platform.
>+    host = Host()
>+    try:
>+        port = host.port_factory.get(options.platform, options)
>+    except NotImplementedError, e:
>+        _log.error(str(e))
>+        sys.exit(-1)
>+
>+    # If necessary, inject the jhbuild wrapper.
>+    if port.name() in ['gtk', 'wpe']:
>+        filesystem = host.filesystem
>+        top_level_directory = filesystem.normpath(filesystem.join(filesystem.dirname(__file__), '..', '..', '..', '..'))
>+        sys.path.insert(0, filesystem.join(top_level_directory, 'Tools', 'jhbuild'))
>+        import jhbuildutils
>+
>+        if not jhbuildutils.enter_jhbuild_environment_if_available(port.name()):
>+            _log.warning('jhbuild environment not present. Run update-webkitgtk-libs before build-webkit to ensure proper testing.')
>+
>+    # Create the Port-specific driver.
>+    port._display_server = options.display_server
>+    display_driver = port.create_driver(worker_number=0, no_timeout=True)._make_driver(pixel_tests=False)
>+    if not display_driver.check_driver(port):
>+        raise RuntimeError("Failed to check driver %s" % display_driver.__class__.__name__)
>+    os.environ.update(display_driver._setup_environ_for_test())
>+
>+    if not options.child_processes:
>+        options.child_processes = os.environ.get("WEBKIT_TEST_CHILD_PROCESSES",
>+                                                 str(port.default_child_processes()))
>+
>+    runner = WPTRunner(port, port.host, script_name, options)
>+    if not runner.run(args):
>+        sys.exit(1)
>+
>+
>+def parse_args(args):
>+    option_parser = optparse.OptionParser(usage='usage: %prog [options] [test...]')
>+    option_parser.add_option('--verbose', action='store_true',
>+                             help='Show debug message')
>+    option_parser.add_option('--platform', action='store',
>+                             help='Platform to use (e.g., "gtk")')
>+    option_parser.add_option('--gtk', action='store_const', dest='platform', const='gtk',
>+                             help='Alias for --platform=gtk')
>+    option_parser.add_option('--child-processes',
>+                             help='Number of tests to run in parallel'),
>+    option_parser.add_option('--wpt-checkout', default=None,
>+                             help='web-platform-tests repository location')
>+    option_parser.add_option('--display-server', choices=['xvfb', 'xorg', 'weston', 'wayland'], default='xvfb',
>+                             help='"xvfb": Use a virtualized X11 server. "xorg": Use the current X11 session. '
>+                                  '"weston": Use a virtualized Weston server. "wayland": Use the current wayland session.')
>+
>+    return option_parser.parse_args(args)
>+
>+
>+def create_webdriver(port):
>+    return create_driver(port)
>+
>+
>+def spawn_wpt(script_name, wpt_checkout, wpt_args):
>+    # Import the WPT python module.
>+    sys.path.insert(0, wpt_checkout)
>+    try:
>+        from tools.wpt import wpt
>+    except ImportError:
>+        _log.error("Failed to import wpt Python modules from the web-platform-tests directory")
>+        sys.exit(-1)
>+
>+    wpt.main(script_name, wpt_args)
>+
>+
>+class WPTRunner(object):
>+    def __init__(self, port, host, script_name, options, downloader_class=TestDownloader,
>+        create_webdriver_func=create_webdriver, spawn_wpt_func=spawn_wpt):
>+        self._port = port
>+        self._host = host
>+        self._finder = WebKitFinder(self._host.filesystem)
>+
>+        self._script_name = script_name
>+        self._options = options
>+
>+        self._downloader_class = downloader_class
>+        self._create_webdriver_func = create_webdriver_func
>+        self._spawn_wpt_func = spawn_wpt_func
>+
>+    def _prepare_wpt_checkout(self):
>+        if not self._options.wpt_checkout:
>+            test_downloader = self._downloader_class(WPTPaths.checkout_directory(self._finder),
>+                self._host, self._downloader_class.default_options())
>+            test_downloader.clone_tests()
>+            self._options.wpt_checkout = WPTPaths.wpt_checkout_path(self._finder)
>+
>+        if not self._options.wpt_checkout or not self._host.filesystem.exists(self._options.wpt_checkout):
>+            _log.error("Valid web-platform-tests directory required")
>+            return False
>+        return True
>+
>+    def _generate_metadata_directory(self, metadata_path):
>+        expectations_file = self._finder.path_from_webkit_base("WebPlatformTests", self._port.name(), "TestExpectations.json")
>+        if not self._host.filesystem.exists(expectations_file):
>+            _log.error("Port-specific expectation .json file does not exist")
>+            return False
>+
>+        with self._host.filesystem.open_text_file_for_reading(expectations_file) as fd:
>+            expectations = json.load(fd)
>+
>+        for test_name, test_data in expectations.iteritems():
>+            ini_file = self._host.filesystem.join(metadata_path, test_name + ".ini")
>+            self._host.filesystem.maybe_make_directory(self._host.filesystem.dirname(ini_file))
>+            with self._host.filesystem.open_text_file_for_writing(ini_file) as fd:
>+                fd.write("[{}]\n".format(test_data.get("test_name", self._host.filesystem.basename(test_name))))
>+                if "disabled" in test_data:
>+                    fd.write("    disabled: {}\n".format(test_data["disabled"]))
>+                elif "expected" in test_data:
>+                    fd.write("    expected: {}\n".format(test_data["expected"]))
>+                elif "subtests" in test_data:
>+                    for subtest_name, subtest_data in test_data["subtests"].iteritems():
>+                        fd.write("    [{}]\n".format(subtest_name))
>+                        if "expected" in subtest_data:
>+                            fd.write("        expected: {}\n".format(subtest_data["expected"]))
>+                        else:
>+                            _log.error("Invalid format of TestExpectations.json")
>+                            return False
>+                else:
>+                    _log.error("Invalid format of TestExpectations.json")
>+                    return False
>+
>+        return True
>+
>+    def run(self, args):
>+        if not self._prepare_wpt_checkout():
>+            return False
>+
>+        # Parse the test expectations JSON and construct corresponding metadata files.
>+        metadata_path = self._port.wpt_metadata_directory()
>+        if not self._generate_metadata_directory(metadata_path):
>+            return False
>+
>+        include_manifest_path = self._finder.path_from_webkit_base("WebPlatformTests", self._port.name(), "TestManifest.ini")
>+        if not self._host.filesystem.exists(include_manifest_path):
>+            _log.error("Port-specific manifest .ini file does not exist")
>+            return False
>+
>+        # Construct the WebDriver instance and run WPT tests via the 'webkit' product.
>+        manifest_path = self._port.wpt_manifest_file()
>+        webdriver = self._create_webdriver_func(self._port)
>+
>+        wpt_args = ["run", "--webkit-port={}".format(self._port.name()),
>+            "--processes={}".format(self._options.child_processes),
>+            "--metadata={}".format(metadata_path),
>+            "--manifest={}".format(manifest_path),
>+            "--include-manifest={}".format(include_manifest_path),
>+            "--webdriver-binary={}".format(webdriver.binary_path()),
>+            "--binary={}".format(webdriver.browser_path())]
>+        wpt_args += ["--binary-arg={}".format(arg) for arg in webdriver.browser_args()]
>+        wpt_args += ["webkit"] + args
>+
>+        self._spawn_wpt_func(self._script_name, self._options.wpt_checkout, wpt_args)
>+        return True
>diff --git a/Tools/Scripts/webkitpy/w3c/wpt_runner_unittest.py b/Tools/Scripts/webkitpy/w3c/wpt_runner_unittest.py
>new file mode 100644
>index 0000000000000000000000000000000000000000..da399bf261f95e88d460fbbd7bf2701a27966bb6
>--- /dev/null
>+++ b/Tools/Scripts/webkitpy/w3c/wpt_runner_unittest.py
>@@ -0,0 +1,270 @@
>+# Copyright (C) 2018 Igalia S.L.
>+#
>+# Redistribution and use in source and binary forms, with or without
>+# modification, are permitted provided that the following conditions
>+# are met:
>+# 1.  Redistributions of source code must retain the above copyright
>+#     notice, this list of conditions and the following disclaimer.
>+# 2.  Redistributions in binary form must reproduce the above copyright
>+#     notice, this list of conditions and the following disclaimer in the
>+#     documentation and/or other materials provided with the distribution.
>+#
>+# THIS SOFTWARE IS PROVIDED BY APPLE INC. AND ITS CONTRIBUTORS ``AS IS'' AND ANY
>+# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
>+# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
>+# DISCLAIMED. IN NO EVENT SHALL APPLE INC. OR ITS CONTRIBUTORS BE LIABLE FOR ANY
>+# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
>+# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
>+# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
>+# ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
>+# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
>+# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
>+
>+import unittest
>+
>+from webkitpy.common.config.ports_mock import MockPort
>+from webkitpy.common.host_mock import MockHost
>+from webkitpy.w3c.wpt_runner import WPTRunner, parse_args
>+
>+
>+TEST_EXPECTATIONS_JSON_CONTENT = """{
>+    "passing_test.html": { "expected": "PASS" },
>+    "failing_test.html": { "expected": "FAIL" },
>+    "disabled_test.html": { "disabled": "Test Name #1" },
>+    "custom_test_name.html": {
>+        "test_name": "Custom Test Name",
>+        "expected": "FAIL"
>+    },
>+    "test_with_subtests.html": {
>+        "subtests": {
>+            "Subtest #1": { "expected": "PASS" },
>+            "Subtest #2": { "expected": "TIMEOUT" }
>+        }
>+    },
>+    "nested/test_file.html": { "expected": "PASS" },
>+    "nested/nested/test_file.html": {
>+        "test_name": "Deeper-nested test case",
>+        "subtests": {
>+            "Only Subtest": { "expected": "PASS" }
>+        }
>+    }
>+}"""
>+
>+EXPECTED_TEST_MANIFESTS = {
>+    "passing_test.html.ini":
>+"""[passing_test.html]
>+    expected: PASS
>+""",
>+
>+    "failing_test.html.ini":
>+"""[failing_test.html]
>+    expected: FAIL
>+""",
>+
>+    "disabled_test.html.ini":
>+"""[disabled_test.html]
>+    disabled: Test Name #1
>+""",
>+
>+    "custom_test_name.html.ini":
>+"""[Custom Test Name]
>+    expected: FAIL
>+""",
>+
>+    "test_with_subtests.html.ini":
>+"""[test_with_subtests.html]
>+    [Subtest #1]
>+        expected: PASS
>+    [Subtest #2]
>+        expected: TIMEOUT
>+""",
>+
>+    "nested/test_file.html.ini":
>+"""[test_file.html]
>+    expected: PASS
>+""",
>+
>+    "nested/nested/test_file.html.ini":
>+"""[Deeper-nested test case]
>+    [Only Subtest]
>+        expected: PASS
>+"""
>+}
>+
>+
>+class WPTRunnerTest(unittest.TestCase):
>+
>+    class MockTestDownloader(object):
>+        @staticmethod
>+        def default_options():
>+            return {}
>+
>+        def __init__(self, repository_directory, host, options):
>+            self._repository_directory = repository_directory
>+            self._host = host
>+
>+        def clone_tests(self):
>+            self._host.filesystem.maybe_make_directory(self._repository_directory, "web-platform-tests")
>+
>+    class MockWebDriver(object):
>+        @staticmethod
>+        def create(port):
>+            return WPTRunnerTest.MockWebDriver()
>+
>+        def binary_path(self):
>+            return "/mock-webdriver/bin/webdriver"
>+
>+        def browser_path(self):
>+            return "/mock-webdriver/bin/browser"
>+
>+        def browser_args(self):
>+            return ["webdriver_arg1", "webdriver_arg2"]
>+
>+    class MockSpawnWPT(object):
>+        def __init__(self, test_case, expected_wpt_checkout=None, expected_wpt_args=None):
>+            self._test_case = test_case
>+            self._expected_wpt_checkout = expected_wpt_checkout
>+            self._expected_wpt_args = expected_wpt_args
>+
>+        def __call__(self, script_name, wpt_checkout, wpt_args):
>+            self._test_case.assertEquals(script_name, "wptrunner_unittest")
>+            self._test_case.assertEquals(wpt_checkout, self._expected_wpt_checkout)
>+            self._test_case.assertEquals(wpt_args, self._expected_wpt_args)
>+
>+    class TestInstance(object):
>+        def __init__(self, options, spawn_wpt_func=None):
>+            self.port = MockPort()
>+            self.host = MockHost()
>+
>+            # In non-test environments, this value is by default set to the WEBKIT_TEST_CHILD_PROCESSES
>+            # env value or, if that's not present, to the default number of child processes. Here we
>+            # manually set it to 4 for consistency, unless the test case specifies it.
>+            if not options.child_processes:
>+                options.child_processes = 4
>+
>+            self.runner = WPTRunner(self.port, self.host, "wptrunner_unittest", options,
>+                WPTRunnerTest.MockTestDownloader, WPTRunnerTest.MockWebDriver.create, spawn_wpt_func)
>+
>+        def prepare_mock_files_for_run(self):
>+            self.host.filesystem.maybe_make_directory(self.port.wpt_metadata_directory())
>+            self.host.filesystem.write_text_file(
>+                "/mock-checkout/WebPlatformTests/MockPort/TestExpectations.json", "{}")
>+            self.host.filesystem.write_text_file(
>+                "/mock-checkout/WebPlatformTests/MockPort/TestManifest.ini", "{}")
>+
>+    def test_prepare_wpt_checkout(self):
>+        # Tests the _prepare_wpt_checkout() method with no WPT checkout specified in options.
>+
>+        options, _ = parse_args([])
>+        instance = WPTRunnerTest.TestInstance(options)
>+
>+        self.assertTrue(instance.runner._prepare_wpt_checkout())
>+
>+        expected_wpt_checkout = "/mock-checkout/WebKitBuild/w3c-tests/web-platform-tests"
>+        self.assertEquals(instance.runner._options.wpt_checkout, expected_wpt_checkout)
>+        self.assertTrue(instance.host.filesystem.isdir(expected_wpt_checkout))
>+
>+    def test_prepare_wpt_checkout_specified_path(self):
>+        # Tests the _prepare_wpt_checkout() method with WPT checkout specified in options.
>+
>+        specified_wpt_checkout = "/mock-path/web-platform-tests"
>+        options, _ = parse_args(["--wpt-checkout", specified_wpt_checkout])
>+        instance = WPTRunnerTest.TestInstance(options)
>+        instance.host.filesystem.maybe_make_directory(specified_wpt_checkout)
>+
>+        self.assertTrue(instance.runner._prepare_wpt_checkout())
>+        self.assertEquals(instance.runner._options.wpt_checkout, specified_wpt_checkout)
>+
>+    def test_generate_metadata_directory(self):
>+        # Tests the _generate_metadata_directory() method. TestExpectations.json file is mocked with
>+        # TEST_EXPECTATIONS_JSON_CONTENT value as contents. Expected metadata is generated under
>+        # /mock-metadata and should correspond to files and files' content in the
>+        # EXPECTED_TEST_MANIFESTS dictionary.
>+
>+        expectations_json_path = "/mock-checkout/WebPlatformTests/MockPort/TestExpectations.json"
>+        metadata_path = "/mock-metadata"
>+
>+        options, _ = parse_args([])
>+        instance = WPTRunnerTest.TestInstance(options)
>+
>+        instance.host.filesystem.write_text_file(
>+            "/mock-checkout/WebPlatformTests/MockPort/TestExpectations.json",
>+            TEST_EXPECTATIONS_JSON_CONTENT)
>+
>+        self.assertTrue(instance.runner._generate_metadata_directory(metadata_path))
>+        for path, content in EXPECTED_TEST_MANIFESTS.items():
>+            manifest_path = instance.host.filesystem.join(metadata_path, path)
>+            self.assertTrue(instance.host.filesystem.isfile(manifest_path))
>+            self.assertEquals(instance.host.filesystem.read_text_file(manifest_path), content)
>+
>+    def test_run(self):
>+        # Tests the run() method. Files are mocked to the point that helper methods don't fail.
>+        # Goal of this test is for the WPT spawn command to match the desired WPT directory and
>+        # arguments. No options or arguments are used.
>+
>+        spawn_wpt_func = WPTRunnerTest.MockSpawnWPT(self,
>+            "/mock-checkout/WebKitBuild/w3c-tests/web-platform-tests",
>+            ["run", "--webkit-port=MockPort", "--processes=4",
>+                "--metadata=/mock-path/mock-wpt-tests-metadata",
>+                "--manifest=/mock-path/mock-wpt-manifest.json",
>+                "--include-manifest=/mock-checkout/WebPlatformTests/MockPort/TestManifest.ini",
>+                "--webdriver-binary=/mock-webdriver/bin/webdriver",
>+                "--binary=/mock-webdriver/bin/browser",
>+                "--binary-arg=webdriver_arg1", "--binary-arg=webdriver_arg2", "webkit"])
>+
>+        options, _ = parse_args([])
>+        instance = WPTRunnerTest.TestInstance(options, spawn_wpt_func)
>+        instance.prepare_mock_files_for_run()
>+
>+        self.assertTrue(instance.runner.run([]))
>+
>+    def test_run_with_specified_options(self):
>+        # Tests the run() method. Files are mocked to the point that helper methods don't fail.
>+        # Goal of this test is for the WPT spawn command to match the desired WPT directory and
>+        # arguments. Custom WPT checkout and child process count are specified. Note that the
>+        # WPT checkout doesn't have an impact on the resulting WPT argument list, as intended.
>+        specified_wpt_checkout = "/mock-path/web-platform-tests"
>+        specified_child_processes = 16
>+
>+        spawn_wpt_func = WPTRunnerTest.MockSpawnWPT(self, specified_wpt_checkout,
>+            ["run", "--webkit-port=MockPort", "--processes=16",
>+                "--metadata=/mock-path/mock-wpt-tests-metadata",
>+                "--manifest=/mock-path/mock-wpt-manifest.json",
>+                "--include-manifest=/mock-checkout/WebPlatformTests/MockPort/TestManifest.ini",
>+                "--webdriver-binary=/mock-webdriver/bin/webdriver",
>+                "--binary=/mock-webdriver/bin/browser",
>+                "--binary-arg=webdriver_arg1", "--binary-arg=webdriver_arg2", "webkit"])
>+
>+        options, _ = parse_args(["--wpt-checkout", specified_wpt_checkout,
>+            "--child-processes", specified_child_processes])
>+        instance = WPTRunnerTest.TestInstance(options, spawn_wpt_func)
>+        instance.prepare_mock_files_for_run()
>+
>+        # Also create the mock WPT checkout directory.
>+        instance.host.filesystem.maybe_make_directory(specified_wpt_checkout)
>+
>+        self.assertTrue(instance.runner.run([]))
>+
>+    def test_run_with_args(self):
>+        # Tests the run() method. Files are mocked to the point that helper methods don't fail.
>+        # Goal of this test is for the WPT spawn command to match the desired WPT directory and
>+        # arguments. A custom two-element argument list is used. It's expected to be appended
>+        # to the resulting WPT argument list.
>+
>+        specified_args = ["test1.html", "test2.html"]
>+
>+        spawn_wpt_func = WPTRunnerTest.MockSpawnWPT(self,
>+            "/mock-checkout/WebKitBuild/w3c-tests/web-platform-tests",
>+            ["run", "--webkit-port=MockPort", "--processes=4",
>+                "--metadata=/mock-path/mock-wpt-tests-metadata",
>+                "--manifest=/mock-path/mock-wpt-manifest.json",
>+                "--include-manifest=/mock-checkout/WebPlatformTests/MockPort/TestManifest.ini",
>+                "--webdriver-binary=/mock-webdriver/bin/webdriver",
>+                "--binary=/mock-webdriver/bin/browser",
>+                "--binary-arg=webdriver_arg1", "--binary-arg=webdriver_arg2", "webkit"] + specified_args)
>+
>+        options, _ = parse_args([])
>+        instance = WPTRunnerTest.TestInstance(options, spawn_wpt_func)
>+        instance.prepare_mock_files_for_run()
>+
>+        self.assertTrue(instance.runner.run(specified_args))
>diff --git a/ChangeLog b/ChangeLog
>index 3557d94ff4277c472852d4bbf7ab076168010e90..9d18f3bb15abc06b8f33eb7e8da8ad3cf6bf2022 100644
>--- a/ChangeLog
>+++ b/ChangeLog
>@@ -1,3 +1,30 @@
>+2018-05-31  Zan Dobersek  <zdobersek@igalia.com>
>+
>+        Add run-web-platform-tests script
>+        https://bugs.webkit.org/show_bug.cgi?id=183356
>+
>+        Reviewed by NOBODY (OOPS!).
>+
>+        Store port-specific test expectations and test manifest files for the
>+        web-platform-tests test runs under the top-level WebPlatformTests
>+        directory.
>+
>+        TestExpectations.json file lists all the failing or disabled tests
>+        or subtests that are to be expected in the test run. This information is
>+        parsed in the run-web-platform-tests script and used to build
>+        test-specific metadata files (ending with .ini) inside a temporary
>+        directory that is then used during the test run. JSON format is used to
>+        follow the WebDriver test suite in how it manages expectations, and to
>+        avoid having to manually manage .ini files for every deviant test case.
>+
>+        TestManifest.ini file is used to fine-tune which tests are to be enabled
>+        by default. For the GTK+ port, we currently disable all tests by default
>+        but then specifically enable tests under the 2dcontext and WebCryptoAPI
>+        directories. This will allow for gradual enabling of further tests.
>+
>+        * WebPlatformTests/gtk/TestExpectations.json: Added.
>+        * WebPlatformTests/gtk/TestManifest.ini: Added.
>+
> 2018-05-28  Carlos Garcia Campos  <cgarcia@igalia.com>
> 
>         Unreviewed. Update OptionsGTK.cmake and NEWS for 2.21.3 release.
>diff --git a/WebPlatformTests/gtk/TestExpectations.json b/WebPlatformTests/gtk/TestExpectations.json
>new file mode 100644
>index 0000000000000000000000000000000000000000..a73b63f3eb779a73a30da69329bfa08ec60d0eaf
>--- /dev/null
>+++ b/WebPlatformTests/gtk/TestExpectations.json
>@@ -0,0 +1,559 @@
>+{
>+    "2dcontext/building-paths/canvas_complexshapes_arcto_001.htm": {
>+        "expected": "FAIL"
>+    },
>+    "2dcontext/building-paths/canvas_complexshapes_beziercurveto_001.htm": {
>+        "expected": "FAIL"
>+    },
>+    "2dcontext/compositing/2d.composite.globalAlpha.canvascopy.html": {
>+        "subtests": {
>+            "Canvas test: 2d.composite.globalAlpha.canvascopy": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/compositing/2d.composite.operation.darker.html": {
>+        "subtests": {
>+            "Canvas test: 2d.composite.operation.darker": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/drawing-images-to-the-canvas/2d.drawImage.animated.apng.html": {
>+        "subtests": {
>+            "drawImage() of an APNG with no poster frame draws the first frame": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/drawing-images-to-the-canvas/2d.drawImage.animated.gif.html": {
>+        "subtests": {
>+            "drawImage() of an animated GIF draws the first frame": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/drawing-images-to-the-canvas/2d.drawImage.animated.poster.html": {
>+        "subtests": {
>+            "drawImage() of an APNG draws the poster frame": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/drawing-images-to-the-canvas/2d.drawImage.zerosource.html": {
>+        "subtests": {
>+            "drawImage with zero-sized source rectangle draws nothing without exception": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/drawing-images-to-the-canvas/2d.drawImage.zerosource.image.html": {
>+        "subtests": {
>+            "drawImage with zero-sized source rectangle from image throws INDEX_SIZE_ERR": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/drawing-images-to-the-canvas/drawimage_canvas.html": {
>+        "subtests": {
>+            "Test scenario 8: sx = 25, sy = 25, sw = 50, sh = 50, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 0,20 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 8: sx = 25, sy = 25, sw = 50, sh = 50, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 20,0 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 8: sx = 25, sy = 25, sw = 50, sh = 50, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 20,20 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 8: sx = 25, sy = 25, sw = 50, sh = 50, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 24,24 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 8: sx = 25, sy = 25, sw = 50, sh = 50, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 0,0 should be black.": { "expected": "FAIL" },
>+            "Test scenario 8: sx = 25, sy = 25, sw = 50, sh = 50, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 0,19 should be black.": { "expected": "FAIL" },
>+            "Test scenario 8: sx = 25, sy = 25, sw = 50, sh = 50, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 19,0 should be black.": { "expected": "FAIL" },
>+            "Test scenario 8: sx = 25, sy = 25, sw = 50, sh = 50, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 19,19 should be black.": { "expected": "FAIL" },
>+            "Test scenario 9: sx = 0, sy = 0, sw = 50, sh = 50, dx = 100, dy = 100, dw = -50, dh = -50 --- Pixel 50,50 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 9: sx = 0, sy = 0, sw = 50, sh = 50, dx = 100, dy = 100, dw = -50, dh = -50 --- Pixel 50,99 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 9: sx = 0, sy = 0, sw = 50, sh = 50, dx = 100, dy = 100, dw = -50, dh = -50 --- Pixel 99,50 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 9: sx = 0, sy = 0, sw = 50, sh = 50, dx = 100, dy = 100, dw = -50, dh = -50 --- Pixel 99,99 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 9: sx = 0, sy = 0, sw = 50, sh = 50, dx = 100, dy = 100, dw = -50, dh = -50 --- Pixel 55,55 should be black.": { "expected": "FAIL" },
>+            "Test scenario 9: sx = 0, sy = 0, sw = 50, sh = 50, dx = 100, dy = 100, dw = -50, dh = -50 --- Pixel 55,94 should be black.": { "expected": "FAIL" },
>+            "Test scenario 9: sx = 0, sy = 0, sw = 50, sh = 50, dx = 100, dy = 100, dw = -50, dh = -50 --- Pixel 94,55 should be black.": { "expected": "FAIL" },
>+            "Test scenario 9: sx = 0, sy = 0, sw = 50, sh = 50, dx = 100, dy = 100, dw = -50, dh = -50 --- Pixel 94,94 should be black.": { "expected": "FAIL" },
>+            "Test scenario 11: sx = 0, sy = 0, sw = 100, sh = 100, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 0,0 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 11: sx = 0, sy = 0, sw = 100, sh = 100, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 1,1 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 11: sx = 0, sy = 0, sw = 100, sh = 100, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 23,23 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 11: sx = 0, sy = 0, sw = 100, sh = 100, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 24,24 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 11: sx = 0, sy = 0, sw = 100, sh = 100, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 3,3 should be black.": { "expected": "FAIL" },
>+            "Test scenario 11: sx = 0, sy = 0, sw = 100, sh = 100, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 3,21 should be black.": { "expected": "FAIL" },
>+            "Test scenario 11: sx = 0, sy = 0, sw = 100, sh = 100, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 21,3 should be black.": { "expected": "FAIL" },
>+            "Test scenario 11: sx = 0, sy = 0, sw = 100, sh = 100, dx = 0, dy = 0, dw = 50, dh = 50 --- Pixel 21,21 should be black.": { "expected": "FAIL" },
>+            "Test scenario 12: sx = -20, sy = -20, sw = 50, sh = 50, dx = 20, dy = 20, dw = 125, dh = 125 --- Pixel 70,70 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 12: sx = -20, sy = -20, sw = 50, sh = 50, dx = 20, dy = 20, dw = 125, dh = 125 --- Pixel 70,99 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 12: sx = -20, sy = -20, sw = 50, sh = 50, dx = 20, dy = 20, dw = 125, dh = 125 --- Pixel 99,70 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 12: sx = -20, sy = -20, sw = 50, sh = 50, dx = 20, dy = 20, dw = 125, dh = 125 --- Pixel 82,82 should be blue.": { "expected": "FAIL" },
>+            "Test scenario 12: sx = -20, sy = -20, sw = 50, sh = 50, dx = 20, dy = 20, dw = 125, dh = 125 --- Pixel 84,84 should be black.": { "expected": "FAIL" },
>+            "Test scenario 12: sx = -20, sy = -20, sw = 50, sh = 50, dx = 20, dy = 20, dw = 125, dh = 125 --- Pixel 84,99 should be black.": { "expected": "FAIL" },
>+            "Test scenario 12: sx = -20, sy = -20, sw = 50, sh = 50, dx = 20, dy = 20, dw = 125, dh = 125 --- Pixel 99,84 should be black.": { "expected": "FAIL" },
>+            "Test scenario 12: sx = -20, sy = -20, sw = 50, sh = 50, dx = 20, dy = 20, dw = 125, dh = 125 --- Pixel 99,99 should be black.": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/drawing-images-to-the-canvas/drawimage_html_image.html": {
>+        "subtests": {
>+            "Test scenario 12: sx = -20, sy = -20, sw = 50, sh = 50, dx = 20, dy = 20, dw = 125, dh = 125 --- Pixel 70,99 should be light purple.": { "expected": "FAIL" },
>+            "Test scenario 12: sx = -20, sy = -20, sw = 50, sh = 50, dx = 20, dy = 20, dw = 125, dh = 125 --- Pixel 99,70 should be light purple.": { "expected": "FAIL" },
>+            "Test scenario 12: sx = -20, sy = -20, sw = 50, sh = 50, dx = 20, dy = 20, dw = 125, dh = 125 --- Pixel 99,99 should be light purple.": { "expected": "FAIL" },
>+            "Test scenario 12: sx = -20, sy = -20, sw = 50, sh = 50, dx = 20, dy = 20, dw = 125, dh = 125 --- Pixel 69,69 should be red.": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/drawing-images-to-the-canvas/drawimage_svg_image_1.html": {
>+        "subtests": {
>+            "Load a 100x100 image to a SVG image and draw it to a 100x100 canvas.": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/drawing-paths-to-the-canvas/canvas_focus_drawCustomFocusRing_001.html": {
>+        "subtests": {
>+            "drawCustomFocusRing must return false for an element that is not focused.": { "expected": "FAIL" },
>+            "drawCustomFocusRing must return false for an element that is not a descendant of the canvas element.": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/drawing-text-to-the-canvas/2d.text.draw.fontface.notinpage.html": {
>+        "subtests": {
>+            "@font-face fonts should work even if they are not used in the page": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsl-1.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsl-1": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsl-2.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsl-2": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsl-3.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsl-3": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsl-4.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsl-4": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsl-5.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsl-5": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsl-6.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsl-6": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsl-7.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsl-7": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsl-8.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsl-8": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsl-9.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsl-9": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsla-1.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsla-1": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsla-2.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsla-2": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsla-3.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsla-3": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsla-4.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsla-4": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsla-5.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsla-5": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsla-6.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsla-6": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsla-7.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsla-7": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsla-8.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsla-8": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-hsla-9.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-hsla-9": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-rgb-1.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-rgb-1": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-rgb-2.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-rgb-2": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-rgb-3.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-rgb-3": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-rgb-4.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-rgb-4": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-rgb-5.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-rgb-5": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-rgb-6.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-rgb-6": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-rgba-1.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-rgba-1": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-rgba-3.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-rgba-3": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-rgba-4.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-rgba-4": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-rgba-5.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-rgba-5": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.fillStyle.parse.css-color-4-rgba-6.html": {
>+        "subtests": {
>+            "Canvas test: 2d.fillStyle.parse.css-color-4-rgba-6": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.pattern.animated.gif.html": {
>+        "subtests": {
>+            "createPattern() of an animated GIF draws the first frame": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/fill-and-stroke-styles/2d.pattern.image.broken.html": {
>+        "subtests": {
>+            "Canvas test: 2d.pattern.image.broken": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/hit-regions/addHitRegions-NotSupportedError-01.html": {
>+        "subtests": {
>+            "fillRect should not affect current default path and NotSupportedError should be thrown.": { "expected": "FAIL" },
>+            "strokeRect should not affect current default path and NotSupportedError should be thrown.": { "expected": "FAIL" },
>+            "fillText should not affect current default path and NotSupportedError should be thrown.": { "expected": "FAIL" },
>+            "strokeText should not affect current default path and NotSupportedError shuld be thrown.": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/hit-regions/hitregions-members-exist.html": {
>+        "subtests": {
>+            "context.addHitRegion Exists": { "expected": "FAIL" },
>+            "context.removeHitRegion Exists": { "expected": "FAIL" },
>+            "context.clearHitRegions Exists": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/image-smoothing/imagesmoothing.html": {
>+        "subtests": {
>+            "Test that imageSmoothingEnabled = false (nearest-neighbor interpolation) works with fillRect and createPattern().": { "expected": "FAIL" },
>+            "Test that imageSmoothingEnabled = false (nearest-neighbor interpolation) works with fill() and createPattern().": { "expected": "FAIL" },
>+            "Test that imageSmoothingEnabled = false (nearest-neighbor interpolation) works with stroke() and createPattern().": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/imagebitmap/createImageBitmap-drawImage.html": {
>+        "subtests": {
>+            "createImageBitmap from an HTMLCanvasElement with negative sw/sh, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an HTMLVideoElement with negative sw/sh, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an HTMLVideoElement from a data URL with negative sw/sh, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a bitmap HTMLImageElement with negative sw/sh, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a vector HTMLImageElement with negative sw/sh, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a bitmap SVGImageElement, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a bitmap SVGImageElement scaled down, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a bitmap SVGImageElement scaled up, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a bitmap SVGImageElement resized, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a bitmap SVGImageElement with negative sw/sh, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a vector SVGImageElement, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a vector SVGImageElement scaled down, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a vector SVGImageElement scaled up, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a vector SVGImageElement resized, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a vector SVGImageElement with negative sw/sh, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an OffscreenCanvas, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an OffscreenCanvas scaled down, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an OffscreenCanvas scaled up, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an OffscreenCanvas resized, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an OffscreenCanvas with negative sw/sh, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an ImageData, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an ImageData scaled down, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an ImageData scaled up, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an ImageData resized, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an ImageData with negative sw/sh, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from an ImageBitmap with negative sw/sh, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a Blob, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a Blob scaled down, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a Blob scaled up, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a Blob resized, and drawImage on the created ImageBitmap": { "expected": "FAIL" },
>+            "createImageBitmap from a Blob with negative sw/sh, and drawImage on the created ImageBitmap": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/imagebitmap/createImageBitmap-invalid-args.html": {
>+        "subtests": {
>+            "createImageBitmap with a an HTMLCanvasElement source and oversized (unallocatable) crop region": { "expected": "FAIL" },
>+            "createImageBitmap with a an HTMLVideoElement source and oversized (unallocatable) crop region": { "expected": "FAIL" },
>+            "createImageBitmap with a an HTMLVideoElement from a data URL source and oversized (unallocatable) crop region": { "expected": "FAIL" },
>+            "createImageBitmap with a a bitmap HTMLImageElement source and oversized (unallocatable) crop region": { "expected": "FAIL" },
>+            "createImageBitmap with a a vector HTMLImageElement source and oversized (unallocatable) crop region": { "expected": "FAIL" },
>+            "createImageBitmap with a a bitmap SVGImageElement source and sw set to 0": { "expected": "FAIL" },
>+            "createImageBitmap with a a bitmap SVGImageElement source and sh set to 0": { "expected": "FAIL" },
>+            "createImageBitmap with a a bitmap SVGImageElement source and oversized (unallocatable) crop region": { "expected": "FAIL" },
>+            "createImageBitmap with a a vector SVGImageElement source and sw set to 0": { "expected": "FAIL" },
>+            "createImageBitmap with a a vector SVGImageElement source and sh set to 0": { "expected": "FAIL" },
>+            "createImageBitmap with a a vector SVGImageElement source and oversized (unallocatable) crop region": { "expected": "FAIL" },
>+            "createImageBitmap with a an OffscreenCanvas source and sw set to 0": { "expected": "FAIL" },
>+            "createImageBitmap with a an OffscreenCanvas source and sh set to 0": { "expected": "FAIL" },
>+            "createImageBitmap with a an OffscreenCanvas source and oversized (unallocatable) crop region": { "expected": "FAIL" },
>+            "createImageBitmap with a an ImageData source and oversized (unallocatable) crop region": { "expected": "FAIL" },
>+            "createImageBitmap with a an ImageBitmap source and oversized (unallocatable) crop region": { "expected": "FAIL" },
>+            "createImageBitmap with a a Blob source and oversized (unallocatable) crop region": { "expected": "FAIL" },
>+            "createImageBitmap with an invalid OffscreenCanvas source.": { "expected": "FAIL" },
>+            "createImageBitmap with an undecodable blob source.": { "expected": "FAIL" },
>+            "createImageBitmap with an available but undecodable image source.": { "expected": "FAIL" },
>+            "createImageBitmap with an available but zero height image source.": { "expected": "FAIL" },
>+            "createImageBitmap with an available but zero width image source.": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/imagebitmap/createImageBitmap-origin.sub.html": {
>+        "subtests": {
>+            "cross-origin SVGImageElement": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/imagebitmap/createImageBitmap-sizeOverflow.html": {
>+        "subtests": {
>+            "createImageBitmap does not crash or reject the promise when passing very large sx": { "expected": "FAIL" },
>+            "createImageBitmap does not crash or reject the promise when passing very large sy": { "expected": "FAIL" },
>+            "createImageBitmap does not crash or reject the promise when passing very large sw": { "expected": "FAIL" },
>+            "createImageBitmap does not crash or reject the promise when passing very large sh": { "expected": "FAIL" },
>+            "createImageBitmap does not crash or reject the promise when passing very large sx, sy, sw and sh": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/pixel-manipulation/2d.imageData.create2.double.html": {
>+        "subtests": {
>+            "createImageData(w, h) double is converted to long": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/pixel-manipulation/2d.imageData.create2.zero.html": {
>+        "subtests": {
>+            "createImageData(sw, sh) throws INDEX_SIZE_ERR if size is zero": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/pixel-manipulation/2d.imageData.get.double.html": {
>+        "subtests": {
>+            "createImageData(w, h) double is converted to long": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/pixel-manipulation/2d.imageData.get.zero.html": {
>+        "subtests": {
>+            "getImageData() throws INDEX_SIZE_ERR if size is zero": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/pixel-manipulation/2d.imageData.object.ctor.array.html": {
>+        "subtests": {
>+            "ImageData has a usable constructor": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/text-styles/2d.text.draw.baseline.bottom.html": {
>+        "subtests": {
>+            "textBaseline bottom is the bottom of the em square (not the bounding box)": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/text-styles/2d.text.draw.baseline.hanging.html": {
>+        "subtests": {
>+            "Canvas test: 2d.text.draw.baseline.hanging": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/text-styles/2d.text.draw.baseline.ideographic.html": {
>+        "subtests": {
>+            "Canvas test: 2d.text.draw.baseline.ideographic": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/text-styles/2d.text.draw.baseline.middle.html": {
>+        "subtests": {
>+            "textBaseline middle is the middle of the em square (not the bounding box)": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/text-styles/2d.text.draw.baseline.top.html": {
>+        "subtests": {
>+            "textBaseline top is the top of the em square (not the bounding box)": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/text-styles/2d.text.draw.space.collapse.end.html": {
>+        "subtests": {
>+            "Space characters at the end of a line are collapsed (per CSS)": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/text-styles/2d.text.draw.space.collapse.other.html": {
>+        "subtests": {
>+            "Space characters are converted to U+0020, and collapsed (per CSS)": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/text-styles/2d.text.draw.space.collapse.space.html": {
>+        "subtests": {
>+            "Space characters are converted to U+0020, and collapsed (per CSS)": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/text-styles/2d.text.draw.space.collapse.start.html": {
>+        "subtests": {
>+            "Space characters at the start of a line are collapsed (per CSS)": { "expected": "FAIL" }
>+        }
>+    },
>+    "2dcontext/text-styles/2d.text.measure.width.space.html": {
>+        "subtests": {
>+            "Space characters are converted to U+0020 and collapsed (per CSS)": { "expected": "FAIL" }
>+        }
>+    },
>+
>+    "WebCryptoAPI/derive_bits_keys/ecdh_bits.https.worker.js": {
>+        "test_name": "ecdh_bits.https.worker.html",
>+        "subtests": {
>+            "P-256 non-multiple of 8 bits": { "expected": "FAIL" },
>+            "P-384 non-multiple of 8 bits": { "expected": "FAIL" },
>+            "P-521 non-multiple of 8 bits": { "expected": "FAIL" }
>+        }
>+    },
>+    "WebCryptoAPI/derive_bits_keys/hkdf.https.worker.js": {
>+        "test_name": "hkdf.https.worker.html",
>+        "disabled": "Lousy HKDF support"
>+    },
>+    "WebCryptoAPI/derive_bits_keys/pbkdf2.https.worker.js": {
>+        "test_name": "pbkdf2.https.worker.html",
>+        "disabled": "Lousy PBKDF2 support"
>+    },
>+    "WebCryptoAPI/derive_bits_keys/test_ecdh_bits.https.html": {
>+        "subtests": {
>+            "P-256 non-multiple of 8 bits": { "expected": "FAIL" },
>+            "P-384 non-multiple of 8 bits": { "expected": "FAIL" },
>+            "P-521 non-multiple of 8 bits": { "expected": "FAIL" }
>+        }
>+    },
>+    "WebCryptoAPI/derive_bits_keys/test_hkdf.https.html": {
>+        "disabled": "Lousy HKDF support"
>+    },
>+    "WebCryptoAPI/derive_bits_keys/test_pbkdf2_empty_empty.https.html": {
>+        "disabled": "Lousy PBKDF2 support"
>+    },
>+    "WebCryptoAPI/derive_bits_keys/test_pbkdf2_empty_long.https.html": {
>+        "disabled": "Lousy PBKDF2 support"
>+    },
>+    "WebCryptoAPI/derive_bits_keys/test_pbkdf2_empty_short.https.html": {
>+        "disabled": "Lousy PBKDF2 support"
>+    },
>+    "WebCryptoAPI/derive_bits_keys/test_pbkdf2_long_empty.https.html": {
>+        "disabled": "Lousy PBKDF2 support"
>+    },
>+    "WebCryptoAPI/derive_bits_keys/test_pbkdf2_short_empty.https.html": {
>+        "disabled": "Lousy PBKDF2 support"
>+    },
>+    "WebCryptoAPI/encrypt_decrypt/aes_gcm.https.worker.js": {
>+        "test_name": "aes_gcm.https.worker.html",
>+        "subtests": {
>+            "AES-GCM 128-bit key, illegal tag length 256-bits": { "expected": "FAIL" },
>+            "AES-GCM 192-bit key, illegal tag length 256-bits": { "expected": "FAIL" },
>+            "AES-GCM 256-bit key, illegal tag length 256-bits": { "expected": "FAIL" },
>+            "AES-GCM 128-bit key, illegal tag length 256-bits decryption": { "expected": "FAIL" },
>+            "AES-GCM 192-bit key, illegal tag length 256-bits decryption": { "expected": "FAIL" },
>+            "AES-GCM 256-bit key, illegal tag length 256-bits decryption": { "expected": "FAIL" }
>+        }
>+    },
>+    "WebCryptoAPI/encrypt_decrypt/test_aes_gcm.https.html": {
>+        "subtests": {
>+            "AES-GCM 128-bit key, illegal tag length 256-bits": { "expected": "FAIL" },
>+            "AES-GCM 192-bit key, illegal tag length 256-bits": { "expected": "FAIL" },
>+            "AES-GCM 256-bit key, illegal tag length 256-bits": { "expected": "FAIL" },
>+            "AES-GCM 128-bit key, illegal tag length 256-bits decryption": { "expected": "FAIL" },
>+            "AES-GCM 192-bit key, illegal tag length 256-bits decryption": { "expected": "FAIL" },
>+            "AES-GCM 256-bit key, illegal tag length 256-bits decryption": { "expected": "FAIL" }
>+        }
>+    },
>+    "WebCryptoAPI/idlharness.https.worker.js": {
>+        "test_name": "idlharness.https.worker.html",
>+        "subtests": {
>+            "ArrayBuffer interface: existence and properties of interface object": { "expected": "FAIL" }
>+        }
>+    },
>+    "WebCryptoAPI/secure_context/crypto-subtle-non-secure-context-not-available.sub.html": {
>+        "subtests": {
>+            "Non-secure context window does not have access to crypto.subtle": { "expected": "FAIL" },
>+            "Non-secure context worker does not have access to crypto.subtle": { "expected": "FAI:" }
>+        }
>+    },
>+    "WebCryptoAPI/wrapKey_unwrapKey/test_wrapKey_unwrapKey.https.html": {
>+        "subtests": {
>+            "Can wrap and unwrap ECDSA public key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap ECDSA private key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap ECDSA private key keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap ECDH public key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap ECDH private key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap ECDH private key keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-CTR keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-CTR keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-CBC keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-CBC keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-GCM keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-GCM keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-KW keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-KW keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap HMAC keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap HMAC keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap RSASSA-PKCS1-v1_5 private key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap RSASSA-PKCS1-v1_5 private key keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap RSA-PSS private key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap RSA-PSS private key keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap RSA-OAEP private key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap RSA-OAEP private key keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" }
>+        }
>+    },
>+    "WebCryptoAPI/wrapKey_unwrapKey/wrapKey_unwrapKey.https.worker.js": {
>+        "test_name": "wrapKey_unwrapKey.https.worker.html",
>+        "subtests": {
>+            "Can wrap and unwrap ECDSA public key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap ECDSA private key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap ECDSA private key keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap ECDH public key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap ECDH private key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap ECDH private key keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-CTR keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-CTR keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-CBC keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-CBC keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-GCM keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-GCM keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-KW keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap AES-KW keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap HMAC keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap HMAC keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap RSASSA-PKCS1-v1_5 private key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap RSASSA-PKCS1-v1_5 private key keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap RSA-PSS private key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap RSA-PSS private key keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap RSA-OAEP private key keys using jwk and AES-KW": { "expected": "FAIL" },
>+            "Can wrap and unwrap RSA-OAEP private key keys as non-extractable using jwk and AES-KW": { "expected": "FAIL" }
>+        }
>+    }
>+}
>+
>diff --git a/WebPlatformTests/gtk/TestManifest.ini b/WebPlatformTests/gtk/TestManifest.ini
>new file mode 100644
>index 0000000000000000000000000000000000000000..69fb98a30d9ce18b8e078a0d45e727f7890b0c36
>--- /dev/null
>+++ b/WebPlatformTests/gtk/TestManifest.ini
>@@ -0,0 +1,7 @@
>+skip: true
>+
>+[2dcontext]
>+    skip: false
>+
>+[WebCryptoAPI]
>+    skip: false

Actions: View | Formatted Diff | Diff

Attachments on bug 183356: 335070 | 338095 | 338410 | 339209 | 341651 | 342519