Bug 249720 – import-w3c-tests appears to create a redundant *-expected.html file for each *-ref.html file

• Status: RESOLVED
• Resolution: DUPLICATE
• Component: Tools / Tests

[David Kilzer (:ddkilzer)]

import-w3c-tests appears to create a redundant *-expected.html file for each *-ref.html file.

In Bug 249691, I imported one new WPT (dom/xslt/sort.html), but three files were added:

* LayoutTests/imported/w3c/web-platform-tests/dom/xslt/sort-expected.html: Add.
* LayoutTests/imported/w3c/web-platform-tests/dom/xslt/sort-ref.html: Add.
* LayoutTests/imported/w3c/web-platform-tests/dom/xslt/sort.html: Add.

The `sort-expected.html` file doesn't exist upstream, and appears to have been created by `import-w3c-tests` in Tools/Scripts/webkitpy/w3c/test_importer.py (I haven't debugged this--it's just a guess that this code is responsible):

```
                if 'reference' in test_info.keys():
                    reftests += 1
                    total_tests += 1
                    test_basename = self.filesystem.basename(test_info['test'])

                    # Add the ref file, following WebKit style.
                    # FIXME: Ideally we'd support reading the metadata
                    # directly rather than relying  on a naming convention.
                    # Using a naming convention creates duplicate copies of the
                    # reference files.
                    ref_file = self.filesystem.splitext(test_basename)[0] + '-expected'
                    if 'type' in test_info and test_info['type'] == 'mismatch':
                        ref_file += '-mismatch'
                    ref_file += self.filesystem.splitext(test_info['reference'])[1]

                    copy_list.append({'src': test_info['reference'], 'dest': ref_file, 'reference_support_info': test_info['reference_support_info']})
                    copy_list.append({'src': test_info['test'], 'dest': filename})
```

There are 293 `*-ref.html` files without corresponding `*-expected.html` files:

$ for F in `find LayoutTests/imported/w3c/web-platform-tests -name \*-ref.html`; do G="`echo $F | sed -e 's/-ref.html/-expected.html/'`"; if [ ! -e $G ]; then ls -la $F; fi; done | wc -l
     293

While there are 1947 `*-ref.html` files with a `*-expected.html` file:

$ for F in `find LayoutTests/imported/w3c/web-platform-tests -name \*-ref.html`; do G="`echo $F | sed -e 's/-ref.html/-expected.html/'`"; if [ -e $G ]; then ls -la $G; fi; done | wc -l
    1947

In most cases the `*-ref.html` and `*-expected.html` files are the same, but not always:

$ cksum LayoutTests/imported/w3c/web-platform-tests/quirks/body-fills-html-quirk-{expected,ref}.html
707962029 242 LayoutTests/imported/w3c/web-platform-tests/quirks/body-fills-html-quirk-expected.html
3896252966 153 LayoutTests/imported/w3c/web-platform-tests/quirks/body-fills-html-quirk-ref.html

[Radar WebKit Bug Importer]

<rdar://problem/103600959>

[David Kilzer (:ddkilzer)]

Anyone know what the "expected" (*pun*) behavior is when a `*-ref.html` file exists?

I'm guessing the `*-expected.html` files can be deleted, and that the `import-w3c-tests` script shouldn't be creating them.  Is that correct?

[David Kilzer (:ddkilzer)]

And for "reference" (*pun*), there are no `*-expected.html` files for `*-ref.html` files in the W3C web-platform-tests repository:

```
$ for F in `find WebKitBuild/w3c-tests/web-platform-tests -name \*-ref.html`; do G="`echo $F | sed -e 's/-ref.html/-expected.html/'`"; if [ -e $G ]; then ls -la $G; fi; done | wc -l
       0
```

[David Kilzer (:ddkilzer)]

Pull request: https://github.com/WebKit/WebKit/pull/7971

[David Kilzer (:ddkilzer)]

Dupe of this:

Bug 203784: Add support for finding test references via <link rel="match"> and use that in WPT

*** This bug has been marked as a duplicate of bug 203784 ***