Bug 133852 - W3C test importer should detect manual and reference tests according their filenames
Summary: W3C test importer should detect manual and reference tests according their fi...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: youenn fablet
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-06-13 06:56 PDT by youenn fablet
Modified: 2014-09-23 11:09 PDT (History)
5 users (show)

See Also:


Attachments
Patch (3.13 KB, patch)
2014-06-13 07:07 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch (3.73 KB, patch)
2014-09-15 02:29 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Patch (3.93 KB, patch)
2014-09-15 02:46 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Adding missing reference_support_info field (3.93 KB, patch)
2014-09-16 01:47 PDT, youenn fablet
no flags Details | Formatted Diff | Diff
Improved readability and added a unit test (4.86 KB, patch)
2014-09-23 01:16 PDT, youenn fablet
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description youenn fablet 2014-06-13 06:56:02 PDT
W3C test importer does not currently detect W3C manual tests (names suffixed with -manual.htm).
It classifies wrongly most of them as JS tests since they include testharness.js/testharnessreport.js links.
Comment 1 youenn fablet 2014-06-13 07:07:38 PDT
Created attachment 233047 [details]
Patch
Comment 2 youenn fablet 2014-09-15 02:29:24 PDT
Created attachment 238106 [details]
Patch
Comment 3 youenn fablet 2014-09-15 02:46:01 PDT
Created attachment 238108 [details]
Patch
Comment 4 youenn fablet 2014-09-16 01:47:57 PDT
Created attachment 238160 [details]
Adding missing reference_support_info field
Comment 5 Bem Jones-Bey 2014-09-22 10:40:31 PDT
Rebecca, what do you think?
Comment 6 Rebecca Hauck 2014-09-22 11:01:20 PDT
(In reply to comment #5)
> Rebecca, what do you think?

AFAIK, the CSSWG tests do not make use of the -manual suffix. That may only exist over on the web-platform-tests repo. In CSS land, if it doesn't have a testharness link or a rel=match link, then it's a manual test.  Also, the CSSWG has a metadata flag for manual tests, called 'interact'. 

For example:
<meta name="flags" content="interact" />

See:

http://testthewebforward.org/docs/test-templates.html#requirement-flags


Just curious though, why would we ever want any manual tests to be imported into WebKit?
Comment 7 youenn fablet 2014-09-22 12:43:43 PDT
(In reply to comment #6)
> (In reply to comment #5)
> > Rebecca, what do you think?
> 
> AFAIK, the CSSWG tests do not make use of the -manual suffix. That may only exist over on the web-platform-tests repo. In CSS land, if it doesn't have a testharness link or a rel=match link, then it's a manual test.  Also, the CSSWG has a metadata flag for manual tests, called 'interact'. 
> 
> For example:
> <meta name="flags" content="interact" />

Thanks for that information.
At some point, it should be good for all W3C test suites to be consistent with that respect.
I will send a mail to public-test-infra@w3.org

> See:
> 
> http://testthewebforward.org/docs/test-templates.html#requirement-flags
> 
> 
> Just curious though, why would we ever want any manual tests to be imported into WebKit?

Purpose of this patch is actually to avoid manual  test import. 
To do so, we need to correctly identify them.
Applying CSS rules on WPT leads to importing WPT manual tests.

The same applies to ref tests: in WPT, not all ref tests have a <link rel="match">.
Comment 8 youenn fablet 2014-09-23 01:16:29 PDT
Created attachment 238521 [details]
Improved readability and added a unit test
Comment 9 Bem Jones-Bey 2014-09-23 10:34:04 PDT
Comment on attachment 238521 [details]
Improved readability and added a unit test

r=me
Comment 10 WebKit Commit Bot 2014-09-23 11:09:08 PDT
Comment on attachment 238521 [details]
Improved readability and added a unit test

Clearing flags on attachment: 238521

Committed r173880: <http://trac.webkit.org/changeset/173880>
Comment 11 WebKit Commit Bot 2014-09-23 11:09:14 PDT
All reviewed patches have been landed.  Closing bug.