Bug 133852

Summary: W3C test importer should detect manual and reference tests according their filenames
Product: WebKit Reporter: youenn fablet <youennf>
Component: Tools / TestsAssignee: youenn fablet <youennf>
Status: RESOLVED FIXED    
Severity: Normal CC: bjonesbe, commit-queue, glenn, rhauck, rniwa
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch
none
Patch
none
Patch
none
Adding missing reference_support_info field
none
Improved readability and added a unit test none

Description youenn fablet 2014-06-13 06:56:02 PDT
W3C test importer does not currently detect W3C manual tests (names suffixed with -manual.htm).
It classifies wrongly most of them as JS tests since they include testharness.js/testharnessreport.js links.
Comment 1 youenn fablet 2014-06-13 07:07:38 PDT
Created attachment 233047 [details]
Patch
Comment 2 youenn fablet 2014-09-15 02:29:24 PDT
Created attachment 238106 [details]
Patch
Comment 3 youenn fablet 2014-09-15 02:46:01 PDT
Created attachment 238108 [details]
Patch
Comment 4 youenn fablet 2014-09-16 01:47:57 PDT
Created attachment 238160 [details]
Adding missing reference_support_info field
Comment 5 Bem Jones-Bey 2014-09-22 10:40:31 PDT
Rebecca, what do you think?
Comment 6 Rebecca Hauck 2014-09-22 11:01:20 PDT
(In reply to comment #5)
> Rebecca, what do you think?

AFAIK, the CSSWG tests do not make use of the -manual suffix. That may only exist over on the web-platform-tests repo. In CSS land, if it doesn't have a testharness link or a rel=match link, then it's a manual test.  Also, the CSSWG has a metadata flag for manual tests, called 'interact'. 

For example:
<meta name="flags" content="interact" />

See:

http://testthewebforward.org/docs/test-templates.html#requirement-flags


Just curious though, why would we ever want any manual tests to be imported into WebKit?
Comment 7 youenn fablet 2014-09-22 12:43:43 PDT
(In reply to comment #6)
> (In reply to comment #5)
> > Rebecca, what do you think?
> 
> AFAIK, the CSSWG tests do not make use of the -manual suffix. That may only exist over on the web-platform-tests repo. In CSS land, if it doesn't have a testharness link or a rel=match link, then it's a manual test.  Also, the CSSWG has a metadata flag for manual tests, called 'interact'. 
> 
> For example:
> <meta name="flags" content="interact" />

Thanks for that information.
At some point, it should be good for all W3C test suites to be consistent with that respect.
I will send a mail to public-test-infra@w3.org

> See:
> 
> http://testthewebforward.org/docs/test-templates.html#requirement-flags
> 
> 
> Just curious though, why would we ever want any manual tests to be imported into WebKit?

Purpose of this patch is actually to avoid manual  test import. 
To do so, we need to correctly identify them.
Applying CSS rules on WPT leads to importing WPT manual tests.

The same applies to ref tests: in WPT, not all ref tests have a <link rel="match">.
Comment 8 youenn fablet 2014-09-23 01:16:29 PDT
Created attachment 238521 [details]
Improved readability and added a unit test
Comment 9 Bem Jones-Bey 2014-09-23 10:34:04 PDT
Comment on attachment 238521 [details]
Improved readability and added a unit test

r=me
Comment 10 WebKit Commit Bot 2014-09-23 11:09:08 PDT
Comment on attachment 238521 [details]
Improved readability and added a unit test

Clearing flags on attachment: 238521

Committed r173880: <http://trac.webkit.org/changeset/173880>
Comment 11 WebKit Commit Bot 2014-09-23 11:09:14 PDT
All reviewed patches have been landed.  Closing bug.