Bug 134766 - Add support for automated W3C conformance testing
Summary: Add support for automated W3C conformance testing
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: youenn fablet
URL:
Keywords:
Depends on: 134764 136848
Blocks: 134767
  Show dependency tree / graph
 
Reported: 2014-07-09 07:01 PDT by youenn fablet
Modified: 2016-03-10 12:12 PST (History)
8 users (show)

See Also:

Attachments
Patch (21.44 KB, patch)
2014-07-09 07:07 PDT, youenn fablet 		no flags Details | Formatted Diff | Diff
Adding static files (25.55 KB, patch)
2014-07-09 07:17 PDT, youenn fablet 		no flags Details | Formatted Diff | Diff
Start adding support for ref tests (29.28 KB, patch)
2014-07-16 11:35 PDT, youenn fablet 		no flags Details | Formatted Diff | Diff
Support of all tests, result generated in WebKitBuild sub-folder (37.52 KB, patch)
2014-09-16 02:47 PDT, youenn fablet 		no flags Details | Formatted Diff | Diff
updated according 136848 (38.44 KB, patch)
2014-09-16 03:06 PDT, youenn fablet 		no flags Details | Formatted Diff | Diff
Rebasing (35.33 KB, patch)
2014-12-23 07:21 PST, youenn fablet 		no flags Details | Formatted Diff | Diff
Patch (34.62 KB, patch)
2014-12-23 15:37 PST, youenn fablet 		no flags Details | Formatted Diff | Diff
Adding CSS testsuite support (38.88 KB, patch)
2015-01-13 07:45 PST, youenn fablet 		no flags Details | Formatted Diff | Diff
Rebasing against new TestRepositories description (39.50 KB, patch)
2015-03-02 09:24 PST, youenn fablet 		no flags Details | Formatted Diff | Diff
Refreshing and adding unit tests (46.05 KB, patch)
2015-03-30 06:27 PDT, youenn fablet 		no flags Details | Formatted Diff | Diff
Show Obsolete (9) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description youenn fablet 2014-07-09 07:01:15 PDT 
Automated W3C conformance testing based on run-webkit-tests would enable assessing how WebKit complies with W3C test suite. It could also serve as a tool to ease W3C test suite import witihn WebKit
Comment 1 youenn fablet 2014-07-09 07:07:13 PDT 
Created attachment 234635 [details]
Patch
Comment 2 youenn fablet 2014-07-09 07:17:22 PDT 
Created attachment 234637 [details]
Adding static files
Comment 3 youenn fablet 2014-07-16 11:35:47 PDT 
Created attachment 235012 [details]
Start adding support for ref tests
Comment 4 youenn fablet 2014-09-16 02:47:02 PDT 
Created attachment 238166 [details]
Support of all tests, result generated in WebKitBuild sub-folder
Comment 5 youenn fablet 2014-09-16 03:06:52 PDT 
Created attachment 238169 [details]
updated according 136848
Comment 6 youenn fablet 2014-12-23 07:21:13 PST 
Created attachment 243675 [details]
Rebasing
Comment 7 WebKit Commit Bot 2014-12-23 07:23:55 PST 
Attachment 243675 [details] did not pass style-queue:


ERROR: Tools/Scripts/webkitpy/w3c/run_w3c_conformance_tests.py:43:  No name 'web_platform_test_importer' in module 'webkitpy.w3c'  [pylint/E0611] [5]
Total errors found: 1 in 7 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 8 youenn fablet 2014-12-23 15:37:10 PST 
Created attachment 243703 [details]
Patch
Comment 9 Bem Jones-Bey 2015-01-06 08:03:49 PST 
Comment on attachment 243703 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=243703&action=review

> LayoutTests/imported/w3c/resources/w3c_conformance_results.html:156
> +            (results.tests_directory ? results.tests_directory : "imported-tests")

Your indentation is all confused at a bunch of places in this file. Maybe you're mixing tabs and spaces?
Comment 10 youenn fablet 2015-01-06 13:13:45 PST 
(In reply to comment #9)
> Comment on attachment 243703 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=243703&action=review
> 
> > LayoutTests/imported/w3c/resources/w3c_conformance_results.html:156
> > +            (results.tests_directory ? results.tests_directory : "imported-tests")
> 
> Your indentation is all confused at a bunch of places in this file. Maybe
> you're mixing tabs and spaces?

Woups, I'll clean that.
I will also add the possibility to filter tests according result and add bug-filing links
Comment 11 youenn fablet 2015-01-13 07:45:59 PST 
Created attachment 244513 [details]
Adding CSS testsuite support
Comment 12 youenn fablet 2015-01-13 11:40:47 PST 
(In reply to comment #11)
> Created attachment 244513 [details]
> Adding CSS testsuite support

This patch also generates two files, one to update the whitelist used to import W3C tests and one to update the test expectations (to skip infrastructure folders).
Comment 13 youenn fablet 2015-03-02 09:24:10 PST 
Created attachment 247675 [details]
Rebasing against new TestRepositories description
Comment 14 youenn fablet 2015-03-30 06:27:03 PDT 
Created attachment 249727 [details]
Refreshing and adding unit tests
Comment 15 Brent Fulgham 2016-03-10 10:41:32 PST 
Comment on attachment 249727 [details]
Refreshing and adding unit tests

This seems like a really good tool!
Comment 16 Brent Fulgham 2016-03-10 10:42:37 PST 
I can't get EWS to process this file. Youenn, would you mind rebaselining the patch and uploading it again?
Comment 17 youenn fablet 2016-03-10 12:01:52 PST 
(In reply to comment #16)
> I can't get EWS to process this file. Youenn, would you mind rebaselining
> the patch and uploading it again?

The patch is old and webkitpy infrastructure changed a bit.
Time permitting, I will look at it again.