Bug 134766

Summary:

Add support for automated W3C conformance testing

Product:

WebKit

Reporter:

youenn fablet <youennf>

Component:

Tools / Tests

Assignee:

youenn fablet <youennf>

Status:

NEW

Severity:

Normal

CC:

agouaillard, ap, bfulgham, bjonesbe, commit-queue, glenn, rhauck, rniwa

Priority:

Version:

528+ (Nightly build)

Hardware:

Unspecified

OS:

Unspecified

Bug Depends on:

134764, 136848

Bug Blocks:

134767

Attachments:

Description	Flags
Patch	none
Adding static files	none
Start adding support for ref tests	none
Support of all tests, result generated in WebKitBuild sub-folder	none
updated according 136848	none
Rebasing	none
Patch	none
Adding CSS testsuite support	none
Rebasing against new TestRepositories description	none
Refreshing and adding unit tests	none

youenn fablet

Reported 2014-07-09 07:01:15 PDT

Automated W3C conformance testing based on run-webkit-tests would enable assessing how WebKit complies with W3C test suite. It could also serve as a tool to ease W3C test suite import witihn WebKit

Attachments
Patch (21.44 KB, patch) 2014-07-09 07:07 PDT, youenn fablet	no flags	Details Formatted Diff Diff
Adding static files (25.55 KB, patch) 2014-07-09 07:17 PDT, youenn fablet	no flags	Details Formatted Diff Diff
Start adding support for ref tests (29.28 KB, patch) 2014-07-16 11:35 PDT, youenn fablet	no flags	Details Formatted Diff Diff
Support of all tests, result generated in WebKitBuild sub-folder (37.52 KB, patch) 2014-09-16 02:47 PDT, youenn fablet	no flags	Details Formatted Diff Diff
updated according 136848 (38.44 KB, patch) 2014-09-16 03:06 PDT, youenn fablet	no flags	Details Formatted Diff Diff
Rebasing (35.33 KB, patch) 2014-12-23 07:21 PST, youenn fablet	no flags	Details Formatted Diff Diff
Patch (34.62 KB, patch) 2014-12-23 15:37 PST, youenn fablet	no flags	Details Formatted Diff Diff
Adding CSS testsuite support (38.88 KB, patch) 2015-01-13 07:45 PST, youenn fablet	no flags	Details Formatted Diff Diff
Rebasing against new TestRepositories description (39.50 KB, patch) 2015-03-02 09:24 PST, youenn fablet	no flags	Details Formatted Diff Diff
Refreshing and adding unit tests (46.05 KB, patch) 2015-03-30 06:27 PDT, youenn fablet	no flags	Details Formatted Diff Diff
Show Obsolete (9) View All Add attachment proposed patch, testcase, etc.

youenn fablet

Comment 1 2014-07-09 07:07:13 PDT

Created attachment 234635 [details] Patch

youenn fablet

Comment 2 2014-07-09 07:17:22 PDT

Created attachment 234637 [details] Adding static files

youenn fablet

Comment 3 2014-07-16 11:35:47 PDT

Created attachment 235012 [details] Start adding support for ref tests

youenn fablet

Comment 4 2014-09-16 02:47:02 PDT

Created attachment 238166 [details] Support of all tests, result generated in WebKitBuild sub-folder

youenn fablet

Comment 5 2014-09-16 03:06:52 PDT

Created attachment 238169 [details] updated according 136848

youenn fablet

Comment 6 2014-12-23 07:21:13 PST

Created attachment 243675 [details] Rebasing

WebKit Commit Bot

Comment 7 2014-12-23 07:23:55 PST

Attachment 243675 [details] did not pass style-queue: ERROR: Tools/Scripts/webkitpy/w3c/run_w3c_conformance_tests.py:43: No name 'web_platform_test_importer' in module 'webkitpy.w3c' [pylint/E0611] [5] Total errors found: 1 in 7 files If any of these errors are false positives, please file a bug against check-webkit-style.

youenn fablet

Comment 8 2014-12-23 15:37:10 PST

Created attachment 243703 [details] Patch

Bem Jones-Bey

Comment 9 2015-01-06 08:03:49 PST

Comment on attachment 243703 [details] Patch View in context: https://bugs.webkit.org/attachment.cgi?id=243703&action=review > LayoutTests/imported/w3c/resources/w3c_conformance_results.html:156 > + (results.tests_directory ? results.tests_directory : "imported-tests") Your indentation is all confused at a bunch of places in this file. Maybe you're mixing tabs and spaces?

youenn fablet

Comment 10 2015-01-06 13:13:45 PST

(In reply to comment #9) > Comment on attachment 243703 [details] > Patch > > View in context: > https://bugs.webkit.org/attachment.cgi?id=243703&action=review > > > LayoutTests/imported/w3c/resources/w3c_conformance_results.html:156 > > + (results.tests_directory ? results.tests_directory : "imported-tests") > > Your indentation is all confused at a bunch of places in this file. Maybe > you're mixing tabs and spaces? Woups, I'll clean that. I will also add the possibility to filter tests according result and add bug-filing links

youenn fablet

Comment 11 2015-01-13 07:45:59 PST

Created attachment 244513 [details] Adding CSS testsuite support

youenn fablet

Comment 12 2015-01-13 11:40:47 PST

(In reply to comment #11) > Created attachment 244513 [details] > Adding CSS testsuite support This patch also generates two files, one to update the whitelist used to import W3C tests and one to update the test expectations (to skip infrastructure folders).

youenn fablet

Comment 13 2015-03-02 09:24:10 PST

Created attachment 247675 [details] Rebasing against new TestRepositories description

youenn fablet

Comment 14 2015-03-30 06:27:03 PDT

Created attachment 249727 [details] Refreshing and adding unit tests

Brent Fulgham

Comment 15 2016-03-10 10:41:32 PST

Comment on attachment 249727 [details] Refreshing and adding unit tests This seems like a really good tool!

Brent Fulgham

Comment 16 2016-03-10 10:42:37 PST

I can't get EWS to process this file. Youenn, would you mind rebaselining the patch and uploading it again?

youenn fablet

Comment 17 2016-03-10 12:01:52 PST

(In reply to comment #16) > I can't get EWS to process this file. Youenn, would you mind rebaselining > the patch and uploading it again? The patch is old and webkitpy infrastructure changed a bit. Time permitting, I will look at it again.

Note You need to log in before you can comment on or make changes to this bug.