Now that wptserve can be activated within WebKit test infrastructure, we should populate LayoutTests/imported/w3c/web-platform-tests with wptserve implementation and related infrastructure folders.
Created attachment 245443[details]
Archive of layout-test-results from ews103 for mac-mavericks
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103 Port: mac-mavericks Platform: Mac OS X 10.9.5
(In reply to comment #3)
> Created attachment 245443[details]
> Archive of layout-test-results from ews103 for mac-mavericks
>
> The attached test failures were seen while running run-webkit-tests on the
> mac-ews.
> Bot: ews103 Port: mac-mavericks Platform: Mac OS X 10.9.5
WPT server seems to launch correctly on Mac-WK2 but not Mac WK1 bots.
The trace is the following:
Traceback (most recent call last):
File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_launcher.py", line 14, in <module>
import serve as WebPlatformTestServer
File "./serve.py", line 16, in <module>
from tools.scripts import _env
ImportError: No module named tools.scripts
I am not sure of the exact solution.
Created attachment 245486[details]
Archive of layout-test-results from ews101 for mac-mavericks
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews101 Port: mac-mavericks Platform: Mac OS X 10.9.5
Created attachment 245533[details]
Archive of layout-test-results from ews102 for mac-mavericks
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews102 Port: mac-mavericks Platform: Mac OS X 10.9.5
Created attachment 245543[details]
Archive of layout-test-results from ews103 for mac-mavericks
The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103 Port: mac-mavericks Platform: Mac OS X 10.9.5
(In reply to comment #14)
> Created attachment 245543[details]
> Archive of layout-test-results from ews103 for mac-mavericks
>
> The attached test failures were seen while running run-webkit-tests on the
> mac-ews.
> Bot: ews103 Port: mac-mavericks Platform: Mac OS X 10.9.5
According the WPT log below, the LayoutTests/imported/w3c/web-platform-tests/tools/__init__.py is missing for the Mac WK1 bot, hence the inability to import wpt modules.
Could it be that empty files are not created when applying the patch for this bot?
WPT log is:
DEBUG:web-platform-test-launcher:sys.path is: ['/Volumes/Data/EWS/WebKit/LayoutTests/imported/w3c/web-platform-tests', '/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/servers', '/Volumes/Data/EWS/WebKit/Tools/Scripts', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python27.zip', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-darwin', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/plat-mac/lib-scriptpackages', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-tk', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-old', '/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/lib-dynload', '/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/PyObjC', '/Library/Python/2.7/site-packages']
DEBUG:web-platform-test-launcher:files in cwd are:['_certs', 'common', 'config.default.json', 'config.json', 'domparsing', 'fonts', 'images', 'resources', 'serve.py', 'serve.pyc', 'tools', 'w3c-import.log']
DEBUG:web-platform-test-launcher:files in cwd/tools are:['html5lib', 'pywebsocket', 'runner', 'scripts', 'six', 'sslutils', 'w3c-import.log', 'webdriver', 'wptserve']
Traceback (most recent call last):
File "/Volumes/Data/EWS/WebKit/Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_launcher.py", line 18, in <module>
logger.debug(imp.find_module('tools'))
ImportError: No module named tools
(In reply to comment #19)
> !? Did you accidentally added all the files in the imported directory? The
> patch is 5.7MB.
No, I just use the import script that copies the whole tools folder.
It notably includes full html5lib and pywebsocket modules files.
If size is an issue, we could probably discard the tools/html5lib/html5lib/tests internal tests (2.5 MB) and the pywebsocket/src/tests internal tests (400Kb).
We could be more drastic by removing all modules except than wptserve, pywebsocket and sslutils, which would reduce the patch size to 1 MB probably.
Removing these modules may result in future unexpected test failures though.
(In reply to comment #20)
> (In reply to comment #19)
> > !? Did you accidentally added all the files in the imported directory? The
> > patch is 5.7MB.
>
> No, I just use the import script that copies the whole tools folder.
> It notably includes full html5lib and pywebsocket modules files.
>
> If size is an issue, we could probably discard the
> tools/html5lib/html5lib/tests internal tests (2.5 MB) and the
> pywebsocket/src/tests internal tests (400Kb).
>
> We could be more drastic by removing all modules except than wptserve,
> pywebsocket and sslutils, which would reduce the patch size to 1 MB probably.
> Removing these modules may result in future unexpected test failures though.
Or we auto-install the different tools as we do for thirdparty python modules before launching web-platform-test-server.
Some more scripting, but that would leave webkit repo slim.
(In reply to comment #21)
> (In reply to comment #20)
> > (In reply to comment #19)
> > > !? Did you accidentally added all the files in the imported directory? The
> > > patch is 5.7MB.
> >
> > No, I just use the import script that copies the whole tools folder.
> > It notably includes full html5lib and pywebsocket modules files.
> >
> > If size is an issue, we could probably discard the
> > tools/html5lib/html5lib/tests internal tests (2.5 MB) and the
> > pywebsocket/src/tests internal tests (400Kb).
> >
> > We could be more drastic by removing all modules except than wptserve,
> > pywebsocket and sslutils, which would reduce the patch size to 1 MB probably.
> > Removing these modules may result in future unexpected test failures though.
>
> Or we auto-install the different tools as we do for thirdparty python
> modules before launching web-platform-test-server.
> Some more scripting, but that would leave webkit repo slim.
Yeah, can we do that instead?
Size is reduced to 600 ko, hope this is small enough.
300 ko are due to tools/runner.
It is called by wpt server when some specific URLs are hit.
I doubt that this will ever happen in WebKit layout test context.
Note also that the _certs folder can be dynamically generated by wptserver.
It seems safer to have a repeatable test set-up hence why it is checked in.
Comment on attachment 245703[details]
Adding .gitignore for downloaded modules
View in context: https://bugs.webkit.org/attachment.cgi?id=245703&action=review
r=me for webkitpy change and rs=me for the import.
> Tools/Scripts/webkitpy/layout_tests/servers/web_platform_test_launcher.py:38
> +
Superflous change.
> LayoutTests/TestExpectations:234
> +imported/w3c/web-platform-tests/domparsing/DOMParser-parseFromString-html.html [ Pass ] # Some assertions failing
Shouldn't this have a failing expectation then?
(In reply to comment #28)
> Comment on attachment 245703[details]
> Adding .gitignore for downloaded modules
> > LayoutTests/TestExpectations:234
> > +imported/w3c/web-platform-tests/domparsing/DOMParser-parseFromString-html.html [ Pass ] # Some assertions failing
>
> Shouldn't this have a failing expectation then?
This line just reminds that the expected.txt contains some FAIL assertions lines. It does not need to be there.
It serves more as a reminder that someone should investigate the inconsistency between WebKit and the test as TestImport file will do for failed tests.
It may also be useful when updating the revision of the test suite.
Attachment 245736[details] did not pass style-queue:
ERROR: LayoutTests/TestExpectations:235: Path does not exist. [test/expectations] [5]
Total errors found: 1 in 93 files
If any of these errors are false positives, please file a bug against check-webkit-style.
Comment on attachment 245736[details]
Patch for landing
Rejecting attachment 245736[details] from commit-queue.
Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.appspot.com', '--bot-id=webkit-cq-02', 'land-attachment', '--force-clean', '--non-interactive', '--parent-command=commit-queue', 245736, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit
Last 500 characters of output:
19ba9e139caa3210
r179424 = 7ceb91db1a07084c86cbd7997b85782e66ab8554
r179425 = e37d84c5b7056e35db5e224e50861aee8811eead
r179426 = fff116d7a3b48251caa7bf0a2fa92e847c39865d
Done rebuilding .git/svn/refs/remotes/origin/master/.rev_map.268f45cc-cd09-0410-ab3c-d52691b4dbfc
First, rewinding head to replay your work on top of it...
Fast-forwarded master to refs/remotes/origin/master.
ERROR: LayoutTests/TestExpectations:235: Path does not exist. [test/expectations] [5]
Total errors found: 1 in 1 files
Full output: http://webkit-queues.appspot.com/results/5570474293592064
2015-01-27 05:48 PST, youenn fablet
2015-01-27 07:03 PST, Build Bot
2015-01-27 14:25 PST, youenn fablet
2015-01-27 15:38 PST, Build Bot
2015-01-28 04:03 PST, youenn fablet
2015-01-28 05:44 PST, Build Bot
2015-01-28 06:29 PST, youenn fablet
2015-01-28 07:49 PST, Build Bot
2015-01-28 10:48 PST, youenn fablet
2015-01-29 03:55 PST, youenn fablet
2015-01-29 15:21 PST, youenn fablet
2015-01-29 22:52 PST, youenn fablet
2015-01-30 00:35 PST, youenn fablet
2015-01-30 00:53 PST, youenn fablet
2015-01-30 13:16 PST, youenn fablet
2015-01-30 23:57 PST, youenn fablet