Bug 38031 – [chromium] WebKitTools\Scripts\webkitpy\layout_tests\port\http_server.py causes a syntax error for Vista, XP, Nacl

• Component: Tools / Tests
• Status: RESOLVED WONTFIX

[Yuzo Fujishima]

The following errors were observed when rolling WebKit from r58033 to r58138.


http://build.chromium.org/buildbot/waterfall/builders/Vista%20Tests/builds/18485/steps/ui_tests/logs/stdio
[----------] 6 tests from ExtensionProxyUITest
[ RUN      ] ExtensionProxyUITest.NoSuchExtension
[       OK ] ExtensionProxyUITest.NoSuchExtension (509 ms)
[ RUN      ] ExtensionProxyUITest.EnableDisable
[       OK ] ExtensionProxyUITest.EnableDisable (592 ms)
[ RUN      ] ExtensionProxyUITest.Uninstall
[       OK ] ExtensionProxyUITest.Uninstall (616 ms)
[ RUN      ] ExtensionProxyUITest.ExecuteBrowserActionInActiveTabAsync
Traceback (most recent call last):
  File "c:\b\slave\chromium-rel-vista-tests\build\src\third_party\WebKit\WebKitTools\Scripts\new-run-webkit-httpd", line 50, in ?
    from port import http_server
  File "c:\b\slave\chromium-rel-vista-tests\build\src\third_party\WebKit\WebKitTools\Scripts\webkitpy\layout_tests\port\http_server.py", line 118
    with codecs.open(base_conf_file, "r", "utf-8") as file:
              ^
SyntaxError: invalid syntax


http://build.chromium.org/buildbot/waterfall/builders/XP%20Tests%20(dbg)(4)/builds/6546/steps/ui_tests/logs/stdio
[----------] 2 tests from ExtensionProxyUITest
[ RUN      ] ExtensionProxyUITest.NoSuchExtension
[       OK ] ExtensionProxyUITest.NoSuchExtension (1344 ms)
[ RUN      ] ExtensionProxyUITest.ExecuteBrowserActionInActiveTabAsync
Traceback (most recent call last):
  File "C:\b\slave\chromium-dbg-xp-tests-4\build\src\third_party\WebKit\WebKitTools\Scripts\new-run-webkit-httpd", line 50, in ?
    from port import http_server
  File "C:\b\slave\chromium-dbg-xp-tests-4\build\src\third_party\WebKit\WebKitTools\Scripts\webkitpy\layout_tests\port\http_server.py", line 118
    with codecs.open(base_conf_file, "r", "utf-8") as file:
              ^
SyntaxError: invalid syntax


http://build.chromium.org/buildbot/waterfall/builders/NACL%20Tests/builds/9309/steps/nacl_ui_tests/logs/stdio
c:\b\slave\chromium-rel-nacl-tests\build\src\build\Release\nacl_ui_tests.exe --gtest_print_time
[==========] Running 8 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 8 tests from NaClTest
[ RUN      ] NaClTest.ServerTest
Traceback (most recent call last):
  File "c:\b\slave\chromium-rel-nacl-tests\build\src\third_party\WebKit\WebKitTools\Scripts\new-run-webkit-httpd", line 50, in ?
    from port import http_server
  File "c:\b\slave\chromium-rel-nacl-tests\build\src\third_party\WebKit\WebKitTools\Scripts\webkitpy\layout_tests\port\http_server.py", line 118
    with codecs.open(base_conf_file, "r", "utf-8") as file:
              ^
SyntaxError: invalid syntax


http://build.chromium.org/buildbot/waterfall/builders/NACL%20Tests%20(x64)/builds/926/steps/nacl_ui_tests/logs/stdio
C:\b\slave\chromium-rel-nacl-tests-64\build\src\build\Release\nacl_ui_tests.exe --gtest_print_time
[==========] Running 8 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 8 tests from NaClTest
[ RUN      ] NaClTest.ServerTest
Traceback (most recent call last):
  File "C:\b\slave\chromium-rel-nacl-tests-64\build\src\third_party\WebKit\WebKitTools\Scripts\new-run-webkit-httpd", line 50, in ?
    from port import http_server
  File "C:\b\slave\chromium-rel-nacl-tests-64\build\src\third_party\WebKit\WebKitTools\Scripts\webkitpy\layout_tests\port\http_server.py", line 118
    with codecs.open(base_conf_file, "r", "utf-8") as file:
              ^
SyntaxError: invalid syntax

[Tony Chang]

Tears.

The bots must be using python2.4, which doesn't support 'with'.  I'm not sure how hard it is to upgrade them.  Maybe we should just rewrite this in the meantime.

[Fumitoshi Ukai]

I believe http://trac.webkit.org/changeset/58036 introduced "with" statement.
From this change, webkitpy requires python 2.5, but chromium tree uses third_party/python_24.

Should we upgrade python in chromium?

[Eric Seidel (no email)]

My sincerest apologies for the breakage.

I think it would be a perfectly acceptable solution to roll out the necessary "with" statements to get this running.

The right long term solution is to upgrade the bots since WebKit's python is 2.5+ only and we will continue to break 2.4 clients (not intentionally, just as a result of further fixing/refactoring in webkitpy).

This week's python changes have been a disaster.  Hopefully we're near the end of the tunnel.

[Tony Chang]

*** Bug 38037 has been marked as a duplicate of this bug. ***

[Tony Chang]

Created attachment 54141 [details]
Patch

[Yuzo Fujishima]

non-reviewer comment:
Assignment to file variable should be done before try block.
Otherwise, the variable could be undefined in the finally block
when an exception is raised in evaluating the right hand side of the assignment.

[Eric Seidel (no email)]

Tony SSHed into the bots to see about upgrading them.  He found they already had both 2.4 and 2.6, but he couldn't figure out how to switch between the two for these tests.

[Adam Barth]

Comment on attachment 54141 [details]
Patch

WebKitTools/ChangeLog:5
 +          remove 'with' from files run on chromium buildbots
Please explain why we're making this change.

WebKitTools/Scripts/webkitpy/common/system/executive.py: 
 +          stderr = subprocess.STDOUT if return_stderr else None
Please add a FIXME to explain that we should put this back once we can actually use python 2.5.

This change is extremely fragile.  I have no idea which files need 2.4 support and which don't, which means we're going to be constantly breaking this property in the future.  We need to update the Chromium bots to a supported version of Python or we're going to continue to feel this pain going forward.

[Tony Chang]

Comment on attachment 54141 [details]
Patch

We were able to upgrade python on the bots, so this patch isn't needed.

[Eric Seidel (no email)]

Sweet!  Thank you Tony for taking care of this!