Bug 64594 - test-webkitpy failing with Python 2.5
Summary: test-webkitpy failing with Python 2.5
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Philippe Normand
URL:
Keywords: MakingBotsRed
Depends on:
Blocks:
 
Reported: 2011-07-15 05:45 PDT by WebKit Review Bot
Modified: 2011-07-19 02:04 PDT (History)
5 users (show)

See Also:


Attachments
proposed patch (172.41 KB, patch)
2011-07-15 06:16 PDT, Philippe Normand
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description WebKit Review Bot 2011-07-15 05:45:09 PDT
webkitpy uses urlparse.parse_qs not available in Python 2.5
Requested by philn-tp on #webkit.
Comment 1 Philippe Normand 2011-07-15 05:45:47 PDT
======================================================================
ERROR: test_changelog (webkitpy.tool.servers.gardeningserver_unittest.GardeningServerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/slave/webkitgtk/gtk-linux-32-debug/build/Tools/Scripts/webkitpy/tool/servers/gardeningserver_unittest.py", line 78, in test_changelog
    self._post_to_path("/changelog?revision=2314", expected_stderr=expected_stderr, expected_stdout=expected_stdout)
  File "/home/slave/webkitgtk/gtk-linux-32-debug/build/Tools/Scripts/webkitpy/tool/servers/gardeningserver_unittest.py", line 70, in _post_to_path
    OutputCapture().assert_outputs(self, handler.do_POST, expected_stderr=expected_stderr, expected_stdout=expected_stdout)
  File "/home/slave/webkitgtk/gtk-linux-32-debug/build/Tools/Scripts/webkitpy/common/system/outputcapture.py", line 63, in assert_outputs
    return_value = function(*args, **kwargs)
  File "/home/slave/webkitgtk/gtk-linux-32-debug/build/Tools/Scripts/webkitpy/tool/servers/reflectionhandler.py", line 61, in do_POST
    self._handle_request()
  File "/home/slave/webkitgtk/gtk-linux-32-debug/build/Tools/Scripts/webkitpy/tool/servers/reflectionhandler.py", line 66, in _handle_request
    self.query = urlparse.parse_qs(query_string)
AttributeError: 'module' object has no attribute 'parse_qs'
Comment 2 Philippe Normand 2011-07-15 05:46:30 PDT
In that case we should fallback to cgi.parse_qs
Comment 3 Philippe Normand 2011-07-15 06:11:39 PDT
thirdparty.simplejson also needs an update:


======================================================================
FAIL: test_changelog (webkitpy.tool.servers.gardeningserver_unittest.GardeningServerTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/wk/WebKit/Tools/Scripts/webkitpy/tool/servers/gardeningserver_unittest.py", line 78, in test_changelog
    self._post_to_path("/changelog?revision=2314", expected_stderr=expected_stderr, expected_stdout=expected_stdout)
  File "/wk/WebKit/Tools/Scripts/webkitpy/tool/servers/gardeningserver_unittest.py", line 70, in _post_to_path
    OutputCapture().assert_outputs(self, handler.do_POST, expected_stderr=expected_stderr, expected_stdout=expected_stdout)
  File "/wk/WebKit/Tools/Scripts/webkitpy/common/system/outputcapture.py", line 65, in assert_outputs
    testcase.assertEqual(stdout_string, expected_stdout)
AssertionError: '== Begin JSON Response ==\n{"bug_id": 42, "author_email": "abarth@webkit.org", "reviewer_text": "Darin Adler", "author_name": "Adam Barth", "changed_files": ["path\\/to\\/file", "another\\/file"]}\n== End JSON Response ==\n' != '== Begin JSON Response ==\n{"bug_id": 42, "author_email": "abarth@webkit.org", "reviewer_text": "Darin Adler", "author_name": "Adam Barth", "changed_files": ["path/to/file", "another/file"]}\n== End JSON Response ==\n'
Comment 4 Philippe Normand 2011-07-15 06:16:17 PDT
Created attachment 100964 [details]
proposed patch
Comment 5 WebKit Review Bot 2011-07-15 06:20:18 PDT
Attachment 100964 [details] did not pass style-queue:

Failed to run "['Tools/Scripts/check-webkit-style', '--diff-files', u'Tools/ChangeLog', u'Tools/Scripts/webkitpy..." exit_code: 1

Tools/Scripts/webkitpy/thirdparty/simplejson/__init__.py:413:  trailing whitespace  [pep8/W291] [5]
Total errors found: 1 in 10 files


If any of these errors are false positives, please file a bug against check-webkit-style.
Comment 6 Eric Seidel (no email) 2011-07-15 11:53:50 PDT
Comment on attachment 100964 [details]
proposed patch

View in context: https://bugs.webkit.org/attachment.cgi?id=100964&action=review

> Tools/ChangeLog:11
> +        Updated simplejson to version 2.1.6 that fixes an issue when
> +        dumping slashes. Also use cgi.parse_qs instead of the urlparse
> +        version to be compatible with Python2.5. Verified the fix with
> +        test-webkitpy on python 2.5 and python 2.6.

Ok.  So you're just updating simplejson, and then separately making changes to our code, correct?  You're not changing simplejson besides the update?
Comment 7 Philippe Normand 2011-07-15 15:11:15 PDT
(In reply to comment #6)
> (From update of attachment 100964 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=100964&action=review
> 
> > Tools/ChangeLog:11
> > +        Updated simplejson to version 2.1.6 that fixes an issue when
> > +        dumping slashes. Also use cgi.parse_qs instead of the urlparse
> > +        version to be compatible with Python2.5. Verified the fix with
> > +        test-webkitpy on python 2.5 and python 2.6.
> 
> Ok.  So you're just updating simplejson, and then separately making changes to our code, correct? 

Yes

> You're not changing simplejson besides the update?

I did change two import statements from:

from simplejson import something

to:

import something

I will land this patch later unless you have more comments. I locally checked test-webkitpy still passes with python 2.5 and 2.6.
Comment 8 Daniel Bates 2011-07-17 00:48:06 PDT
What's the status on landing this patch? Can we land this by commit-queue? Both the Leopard Debug and Release bots are failing the unit tests because of this issue.
Comment 9 Adam Barth 2011-07-17 01:00:03 PDT
Comment on attachment 100964 [details]
proposed patch

I mean, this should be two separate patches, but its ok.
Comment 10 WebKit Review Bot 2011-07-17 01:57:24 PDT
Comment on attachment 100964 [details]
proposed patch

Clearing flags on attachment: 100964

Committed r91167: <http://trac.webkit.org/changeset/91167>
Comment 11 WebKit Review Bot 2011-07-17 01:57:29 PDT
All reviewed patches have been landed.  Closing bug.
Comment 12 James Robinson 2011-07-18 14:51:11 PDT
When running NRWT on chromium-linux I'm getting the following at the end of the test run, before results.html is produced:


Traceback (most recent call last):
  File "WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 438, in <module>
    sys.exit(main())
  File "WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 433, in main
    return run(port, options, args)
  File "WebKit/Tools/Scripts/webkitpy/layout_tests/run_webkit_tests.py", line 107, in run
    num_unexpected_results = manager.run(result_summary)
  File "WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 933, in run
    self._upload_json_files(summarized_results, result_summary, individual_test_timings)
  File "WebKit/Tools/Scripts/webkitpy/layout_tests/controllers/manager.py", line 1080, in _upload_json_files
    json_results_generator.write_json(self._fs, times_trie, times_json_path)
  File "WebKit/Tools/Scripts/webkitpy/layout_tests/layout_package/json_results_generator.py", line 65, in write_json
    json_data = simplejson.dumps(json_object, separators=(',', ':'))
  File "WebKit/Tools/Scripts/webkitpy/thirdparty/simplejson/__init__.py", line 269, in dumps
    use_decimal=use_decimal, **kw).encode(obj)
  File "WebKit/Tools/Scripts/webkitpy/thirdparty/simplejson/encoder.py", line 216, in encode
    chunks = self.iterencode(o, _one_shot=True)
  File "WebKit/Tools/Scripts/webkitpy/thirdparty/simplejson/encoder.py", line 277, in iterencode
    self.skipkeys, self.allow_nan, key_memo, self.use_decimal)
TypeError: make_encoder() takes at most 9 arguments (11 given)


Is this related?  What's the fix?
Comment 13 Eric Seidel (no email) 2011-07-18 15:08:05 PDT
I suspect the recent simplejson upgrade?
Comment 14 Philippe Normand 2011-07-19 01:29:37 PDT
(In reply to comment #13)
> I suspect the recent simplejson upgrade?

Yes. Bug 64757 seems to fix the issue. Is the chromium bot using python 2.6?
I wonder if with the patch of that bug and a bot running python 2.5 you'd still get the traceback mentioned above...
Comment 15 Philippe Normand 2011-07-19 02:04:35 PDT
(In reply to comment #14)
> (In reply to comment #13)
> > I suspect the recent simplejson upgrade?
> 
> Yes. Bug 64757 seems to fix the issue. Is the chromium bot using python 2.6?
> I wonder if with the patch of that bug and a bot running python 2.5 you'd still get the traceback mentioned above...

I've just checked NRWT with python2.5 on GTK and we're indeed safe. Thanks Eric and Adam!