Bug 137338

Summary: Remove workarounds for versions of Python before 2.7
Product: WebKit Reporter: Éva Balázsfalvi <evab.u-szeged>
Component: Tools / TestsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: clopez, commit-queue, glenn, ossy
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Bug Depends on: 137392    
Bug Blocks:    
Attachments:
Description Flags
Patch
none
Patch none

Description Éva Balázsfalvi 2014-10-02 02:55:30 PDT
Remove workarounds for versions of Python before 2.7
Comment 1 Éva Balázsfalvi 2014-10-02 03:08:17 PDT
Created attachment 239098 [details]
Patch
Comment 2 WebKit Commit Bot 2014-10-03 03:14:40 PDT
Comment on attachment 239098 [details]
Patch

Clearing flags on attachment: 239098

Committed r174264: <http://trac.webkit.org/changeset/174264>
Comment 3 WebKit Commit Bot 2014-10-03 03:14:43 PDT
All reviewed patches have been landed.  Closing bug.
Comment 4 WebKit Commit Bot 2014-10-03 05:11:39 PDT
Re-opened since this is blocked by bug 137392
Comment 6 Carlos Alberto Lopez Perez 2014-10-03 05:27:47 PDT
run-perf-tests seems to be also broken. Locally I get:

$ Tools/Scripts/run-perf-tests  --platform gtk --release -2 Interactive/window-resize.html
Running 1 tests
Running Interactive/window-resize.html (1 of 1)
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Last character read from DRT stdout line was not a newline!  This indicates either a NRWT or DRT bug.
Traceback (most recent call last):
  File "Tools/Scripts/run-perf-tests", line 40, in <module>
    sys.exit(PerfTestsRunner().run())
  File "/home/clopez/webkit/webkit/Tools/Scripts/webkitpy/performance_tests/perftestsrunner.py", line 205, in run
    unexpected = self._run_tests_set(sorted(list(tests), key=lambda test: test.test_name()))
  File "/home/clopez/webkit/webkit/Tools/Scripts/webkitpy/performance_tests/perftestsrunner.py", line 372, in _run_tests_set
    metrics = test.run(self._options.time_out_ms)
  File "/home/clopez/webkit/webkit/Tools/Scripts/webkitpy/performance_tests/perftest.py", line 132, in run
    if not self._run_with_driver(driver, time_out_ms):
  File "/home/clopez/webkit/webkit/Tools/Scripts/webkitpy/performance_tests/perftest.py", line 179, in _run_with_driver
    self._filter_output(output)
  File "/home/clopez/webkit/webkit/Tools/Scripts/webkitpy/performance_tests/perftest.py", line 273, in _filter_output
    output.text = '\n'.join([line for line in re.split('\n', output.text) if not self._should_ignore_line(self._lines_to_ignore_in_parser_result, line)])
TypeError: sequence item 0: expected string, bytearray found
Comment 7 Csaba Osztrogonác 2014-10-03 05:44:13 PDT
Comment on attachment 239098 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=239098&action=review

> Tools/Scripts/webkitpy/port/driver.py:534
> -        self.content = str()  # FIXME: Should be bytearray() once we require Python 2.6.
> +        self.content = bytearray()

This one ...

> Tools/Scripts/webkitpy/port/server_process.py:99
> -        self._output = str()  # bytesarray() once we require Python 2.6
> -        self._error = str()  # bytesarray() once we require Python 2.6
> +        self._output = bytearray()
> +        self._error = bytearray()

... or this one broke the tests. Éva, could you check what happened?
Comment 8 Éva Balázsfalvi 2014-10-03 06:21:38 PDT
Created attachment 239203 [details]
Patch

Took out the str() to bytearray() modification which seemingly made the layout tests fail. I'll investigate what caused the problem later.
Comment 9 Carlos Alberto Lopez Perez 2014-10-03 06:35:01 PDT
(In reply to comment #8)
> Created an attachment (id=239203) [details]
> Patch
> 
> Took out the str() to bytearray() modification which seemingly made the layout tests fail. I'll investigate what caused the problem later.

I tested this patch, and both run-webkit-test and run-perf-test seems to work as expected on the GTK port.
Comment 10 Csaba Osztrogonác 2014-10-03 07:26:51 PDT
Comment on attachment 239203 [details]
Patch

Let's do it without bytearray(), r=me.
Comment 11 WebKit Commit Bot 2014-10-03 08:03:32 PDT
Comment on attachment 239203 [details]
Patch

Clearing flags on attachment: 239203

Committed r174266: <http://trac.webkit.org/changeset/174266>
Comment 12 WebKit Commit Bot 2014-10-03 08:03:35 PDT
All reviewed patches have been landed.  Closing bug.
Comment 13 Csaba Osztrogonác 2014-10-08 02:28:58 PDT
Comment on attachment 239203 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=239203&action=review

> Tools/Scripts/webkitpy/common/system/autoinstall.py:-248
> -    # This is a replacement for ZipFile.extractall(), which is
> -    # available in Python 2.6 but not in earlier versions.
> -    # NOTE: The version in 2.6.1 (which shipped on Snow Leopard) is broken!
> -    def _extract_all(self, zip_file, target_dir):
> -        for name in zip_file.namelist():
> -            path = os.path.join(target_dir, name)
> -            if not os.path.basename(path):
> -                # Then the path ends in a slash, so it is a directory.
> -                os.makedirs(path)
> -                continue
> -
> -            try:
> -                # We open this file w/o encoding, as we're reading/writing
> -                # the raw byte-stream from the zip file.
> -                outfile = open(path, 'wb')
> -            except IOError:
> -                # Not all zip files seem to list the directories explicitly,
> -                # so try again after creating the containing directory.
> -                _log.debug("Got IOError: retrying after creating directory...")
> -                dirname = os.path.dirname(path)
> -                os.makedirs(dirname)
> -                outfile = open(path, 'wb')
> -
> -            try:
> -                outfile.write(zip_file.read(name))
> -            finally:
> -                outfile.close()
> -

Unfortunately it broke autoinstalling python-irclib-0.4.8.zip with the stock python on 
- Ubuntu 12.04: python 2.7.3
- Ubuntu 14.04: python 2.7.6
- OS X 10.9: python 2.7.5

I think we should add this workaround back with proper comment.

> Tools/Scripts/webkitpy/common/system/autoinstall.py:235
> -            self._extract_all(zip_file, scratch_dir)
> +            zip_file.extractall(zip_file, scratch_dir)

extractall raises the following exception:

Auto-installing package: irclib.py
Traceback (most recent call last):
  File "/Users/oszi/WebKit/Tools/Scripts/webkitpy/test/main.py", line 234, in <module>
    sys.exit(main())
  File "/Users/oszi/WebKit/Tools/Scripts/webkitpy/test/main.py", line 72, in main
    return not tester.run()
  File "/Users/oszi/WebKit/Tools/Scripts/webkitpy/test/main.py", line 122, in run
    return self._run_tests(names)
  File "/Users/oszi/WebKit/Tools/Scripts/webkitpy/test/main.py", line 132, in _run_tests
    autoinstall_everything()
  File "/Users/oszi/WebKit/Tools/Scripts/webkitpy/thirdparty/__init__.py", line 166, in autoinstall_everything
    getattr(_hook, method)()
  File "/Users/oszi/WebKit/Tools/Scripts/webkitpy/thirdparty/__init__.py", line 150, in _install_irc
    url_subpath="irclib.py")
  File "/Users/oszi/WebKit/Tools/Scripts/webkitpy/common/system/autoinstall.py", line 473, in install
    raise Exception(message)
Exception: Error auto-installing the irclib.py package to:
 "/Users/oszi/WebKit/Tools/Scripts/webkitpy/thirdparty/autoinstalled/irc/irclib.py"
 --> Inner message: "There is no item named '/' in the archive"
Comment 14 Csaba Osztrogonác 2014-10-08 08:49:59 PDT
added back in http://trac.webkit.org/changeset/174448