Bug 175709 - bisect-builds doesn't work due to missing requests module
Summary: bisect-builds doesn't work due to missing requests module
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Lucas Forschler
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2017-08-17 21:52 PDT by Ryosuke Niwa
Modified: 2017-09-11 16:09 PDT (History)
6 users (show)

See Also:


Attachments
patch to move requests module to auto installed (2.51 KB, patch)
2017-09-08 12:37 PDT, Lucas Forschler
lforschler: commit-queue-
Details | Formatted Diff | Diff
v2 patch (2.90 KB, patch)
2017-09-08 14:14 PDT, Lucas Forschler
rniwa: review+
lforschler: commit-queue-
Details | Formatted Diff | Diff
fix patch to include urllib3 (3.37 KB, patch)
2017-09-11 15:01 PDT, Lucas Forschler
no flags Details | Formatted Diff | Diff
use urllib2 instead of the requests module. (2.67 KB, patch)
2017-09-11 15:44 PDT, Lucas Forschler
no flags Details | Formatted Diff | Diff
use urllib2 instead of the requests module. (1.99 KB, patch)
2017-09-11 15:51 PDT, Lucas Forschler
rniwa: review+
lforschler: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryosuke Niwa 2017-08-17 21:52:41 PDT
$ ./Tools/Scripts/bisect-builds --help
Traceback (most recent call last):
  File "./Tools/Scripts/bisect-builds", line 33, in <module>
    import requests
ImportError: No module named requests
rniwa-tbmbp:webkit2 rniwa$ ./Tools/Scripts/bisect-builds
Traceback (most recent call last):
  File "./Tools/Scripts/bisect-builds", line 33, in <module>
    import requests
ImportError: No module named requests
Comment 1 Ryosuke Niwa 2017-08-17 21:53:25 PDT
We need to auto-install requests package in webkitpy/thirdparty/__init__.py, and import it from there.
Comment 2 Radar WebKit Bug Importer 2017-08-17 21:53:49 PDT
<rdar://problem/33956917>
Comment 3 Lucas Forschler 2017-09-08 12:37:28 PDT
Created attachment 320289 [details]
patch to move requests module to auto installed

After doing this, the script startup time takes 15 seconds... this is not ideal.
Comment 4 Lucas Forschler 2017-09-08 14:14:55 PDT
Created attachment 320301 [details]
v2 patch
Comment 5 Lucas Forschler 2017-09-08 14:17:15 PDT
using the get_latest_pypi_ur function in the third party stuff creates about 40+ calls out to pypi, which is very slow. I've updated the patch to manually specify the url, which is much faster, nearly instant.
Comment 6 Ryosuke Niwa 2017-09-08 15:22:56 PDT
Comment on attachment 320301 [details]
v2 patch

View in context: https://bugs.webkit.org/attachment.cgi?id=320301&action=review

> Tools/Scripts/webkitpy/thirdparty/__init__.py:114
> +                             "requests-2.18.4/requests")

Wrong indentation. Should be indented by 4 spaces to the right of self._install.
Comment 7 Lucas Forschler 2017-09-08 16:01:10 PDT
Committed revision 221801.
Comment 8 Ryan Haddad 2017-09-09 09:48:53 PDT
(In reply to Lucas Forschler from comment #7)
> Committed revision 221801.

This change introduced a webkitpy test failure:

[1295/1654] webkitpy.thirdparty.__init___unittest.ThirdpartyTest.test_imports erred:
  Traceback (most recent call last):
    File "/Volumes/Data/slave/elcapitan-release-tests-wk2/build/Tools/Scripts/webkitpy/thirdparty/__init___unittest.py", line 66, in test_imports
      import webkitpy.thirdparty.autoinstalled.requests
    File "/Volumes/Data/slave/elcapitan-release-tests-wk2/build/Tools/Scripts/webkitpy/thirdparty/autoinstalled/requests/__init__.py", line 43, in <module>
      import urllib3
  ImportError: No module named urllib3

https://build.webkit.org/builders/Apple%20El%20Capitan%20Release%20WK2%20%28Tests%29/builds/4330
Comment 9 Ryan Haddad 2017-09-09 09:58:51 PDT
Reverted r221801 for reason:

This change introduced a webkitpy test failure.

Committed r221825: <http://trac.webkit.org/changeset/221825>
Comment 10 Lucas Forschler 2017-09-11 15:01:19 PDT
Created attachment 320482 [details]
fix patch to include urllib3
Comment 11 Lucas Forschler 2017-09-11 15:38:37 PDT
The requests module imports urllib3. I don't see an easy way to tell it to use the autoinstalled location. So, this still fails.  I'm going to change the bisect-builds tool to use urllib2, which is installed by default. That will make this go away.
Comment 12 Lucas Forschler 2017-09-11 15:44:41 PDT
Created attachment 320490 [details]
use urllib2 instead of the requests module.
Comment 13 Lucas Forschler 2017-09-11 15:51:04 PDT
Created attachment 320492 [details]
use urllib2 instead of the requests module.
Comment 14 Lucas Forschler 2017-09-11 16:09:21 PDT
Committed revision 221890.