Bug 189412 - download-github-release.py hits errors on Windows bots
Summary: download-github-release.py hits errors on Windows bots
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Fujii Hironori
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2018-09-07 10:12 PDT by Alexey Proskuryakov
Modified: 2018-10-30 18:06 PDT (History)
9 users (show)

See Also:


Attachments
Patch (7.64 KB, patch)
2018-10-28 21:22 PDT, Fujii Hironori
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexey Proskuryakov 2018-09-07 10:12:47 PDT
https://build.webkit.org/builders/Apple%20Win%20Release%20%28Build%29/builds/11718/steps/compile-webkit/logs/stdio

Many errors here:

Downloading to /home/buildbot/slave/win-release/build/WebKitLibraries/win/tools...
Traceback (most recent call last):
  File "Tools/Scripts/update-vswhere.py", line 37, in <module>
    result = download.main(['-o', output, repo, file])
  File "/home/buildbot/slave/win-release/build/Tools/Scripts/download-github-release.py", line 146, in main
    download_release(release_url, binary_path, args.token)
  File "/home/buildbot/slave/win-release/build/Tools/Scripts/download-github-release.py", line 94, in download_release
    file.write(urllib2.urlopen(request).read())
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 469, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 656, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1240, in https_open
    context=self._context)
  File "/usr/lib/python2.7/urllib2.py", line 1197, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 116] Connection timed out>
Error getting Visual Studio Location: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 679.

Can't exec "/home/buildbot/slave/win-release/build/WebKitLibraries/win/tools/vswhere": Permission denied at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 676.
Can't exec "/home/buildbot/slave/win-release/build/WebKitLibraries/win/tools/vswhere": Permission denied at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 676.
Error getting Visual Studio Location: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 679.

Error getting Visual Studio Location: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 679.

Can't exec "/home/buildbot/slave/win-release/build/WebKitLibraries/win/tools/vswhere": Permission denied at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 676.
Error getting Visual Studio Location: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 679.

Can't exec "/home/buildbot/slave/win-release/build/WebKitLibraries/win/tools/vswhere": Permission denied at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 676.
Use of uninitialized value $vsInstallDir in -e at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 702.
Comment 1 Ross Kirsling 2018-09-07 11:03:47 PDT
Is there a network issue on the AppleWin bots?
Nothing about how update-vswhere is called has changed since it was introduced in bug 175275, but it's tricky to know when this started occurring, as all of the existing logs contain this error.
Comment 2 Alexey Proskuryakov 2018-09-07 13:24:16 PDT
Could be intentionally limited network ACL. May have well been happening from the start, I just didn’t look before. 

Is this script doing something that Apple Windows port needs?
Comment 3 Stephan Szabo 2018-09-07 18:20:51 PDT
With the move to using VS 2017, it no longer really has a singular install location to use, nor can you rely on getting the version directly from the install location. vswhere finds the install location and version, the update script is just checking for the newest version of vswhere to upgrade to.

That might not be important for applewin depending on whether apple's IT policy prevents anything other than the default community location.
Comment 4 Don Olmstead 2018-09-07 21:51:21 PDT
For wincairo builds we’re using the build tools version https://visualstudio.microsoft.com/downloads/ which is meant for build servers. There’s no IDE included. It installs in a different location than community.
Comment 5 Fujii Hironori 2018-10-28 21:22:03 PDT
Created attachment 353273 [details]
Patch
Comment 6 Don Olmstead 2018-10-29 09:17:57 PDT
Comment on attachment 353273 [details]
Patch

Informal r-

The vswhere call is there for a reason. Its to verify that there is a visual studio version that can build webkit. If the thing is failing it should have more error checking and a potential fallback path.
Comment 7 Fujii Hironori 2018-10-29 18:12:08 PDT
Using vswhere causes the compiler version mismatch problem (Bug 190797 Comment 24).
CMake looks for VS by its own logic.

We should use vcvars64.bat to safely choose a VS installation.
If you have multiple VS installations, there are mulptile
"x64 Native Tools Command Prompt for Visual Studio 2017" menu items in Windows Start menu.
You can use which you want to choose VS.
Comment 8 Fujii Hironori 2018-10-30 18:06:55 PDT
It seems that AppleWin BuildBot issue has been fixed.
I will file another bug for the mismatch issue.