Bug 189412 – download-github-release.py hits errors on Windows bots

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Alexey Proskuryakov]

https://build.webkit.org/builders/Apple%20Win%20Release%20%28Build%29/builds/11718/steps/compile-webkit/logs/stdio

Many errors here:

Downloading to /home/buildbot/slave/win-release/build/WebKitLibraries/win/tools...
Traceback (most recent call last):
  File "Tools/Scripts/update-vswhere.py", line 37, in <module>
    result = download.main(['-o', output, repo, file])
  File "/home/buildbot/slave/win-release/build/Tools/Scripts/download-github-release.py", line 146, in main
    download_release(release_url, binary_path, args.token)
  File "/home/buildbot/slave/win-release/build/Tools/Scripts/download-github-release.py", line 94, in download_release
    file.write(urllib2.urlopen(request).read())
  File "/usr/lib/python2.7/urllib2.py", line 154, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 437, in open
    response = meth(req, response)
  File "/usr/lib/python2.7/urllib2.py", line 550, in http_response
    'http', request, response, code, msg, hdrs)
  File "/usr/lib/python2.7/urllib2.py", line 469, in error
    result = self._call_chain(*args)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 656, in http_error_302
    return self.parent.open(new, timeout=req.timeout)
  File "/usr/lib/python2.7/urllib2.py", line 431, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 449, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 409, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1240, in https_open
    context=self._context)
  File "/usr/lib/python2.7/urllib2.py", line 1197, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error [Errno 116] Connection timed out>
Error getting Visual Studio Location: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 679.

Can't exec "/home/buildbot/slave/win-release/build/WebKitLibraries/win/tools/vswhere": Permission denied at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 676.
Can't exec "/home/buildbot/slave/win-release/build/WebKitLibraries/win/tools/vswhere": Permission denied at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 676.
Error getting Visual Studio Location: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 679.

Error getting Visual Studio Location: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 679.

Can't exec "/home/buildbot/slave/win-release/build/WebKitLibraries/win/tools/vswhere": Permission denied at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 676.
Error getting Visual Studio Location: malformed JSON string, neither array, object, number, string or atom, at character offset 0 (before "(end of string)") at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 679.

Can't exec "/home/buildbot/slave/win-release/build/WebKitLibraries/win/tools/vswhere": Permission denied at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 676.
Use of uninitialized value $vsInstallDir in -e at /home/buildbot/slave/win-release/build/Tools/Scripts/webkitdirs.pm line 702.

[Ross Kirsling]

Is there a network issue on the AppleWin bots?
Nothing about how update-vswhere is called has changed since it was introduced in bug 175275, but it's tricky to know when this started occurring, as all of the existing logs contain this error.

[Alexey Proskuryakov]

Could be intentionally limited network ACL. May have well been happening from the start, I just didn’t look before. 

Is this script doing something that Apple Windows port needs?

[Stephan Szabo]

With the move to using VS 2017, it no longer really has a singular install location to use, nor can you rely on getting the version directly from the install location. vswhere finds the install location and version, the update script is just checking for the newest version of vswhere to upgrade to.

That might not be important for applewin depending on whether apple's IT policy prevents anything other than the default community location.

[Don Olmstead]

For wincairo builds we’re using the build tools version https://visualstudio.microsoft.com/downloads/ which is meant for build servers. There’s no IDE included. It installs in a different location than community.

[Fujii Hironori]

Created attachment 353273 [details]
Patch

[Don Olmstead]

Comment on attachment 353273 [details]
Patch

Informal r-

The vswhere call is there for a reason. Its to verify that there is a visual studio version that can build webkit. If the thing is failing it should have more error checking and a potential fallback path.

[Fujii Hironori]

Using vswhere causes the compiler version mismatch problem (Bug 190797 Comment 24).
CMake looks for VS by its own logic.

We should use vcvars64.bat to safely choose a VS installation.
If you have multiple VS installations, there are mulptile
"x64 Native Tools Command Prompt for Visual Studio 2017" menu items in Windows Start menu.
You can use which you want to choose VS.

[Fujii Hironori]

It seems that AppleWin BuildBot issue has been fixed.
I will file another bug for the mismatch issue.