Chromium builders fail "gclient" step every so often This is the 3rd time I've seen this. It's not very common, but it's common enough that gclient fails that we should investigate making it retry. Yaar mentioned over email that chromium.org had some retry magic in their buildbot that we should consider using. http://build.webkit.org/builders/Chromium%20Mac%20Release/builds/1732/steps/gclient/logs/stdio This failure is actually probably a bad example as it may be a legit src.chromium.org downtime: ________ running 'svn update /Users/cltbld/Desktop/BuildSlaveData/WebKit-BuildSlave/chromium-mac-release/build/WebKit/chromium/net --revision 31834' in '/Users/cltbld/Desktop/BuildSlaveData/WebKit-BuildSlave/chromium-mac-release/build/WebKit/chromium' svn: REPORT request failed on '/svn/!svn/vcc/default' svn: REPORT of '/svn/!svn/vcc/default': 502 Bad Gateway (http://src.chromium.org) Error: failed to run command: svn update /Users/cltbld/Desktop/BuildSlaveData/WebKit-BuildSlave/chromium-mac-release/build/WebKit/chromium/net --revision 31834 Died at ./WebKitTools/Scripts/update-webkit-chromium line 55.
Another gclient failure: http://build.webkit.org/builders/Chromium%20Win%20Release/builds/1062/steps/gclient/logs/stdio ________ running 'svn update C:\WebKitBuildSlave\chromium-win-release\build\WebKit\chromium\third_party\icu --revision 31724' in 'C:\WebKitBuildSlave\chromium-win-release\build\WebKit\chromium' svn: Server sent unexpected return value (502 Bad Gateway) in response to PROPFIND request for '/svn/trunk/deps/third_party/icu42' Error: failed to run command: svn update C:\WebKitBuildSlave\chromium-win-release\build\WebKit\chromium\third_party\icu --revision 31724 No such file or directory at ./WebKitTools/Scripts/update-webkit-chromium line 55. program finished with exit code 2
Opened a bug on gclient: http://code.google.com/p/gclient/issues/detail?id=16
Failed again: http://build.webkit.org/builders/Chromium%20Win%20Release/builds/1448/steps/gclient/logs/stdio ________ running 'svn update C:\WebKitBuildSlave\chromium-win-release\build\WebKit\chromium\third_party\npapi --revision 31834' in 'C:\WebKitBuildSlave\chromium-win-release\build\WebKit\chromium' svn: Server sent unexpected return value (502 Bad Gateway) in response to OPTIONS request for 'http://src.chromium.org/svn/trunk/src/third_party/npapi' Error: failed to run command: svn update C:\WebKitBuildSlave\chromium-win-release\build\WebKit\chromium\third_party\npapi --revision 31834 No such file or directory at ./WebKitTools/Scripts/update-webkit-chromium line 55.
Failed again: http://build.webkit.org/builders/Chromium%20Linux%20Release/builds/1557/steps/gclient/logs/stdio
Again: http://build.webkit.org/builders/Chromium%20Linux%20Release/builds/1692/steps/gclient/logs/stdio ________ running 'svn update /WebKit-BuildSlave/chromium-linux-release/build/WebKit/chromium/third_party/libxslt --revision 31834' in '/WebKit-BuildSlave/chromium-linux-release/build/WebKit/chromium' svn: Server sent unexpected return value (502 Bad Gateway) in response to PROPFIND request for '/svn/trunk/src/third_party/libxslt' Error: failed to run command: svn update /WebKit-BuildSlave/chromium-linux-release/build/WebKit/chromium/third_party/libxslt --revision 31834 Bad file descriptor at ./WebKitTools/Scripts/update-webkit-chromium line 55.
This should be helped by: http://code.google.com/p/gclient/issues/detail?id=16
Syncing projects: 46% (12/26) ________ running 'svn update /WebKit-BuildSlave/chromium-linux-release/build/WebKit/chromium/third_party/icu --revision 31724' in '/WebKit-BuildSlave/chromium-linux-release/build/WebKit/chromium' svn: Working copy '/WebKit-BuildSlave/chromium-linux-release/build/WebKit/chromium/third_party/icu' locked svn: run 'svn cleanup' to remove locks (type 'svn help cleanup' for details) Error: failed to run command: svn update /WebKit-BuildSlave/chromium-linux-release/build/WebKit/chromium/third_party/icu --revision 31724 Bad file descriptor at ./WebKitTools/Scripts/update-webkit-chromium line 55. The update step for gclient doesnt' know how to rm -rf after failure like the normal svn update step for buildbot does.
Failed again: http://build.webkit.org/builders/Chromium%20Win%20Release/builds/3956/steps/gclient/logs/stdio
We may need to consider removing chromium from core builders. This is the third bot outage for CR bots this week. :(
Again: http://build.webkit.org/builders/Chromium%20Win%20Release/builds/4026/steps/gclient/logs/stdio
This still happens! http://build.webkit.org/builders/Chromium%20Linux%20Release/builds/7681/steps/gclient/logs/stdio We need to either hack around this, fix the problem, or kick Chromium out of the Core builders. This is unacceptable. Qt hacked around their builder-lameness with https://bugs.webkit.org/show_bug.cgi?id=39282 (which I'm not encouraging). But even that would be better than the current situation of commit-queue stalls and ever-so-often false blames.
Ok, that one was an actual real DEPS problem. But there was at least one example earlier today as well: http://build.webkit.org/builders/Chromium%20Win%20Release/builds/7264/steps/gclient/logs/stdio That one was a bad-gateway error talking to chromium svn.
And again: http://build.webkit.org/builders/Chromium%20Linux%20Release/builds/7688/steps/gclient/logs/stdio Just now some sort of XML error.
LayoutTest failures for Chromium are being marked WontFix. The Bug is still accessible and referenced from TestExpectations.