Bug 141946 – setup-git-clone fails due to the failure to download python-irclib-0.4.8

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Ryosuke Niwa]

We can't use this command due to an autoinstall failure.

[Ryosuke Niwa]

Created attachment 247187 [details]
Fixes the bug

[Csaba Osztrogonác]

Comment on attachment 247187 [details]
Fixes the bug

I don't agree with modifying urls on each short network error.

You can also use SOURCEFORGE_MIRRORS and LOCAL_AUTOINSTALL_CACHE
environment variables to avoid broken webkit-patch on network error.
(See autoinstall.py for details.)

Of course it is possible to improce this mirror and local cache mechanism.

[Darin Adler]

(In reply to comment #2)
> I don't agree with modifying urls on each short network error.

OK, but long term is there some reason to prefer sourceforge.net over launchpad.net?

[Csaba Osztrogonác]

(In reply to comment #3)
> (In reply to comment #2)
> > I don't agree with modifying urls on each short network error.
> 
> OK, but long term is there some reason to prefer sourceforge.net over
> launchpad.net?

I don't prefer sourceforge or launchpad. But if we 
change, we should change the mirroring mechanism too.

But is there any reason why can't we check in irclib.py, 
ircbot.py and eliza.py instead of autoinstalling? 
In this case we would depend only on pypi.

[Ryosuke Niwa]

(In reply to comment #4)
>
> But is there any reason why can't we check in irclib.py, 
> ircbot.py and eliza.py instead of autoinstalling? 
> In this case we would depend only on pypi.

I suppose we can since the licensing is MIT. It would take a while to do it because I have to go through the corporate approval process if I were to do it though :(

[Csaba Osztrogonác]

irclib.py and ircbot.py have LGPL license,
but I can't find any license for the eliza.py.

[Ryosuke Niwa]

(In reply to comment #6)
> irclib.py and ircbot.py have LGPL license,
> but I can't find any license for the eliza.py.

Oh, no :(

[Csaba Osztrogonác]

Created attachment 247329 [details]
Patch

Let's check in ircbot.py and irclib.py and don't touch eliza

[WebKit Commit Bot]

Attachment 247329 [details] did not pass style-queue:

Traceback (most recent call last):
  File "/Volumes/Data/StyleQueue/Webkit/Tools/Scripts/webkit-patch", line 44, in <module>
    from webkitpy.tool.main import WebKitPatch
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/tool/main.py", line 38, in <module>
    from webkitpy.common.net.irc import ircproxy
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/net/irc/ircproxy.py", line 32, in <module>
    from webkitpy.common.net.irc.ircbot import IRCBot
  File "/Volumes/Data/StyleQueue/WebKit/Tools/Scripts/webkitpy/common/net/irc/ircbot.py", line 32, in <module>
    from webkitpy.thirdparty.irc import ircbot
ImportError: No module named irc


If any of these errors are false positives, please file a bug against check-webkit-style.

[Csaba Osztrogonác]

Comment on attachment 247329 [details]
Patch

remove flags to unbreak EWS bots

[Csaba Osztrogonác]

Created attachment 247330 [details]
Patch

Let's see if EWS bots like non empty __init__.py

[Csaba Osztrogonác]

Comment on attachment 247330 [details]
Patch

EWS is crazy ... it can't handle adding new module.

[Csaba Osztrogonác]

Any objection against check in ircbot.py and irclib.py ?

[WebKit Commit Bot]

Comment on attachment 247330 [details]
Patch

Clearing flags on attachment: 247330

Committed r181145: <http://trac.webkit.org/changeset/181145>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[David Kilzer (:ddkilzer)]

(In reply to comment #13)
> Any objection against check in ircbot.py and irclib.py ?

This is causing failures on test bots, not just EWS:

<https://build.webkit.org/builders/Apple%20Mavericks%20Release%20WK2%20(Tests)?numbuilds=50>

Is that because we're trying to replace a locally downloaded file with one from the svn repository?

[Csaba Osztrogonác]

(In reply to comment #16)
> (In reply to comment #13)
> > Any objection against check in ircbot.py and irclib.py ?
> 
> This is causing failures on test bots, not just EWS:
> 
> <https://build.webkit.org/builders/
> Apple%20Mavericks%20Release%20WK2%20(Tests)?numbuilds=50>
> 
> Is that because we're trying to replace a locally downloaded file with one
> from the svn repository?

Ihanks for noticing, I'll investigate it right now.

[David Kilzer (:ddkilzer)]

(In reply to comment #17)
> (In reply to comment #16)
> > (In reply to comment #13)
> > > Any objection against check in ircbot.py and irclib.py ?
> > 
> > This is causing failures on test bots, not just EWS:
> > 
> > <https://build.webkit.org/builders/
> > Apple%20Mavericks%20Release%20WK2%20(Tests)?numbuilds=50>
> > 
> > Is that because we're trying to replace a locally downloaded file with one
> > from the svn repository?
> 
> Ihanks for noticing, I'll investigate it right now.

Or maybe we just need to blow away all *.pyc files before running the tests on the bots?

[Csaba Osztrogonác]

I have no idea what's happening ... I can't reproduce this issue on Mavericks.
I tried to run tests on 181144 and then on 181145 and again on 181144, ...
and didn't get any failure.

But something is very strange with the Mac bots:
https://build.webkit.org/buildslaves/bot199 - good revision order, no issue
https://build.webkit.org/buildslaves/bot151 - 181145, 181143, 181149 - BANG!

[Csaba Osztrogonác]

Something is very buggy with these bots ...

https://build.webkit.org/builders/Apple%20Mavericks%20Release%20WK1%20%28Tests%29/builds/12738/steps/svn/logs/stdio

   C Tools/Scripts/webkitpy/thirdparty/irc ----> what???????
   A Tools/Scripts/webkitpy/thirdparty/irc/__init__.py
   A Tools/Scripts/webkitpy/thirdparty/irc/ircbot.py
   A Tools/Scripts/webkitpy/thirdparty/irc/irclib.py

Let me check it with pure svn checkout, I only checked in git.

[Csaba Osztrogonác]

It would be great if somebody has access to these bots 
could check what is happening?

[Csaba Osztrogonác]

I kicked the buggy mavericks bots, let's see the results.
( force build + stop during svn update to remove everything + force build )

[David Kilzer (:ddkilzer)]

(In reply to comment #21)
> It would be great if somebody has access to these bots 
> could check what is happening?

This change and Bug 142083 has left bots in this state:

$ svn stat .
A  +  C LayoutTests/imported/w3c/web-platform-tests/_certs
      >   local edit, incoming delete upon update
C  +    LayoutTests/imported/w3c/web-platform-tests/_certs/01.pem
C  +    LayoutTests/imported/w3c/web-platform-tests/_certs/02.pem
C  +    LayoutTests/imported/w3c/web-platform-tests/_certs/cacert.pem
C  +    LayoutTests/imported/w3c/web-platform-tests/_certs/cakey.pem
C  +    LayoutTests/imported/w3c/web-platform-tests/_certs/index.txt.old
C  +    LayoutTests/imported/w3c/web-platform-tests/_certs/localhost.key
C  +    LayoutTests/imported/w3c/web-platform-tests/_certs/localhost.pem
D     C LayoutTests/imported/w3c/web-platform-tests/tools/scripts/__init__.py
      >   local unversioned, incoming add upon update
D     C LayoutTests/imported/w3c/web-platform-tests/tools/webdriver/webdriver/__init__.py
      >   local unversioned, incoming add upon update
D     C Tools/Scripts/webkitpy/thirdparty/irc
      >   local unversioned, incoming add upon update
D       Tools/Scripts/webkitpy/thirdparty/irc/__init__.py
D       Tools/Scripts/webkitpy/thirdparty/irc/ircbot.py
D       Tools/Scripts/webkitpy/thirdparty/irc/irclib.py
?       layout-test-results.zip
Summary of conflicts:
  Text conflicts: 7
  Tree conflicts: 4

[Csaba Osztrogonác]

filed a new bug report to handle this kind of issues in the future - bug142400

[David Kilzer (:ddkilzer)]

(In reply to comment #24)
> filed a new bug report to handle this kind of issues in the future -
> bug142400

Log of attempted cleanup of one bot.

svn-resolve doesn't do what you'd hope:

$ svn resolve --accept base --recursive Tools/Scripts/webkitpy/thirdparty
svn: warning: W155027: Tree conflicts can only be resolved to 'working' state; '/Volumes/Data/slave/gala-production-wk2-tests/build/OpenSource/Tools/Scripts/webkitpy/thirdparty/irc' not resolved
bot172:OpenSource buildbot$ svn stat .
D     C Tools/Scripts/webkitpy/thirdparty/irc
      >   local unversioned, incoming add upon update
D       Tools/Scripts/webkitpy/thirdparty/irc/__init__.py
D       Tools/Scripts/webkitpy/thirdparty/irc/ircbot.py
D       Tools/Scripts/webkitpy/thirdparty/irc/irclib.py
?       layout-test-results.zip
?       local
Summary of conflicts:
  Tree conflicts: 1

So you have to use svn-revert:

$ svn revert --recursive Tools/Scripts/webkitpy/thirdparty/
Reverted 'Tools/Scripts/webkitpy/thirdparty/irc'
Reverted 'Tools/Scripts/webkitpy/thirdparty/irc/__init__.py'
Reverted 'Tools/Scripts/webkitpy/thirdparty/irc/ircbot.py'
Reverted 'Tools/Scripts/webkitpy/thirdparty/irc/irclib.py'
bot172:OpenSource buildbot$ svn stat .
?       layout-test-results.zip
?       local

After that, you're done.