Bug 105976 – Work around flakyness during autoinstall

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[Tim 'mithro' Ansell]

Work around SourceForge flakyness.

[Tim 'mithro' Ansell]

Created attachment 181127 [details]
Patch

[Adam Barth]

@eric: Thoughts?

[Eric Seidel (no email)]

What do you mean "Flaky"?  Could we list a different SF mirror which is not flaky instead?

[Tim 'mithro' Ansell]

The download link is 
 http://downloads.sourceforge.net/project/python-irclib/python-irclib/0.4.8/python-irclib-0.4.8.zip

Which is the SourceForge redirector that sends you to a local mirror.

This redirector very occasionally (like 1 in 100) sends you to a broken mirror and you get an error. The crappy mirrors seem to be mostly in Asia pacific, so you probably don't seen them in the US.

Maybe we should retry all failed requests multiple times?

Tim

[Eric Seidel (no email)]

I think it's OK for the autoinstaller to have retry behavior, yes.  I don't think the retry behavior should be limited to sourceforge.  I'm surprised we dont' already have some sort of retry-behavior-aware fetch function for you to use?

[Tim 'mithro' Ansell]

If you can find one, I'd be happy to use it. Will upload a patch without the SourceForge specific nature.

[Eric Seidel (no email)]

I think NetworkTransaction in common.system.net is what you want. :)

[Eric Seidel (no email)]

I should note that (despite having created it) I don't know the current autoinstaller "policies".  it's possible that autoinstaller tries not to include other parts of webkitpy, and thus it may not be able to use NetworkTransaction.  I do not know.

[Tim 'mithro' Ansell]

Created attachment 181264 [details]
Patch

[Tim 'mithro' Ansell]

I wasn't sure if autoinstall should depend on the common.system.net, so I upload a patch with the SF specific bits removed.

[Eric Seidel (no email)]

Comment on attachment 181264 [details]
Patch

OK.  We should probably log with each retry.

[Tim 'mithro' Ansell]

Created attachment 181269 [details]
Patch

[Tim 'mithro' Ansell]

Added logging.

[Eric Seidel (no email)]

Comment on attachment 181269 [details]
Patch

LGTM.

[WebKit Review Bot]

Comment on attachment 181269 [details]
Patch

Clearing flags on attachment: 181269

Committed r138776: <http://trac.webkit.org/changeset/138776>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.

[Dirk Pranke]

For the record, I would be very nervous if we tried to use anything else from webkitpy in autoinstall.py; this function gets called very early on when loading files and I could see us easily triggering circular dependencies if networktransaction.py then tried to import something from webkitpy.thirdparty.

I think it was good to make this standalone and we should probably document this accordingly somewhere.

Or maybe just have a general rule that webkitpy.common.system isn't allowed to import anything else from webkitpy (don't know if it tries to today or not).

[Eric Seidel (no email)]

webkitpy.common certainly imports from layout_tests.port, sadly.  My goal would be that layout_tests.port move to webkitpy.common eventually. :)

[Dirk Pranke]

(In reply to comment #18)
> webkitpy.common certainly imports from layout_tests.port, sadly.  My goal would be that layout_tests.port move to webkitpy.common eventually. :)

Sure, we should move port, but I'm not actually bothered by webkitpy.common including  webkitpy.layout_tests.port ... it's webkitpy.common.*system* that I'm concerned about, since it's the lowest layer in the stack.