Bug 105976 - Work around flakyness during autoinstall
Summary: Work around flakyness during autoinstall
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Tim 'mithro' Ansell
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2013-01-02 17:24 PST by Tim 'mithro' Ansell
Modified: 2013-01-09 19:17 PST (History)
5 users (show)

See Also:


Attachments
Patch (2.59 KB, patch)
2013-01-02 18:26 PST, Tim 'mithro' Ansell
no flags Details | Formatted Diff | Diff
Patch (2.54 KB, patch)
2013-01-03 19:37 PST, Tim 'mithro' Ansell
no flags Details | Formatted Diff | Diff
Patch (2.65 KB, patch)
2013-01-03 19:48 PST, Tim 'mithro' Ansell
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tim 'mithro' Ansell 2013-01-02 17:24:11 PST
Work around SourceForge flakyness.
Comment 1 Tim 'mithro' Ansell 2013-01-02 18:26:41 PST
Created attachment 181127 [details]
Patch
Comment 2 Adam Barth 2013-01-03 13:30:11 PST
@eric: Thoughts?
Comment 3 Eric Seidel (no email) 2013-01-03 13:40:05 PST
What do you mean "Flaky"?  Could we list a different SF mirror which is not flaky instead?
Comment 4 Tim 'mithro' Ansell 2013-01-03 15:25:58 PST
The download link is 
 http://downloads.sourceforge.net/project/python-irclib/python-irclib/0.4.8/python-irclib-0.4.8.zip

Which is the SourceForge redirector that sends you to a local mirror.

This redirector very occasionally (like 1 in 100) sends you to a broken mirror and you get an error. The crappy mirrors seem to be mostly in Asia pacific, so you probably don't seen them in the US.

Maybe we should retry all failed requests multiple times?

Tim
Comment 5 Eric Seidel (no email) 2013-01-03 15:31:59 PST
I think it's OK for the autoinstaller to have retry behavior, yes.  I don't think the retry behavior should be limited to sourceforge.  I'm surprised we dont' already have some sort of retry-behavior-aware fetch function for you to use?
Comment 6 Tim 'mithro' Ansell 2013-01-03 15:35:31 PST
If you can find one, I'd be happy to use it. Will upload a patch without the SourceForge specific nature.
Comment 7 Eric Seidel (no email) 2013-01-03 15:42:02 PST
I think NetworkTransaction in common.system.net is what you want. :)
Comment 8 Eric Seidel (no email) 2013-01-03 17:23:26 PST
I should note that (despite having created it) I don't know the current autoinstaller "policies".  it's possible that autoinstaller tries not to include other parts of webkitpy, and thus it may not be able to use NetworkTransaction.  I do not know.
Comment 9 Tim 'mithro' Ansell 2013-01-03 19:37:21 PST
Created attachment 181264 [details]
Patch
Comment 10 Tim 'mithro' Ansell 2013-01-03 19:40:33 PST
I wasn't sure if autoinstall should depend on the common.system.net, so I upload a patch with the SF specific bits removed.
Comment 11 Eric Seidel (no email) 2013-01-03 19:40:59 PST
Comment on attachment 181264 [details]
Patch

OK.  We should probably log with each retry.
Comment 12 Tim 'mithro' Ansell 2013-01-03 19:48:00 PST
Created attachment 181269 [details]
Patch
Comment 13 Tim 'mithro' Ansell 2013-01-03 19:48:16 PST
Added logging.
Comment 14 Eric Seidel (no email) 2013-01-03 19:51:10 PST
Comment on attachment 181269 [details]
Patch

LGTM.
Comment 15 WebKit Review Bot 2013-01-03 20:22:27 PST
Comment on attachment 181269 [details]
Patch

Clearing flags on attachment: 181269

Committed r138776: <http://trac.webkit.org/changeset/138776>
Comment 16 WebKit Review Bot 2013-01-03 20:22:31 PST
All reviewed patches have been landed.  Closing bug.
Comment 17 Dirk Pranke 2013-01-09 18:51:33 PST
For the record, I would be very nervous if we tried to use anything else from webkitpy in autoinstall.py; this function gets called very early on when loading files and I could see us easily triggering circular dependencies if networktransaction.py then tried to import something from webkitpy.thirdparty.

I think it was good to make this standalone and we should probably document this accordingly somewhere.

Or maybe just have a general rule that webkitpy.common.system isn't allowed to import anything else from webkitpy (don't know if it tries to today or not).
Comment 18 Eric Seidel (no email) 2013-01-09 19:13:21 PST
webkitpy.common certainly imports from layout_tests.port, sadly.  My goal would be that layout_tests.port move to webkitpy.common eventually. :)
Comment 19 Dirk Pranke 2013-01-09 19:17:33 PST
(In reply to comment #18)
> webkitpy.common certainly imports from layout_tests.port, sadly.  My goal would be that layout_tests.port move to webkitpy.common eventually. :)

Sure, we should move port, but I'm not actually bothered by webkitpy.common including  webkitpy.layout_tests.port ... it's webkitpy.common.*system* that I'm concerned about, since it's the lowest layer in the stack.