Bug 127296 - Make uploading results to EWS work despite robots.txt
Summary: Make uploading results to EWS work despite robots.txt
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Alexey Proskuryakov
Depends on:
Reported: 2014-01-20 09:43 PST by Alexey Proskuryakov
Modified: 2014-01-20 13:14 PST (History)
3 users (show)

See Also:

proposed patch (2.05 KB, patch)
2014-01-20 09:46 PST, Alexey Proskuryakov
rniwa: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Alexey Proskuryakov 2014-01-20 09:43:32 PST
Turns out that we consult robots.txt before talking to EWS. We shouldn't - it's a waste of time, and prevents us from easily blocking search indexers.
Comment 1 Alexey Proskuryakov 2014-01-20 09:46:47 PST
Created attachment 221667 [details]
proposed patch
Comment 2 Alexey Proskuryakov 2014-01-20 10:20:33 PST
Committed <http://trac.webkit.org/r162357>.
Comment 3 Alexey Proskuryakov 2014-01-20 13:14:51 PST
Fixed webkitpy regression tests (hopefully) in <http://trac.webkit.org/r162371>.