127296 2014-01-20 09:43:32 -0800 Make uploading results to EWS work despite robots.txt 2014-01-20 13:14:51 -0800 1 1 1 Unclassified WebKit Tools / Tests 528+ (Nightly build) Unspecified Unspecified RESOLVED FIXED P2 Normal --- 1 ap ap commit-queue glenn rniwa oldest_to_newest 969891 0 ap 2014-01-20 09:43:32 -0800 Turns out that we consult robots.txt before talking to EWS. We shouldn't - it's a waste of time, and prevents us from easily blocking search indexers. 969893 1 221667 ap 2014-01-20 09:46:47 -0800 Created attachment 221667 proposed patch 969906 2 ap 2014-01-20 10:20:33 -0800 Committed <http://trac.webkit.org/r162357>. 969975 3 ap 2014-01-20 13:14:51 -0800 Fixed webkitpy regression tests (hopefully) in <http://trac.webkit.org/r162371>. 221667 2014-01-20 09:46:47 -0800 2014-01-20 10:13:32 -0800 proposed patch NoRobots.txt text/plain 2097 ap SW5kZXg6IFRvb2xzL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9DaGFuZ2VMb2cJKHJl dmlzaW9uIDE2MjM1MykKKysrIFRvb2xzL0NoYW5nZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwz ICsxLDE3IEBACisyMDE0LTAxLTIwICBBbGV4ZXkgUHJvc2t1cnlha292ICA8YXBAYXBwbGUuY29t PgorCisgICAgICAgIE1ha2UgdXBsb2FkaW5nIHJlc3VsdHMgdG8gRVdTIHdvcmsgZGVzcGl0ZSBy b2JvdHMudHh0CisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNnaT9p ZD0xMjcyOTYKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAgICAg ICAqIFNjcmlwdHMvd2Via2l0cHkvY29tbW9uL25ldC9zdGF0dXNzZXJ2ZXIucHk6IChTdGF0dXNT ZXJ2ZXIuX19pbml0X18pOiBEaXNhYmxlCisgICAgICAgIGNoZWNraW5nIHJvYm90cy50eHQuCisK KyAgICAgICAgKiBTY3JpcHRzL3dlYmtpdHB5L2NvbW1vbi9uZXQvYnVnemlsbGEvYnVnemlsbGEu cHk6IChCdWd6aWxsYS5fZ2V0X2Jyb3dzZXIpOgorICAgICAgICBSZW1vdmVkIGEgY29tbWVudCBz YXlpbmcgdGhhdCB3ZSB3b3VsZCB3YW50IHRvIHN0b3AgZG9pbmcgdGhpcy4gTm90IGNoZWNraW5n CisgICAgICAgIHJvYm90cy50eHQgaXMgbW9yZSBlZmZpY2llbnQgdGhhbiBjaGVja2luZyBpdC4K KwogMjAxNC0wMS0xOSAgUnl1YW4gQ2hvaSAgPHJ5dWFuLmNob2lAc2Ftc3VuZy5jb20+CiAKICAg ICAgICAgW0VGTF1bV0sxXSBSZW1vdmUgYW5kIHJlZmFjdG9yIHRoZSBBUElzIHdoaWNoIHVzZSBz aW5nbGUgYW5kIGJhc2Uga2V5d29yZApJbmRleDogVG9vbHMvU2NyaXB0cy93ZWJraXRweS9jb21t b24vbmV0L3N0YXR1c3NlcnZlci5weQo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBUb29scy9TY3JpcHRzL3dlYmtp dHB5L2NvbW1vbi9uZXQvc3RhdHVzc2VydmVyLnB5CShyZXZpc2lvbiAxNjIzNDkpCisrKyBUb29s cy9TY3JpcHRzL3dlYmtpdHB5L2NvbW1vbi9uZXQvc3RhdHVzc2VydmVyLnB5CSh3b3JraW5nIGNv cHkpCkBAIC00Myw2ICs0Myw3IEBAIGNsYXNzIFN0YXR1c1NlcnZlcjoKICAgICAgICAgc2VsZi5z ZXRfaG9zdChob3N0KQogICAgICAgICBmcm9tIHdlYmtpdHB5LnRoaXJkcGFydHkuYXV0b2luc3Rh bGxlZC5tZWNoYW5pemUgaW1wb3J0IEJyb3dzZXIKICAgICAgICAgc2VsZi5fYnJvd3NlciA9IGJy b3dzZXIgb3IgQnJvd3NlcigpCisgICAgICAgIHNlbGYuX2Jyb3dzZXIuc2V0X2hhbmRsZV9yb2Jv dHMoRmFsc2UpCiAgICAgICAgIHNlbGYuc2V0X2JvdF9pZChib3RfaWQpCiAKICAgICBkZWYgc2V0 X2hvc3Qoc2VsZiwgaG9zdCk6CkluZGV4OiBUb29scy9TY3JpcHRzL3dlYmtpdHB5L2NvbW1vbi9u ZXQvYnVnemlsbGEvYnVnemlsbGEucHkKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gVG9vbHMvU2NyaXB0cy93ZWJr aXRweS9jb21tb24vbmV0L2J1Z3ppbGxhL2J1Z3ppbGxhLnB5CShyZXZpc2lvbiAxNjIzNDkpCisr KyBUb29scy9TY3JpcHRzL3dlYmtpdHB5L2NvbW1vbi9uZXQvYnVnemlsbGEvYnVnemlsbGEucHkJ KHdvcmtpbmcgY29weSkKQEAgLTMxMyw3ICszMTMsNiBAQCBjbGFzcyBCdWd6aWxsYShvYmplY3Qp OgogICAgICAgICAgICAgc2VsZi5zZXRkZWZhdWx0dGltZW91dCg2MDApCiAgICAgICAgICAgICBm cm9tIHdlYmtpdHB5LnRoaXJkcGFydHkuYXV0b2luc3RhbGxlZC5tZWNoYW5pemUgaW1wb3J0IEJy b3dzZXIKICAgICAgICAgICAgIHNlbGYuX2Jyb3dzZXIgPSBCcm93c2VyKCkKLSAgICAgICAgICAg ICMgSWdub3JlIGJ1Z3Mud2Via2l0Lm9yZy9yb2JvdHMudHh0IHVudGlsIHdlIGZpeCBpdCB0byBh bGxvdyB0aGlzIHNjcmlwdC4KICAgICAgICAgICAgIHNlbGYuX2Jyb3dzZXIuc2V0X2hhbmRsZV9y b2JvdHMoRmFsc2UpCiAgICAgICAgIHJldHVybiBzZWxmLl9icm93c2VyCiAK