Bug 203246 - EWS bots failing with to reach amazon aws
Summary: EWS bots failing with to reach amazon aws
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Aakash Jain
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2019-10-22 08:02 PDT by Aakash Jain
Modified: 2019-10-22 21:46 PDT (History)
5 users (show)

See Also:


Attachments
Screenshot - Amazon DNS Resolution Error (96.86 KB, image/png)
2019-10-22 14:28 PDT, Aakash Jain
no flags Details
Screenshot - [RESOLVED] Intermittent DNS Resolution Errors (113.14 KB, image/png)
2019-10-22 21:46 PDT, Aakash Jain
no flags Details

Note You need to log in before you can comment on or make changes to this bug.
Comment 1 Aakash Jain 2019-10-22 08:06:12 PDT
This issue seems to have started this morning (~3 hours ago).

First noticed failure in: https://ews-build.webkit.org/#/builders/24/builds/2224
Comment 2 Ling Ho 2019-10-22 09:31:45 PDT
DNS service Looks ok.

ews116:~ buildbot$ host s3-us-west-2.amazonaws.com
s3-us-west-2.amazonaws.com has address 52.218.229.192
Comment 3 Alexey Proskuryakov 2019-10-22 12:44:04 PDT
Amazon status dashboard is now updated with DNS resolution flakiness: https://status.aws.amazon.com

Aakash is looking into whether switching to a different address will help.
Comment 4 Aakash Jain 2019-10-22 14:28:06 PDT
Created attachment 381607 [details]
Screenshot - Amazon DNS Resolution Error
Comment 5 Aakash Jain 2019-10-22 14:30:57 PDT
The issues are intermittent, and due to DNS Resolution Errors on Amazon side.

I have added a fallback option (to download build archive from build master) in https://bugs.webkit.org/show_bug.cgi?id=203263
Comment 6 Aakash Jain 2019-10-22 21:38:16 PDT
As per https://status.aws.amazon.com:

[RESOLVED] Intermittent DNS Resolution Errors
Between 10:30 AM and 6:30 PM PDT, we experienced intermittent errors with resolution of some AWS DNS names. Beginning at 5:16 PM, a very small number of specific DNS names experienced a higher error rate. These issues have been resolved.
Comment 7 Aakash Jain 2019-10-22 21:46:29 PDT
Created attachment 381652 [details]
Screenshot - [RESOLVED] Intermittent DNS Resolution Errors