Bug 218345

Summary: Allow overriding the S3 endpoint URL
Product: WebKit Reporter: Angelos Oikonomopoulos <angelos>
Component: New BugsAssignee: Nobody <webkit-unassigned>
Status: RESOLVED INVALID    
Severity: Normal CC: aakash_jain, jbedard, lingho
Priority: P2    
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch none

Description Angelos Oikonomopoulos 2020-10-29 09:42:46 PDT
Allow overriding the S3 endpoint URL
Comment 1 Angelos Oikonomopoulos 2020-10-29 09:46:06 PDT
Created attachment 412663 [details]
Patch
Comment 2 lingho@apple.com 2020-10-29 18:29:28 PDT
Looks ok to me. 
boto3.client accepts the endpoint_url argument with no value.

(Note, I am not a reviewer, so can't r+).
Comment 3 Jonathan Bedard 2020-10-29 20:05:47 PDT
What is the motivation for this change?
Comment 4 Angelos Oikonomopoulos 2020-10-30 04:25:12 PDT
(In reply to Jonathan Bedard from comment #3)
> What is the motivation for this change?

The motivation is being able to run a replica of the buildbot infra in a staging environment, so as to be able to try out proposed configuration changes before rolling them out in the live environment.

In order to do that, we need to run separate (throwaway) instances of the buildbot server and also be able to override the use of specific S3 buckets. We could do that by resolving the hardcoded amazon addresses to local IPs, but then we'd also need to create a private CA, propagate the certs etc. With the proposed change, we can simply have the endpoint url be http://something:someport so that things trivially work out of the box.

A separate patch would add the ability to override the hardcoded S3URL in ews-build/steps.py.
Comment 5 Angelos Oikonomopoulos 2020-10-30 08:16:55 PDT
After discussing things with Aakash Jain, I'll be submitting patches along a different direction.
Comment 6 Aakash Jain 2020-11-02 08:47:47 PST
Basically we don't need to modify this script. Instead testing instances should completely skip the transfer-to-s3 step. Done in https://bugs.webkit.org/show_bug.cgi?id=218439