Bug 218345 - Allow overriding the S3 endpoint URL
Summary: Allow overriding the S3 endpoint URL
Status: RESOLVED INVALID
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2020-10-29 09:42 PDT by Angelos Oikonomopoulos
Modified: 2022-02-10 16:54 PST (History)
3 users (show)

See Also:


Attachments
Patch (1.46 KB, patch)
2020-10-29 09:46 PDT, Angelos Oikonomopoulos
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Angelos Oikonomopoulos 2020-10-29 09:42:46 PDT
Allow overriding the S3 endpoint URL
Comment 1 Angelos Oikonomopoulos 2020-10-29 09:46:06 PDT
Created attachment 412663 [details]
Patch
Comment 2 lingho@apple.com 2020-10-29 18:29:28 PDT
Looks ok to me. 
boto3.client accepts the endpoint_url argument with no value.

(Note, I am not a reviewer, so can't r+).
Comment 3 Jonathan Bedard 2020-10-29 20:05:47 PDT
What is the motivation for this change?
Comment 4 Angelos Oikonomopoulos 2020-10-30 04:25:12 PDT
(In reply to Jonathan Bedard from comment #3)
> What is the motivation for this change?

The motivation is being able to run a replica of the buildbot infra in a staging environment, so as to be able to try out proposed configuration changes before rolling them out in the live environment.

In order to do that, we need to run separate (throwaway) instances of the buildbot server and also be able to override the use of specific S3 buckets. We could do that by resolving the hardcoded amazon addresses to local IPs, but then we'd also need to create a private CA, propagate the certs etc. With the proposed change, we can simply have the endpoint url be http://something:someport so that things trivially work out of the box.

A separate patch would add the ability to override the hardcoded S3URL in ews-build/steps.py.
Comment 5 Angelos Oikonomopoulos 2020-10-30 08:16:55 PDT
After discussing things with Aakash Jain, I'll be submitting patches along a different direction.
Comment 6 Aakash Jain 2020-11-02 08:47:47 PST
Basically we don't need to modify this script. Instead testing instances should completely skip the transfer-to-s3 step. Done in https://bugs.webkit.org/show_bug.cgi?id=218439