Bug 174261
Summary: | Teach buildbot testers to download archives from s3 | ||
---|---|---|---|
Product: | WebKit | Reporter: | Lucas Forschler <lforschler> |
Component: | Tools / Tests | Assignee: | Lucas Forschler <lforschler> |
Status: | RESOLVED CONFIGURATION CHANGED | ||
Severity: | Normal | CC: | ap, lforschler, webkit-bug-importer |
Priority: | P2 | Keywords: | InRadar |
Version: | WebKit Nightly Build | ||
Hardware: | Unspecified | ||
OS: | Unspecified |
Lucas Forschler
We are now storing built product archives on s3. We need to teach our build workers to use these archives.
This will involve teaching the download/extract tool about the location of the s3 object.
I see two options:
1. Retrieve the archive via the aws/s3 tool
this is probably the easiest, but will require installing the AWS tools on the bot. I'd prefer not to introduce another dependency.
2. Retrieve the archive via https
this is likely the cleanest solution, since the files on s3 are public and served via https.
Attachments | ||
---|---|---|
Add attachment proposed patch, testcase, etc. |
Radar WebKit Bug Importer
<rdar://problem/33184218>
Lucas Forschler
note: The builders currently upload to the buildbot master. The master then transfers the file to S3. After this is working, we need to find a good way to remove the archive from the master... This could possibly be done in the same script where we transfer to S3. If the archive is uploaded successfully, simply delete the local copy.
Alexey Proskuryakov
Are the S3 versions minified? What exactly is the impact on c ash log quality?
Lucas Forschler
*** Bug 174384 has been marked as a duplicate of this bug. ***
Lucas Forschler
We keep the full AND Minified version in S3.
Full will stick around for 14 days
Minified will stick around for 2.5 years.
This change will have no impact on what is tested, as the bits are still the same as before. Eventually, we can opt to use the minified versions.
Lucas Forschler
<rdar://problem/33184218>
Lucas Forschler
Committed revision 219358.
Lucas Forschler
This will require a master restart.
But, I'm going to wait to give the queues a chance to catch up.... since some non-Mac test queues may not have builds on S3 yet. (that change was landed earlier this morning)
Lucas Forschler
oops. Our nodes cannot reach s3 due to networking permissions. Will need to fix that before restarting the master.
example fail: https://build.webkit.org/builders/Apple%20El%20Capitan%20Debug%20Test262%20%28Tests%29/builds/2613/steps/download-built-product/logs/stdio