Bug 177614 - check in AWS Lambda code used for WebKitArchiveSupport
Summary: check in AWS Lambda code used for WebKitArchiveSupport
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Lucas Forschler
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2017-09-28 12:39 PDT by Lucas Forschler
Modified: 2017-09-28 14:43 PDT (History)
2 users (show)

See Also:


Attachments
v1 patch for review (7.77 KB, patch)
2017-09-28 12:45 PDT, Lucas Forschler
ap: review+
lforschler: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Lucas Forschler 2017-09-28 12:39:18 PDT
Keep AWS Lambda code in source control somewhere... 
This is the code used to support the WebKitArchives.
Comment 1 Radar WebKit Bug Importer 2017-09-28 12:40:23 PDT
<rdar://problem/34721629>
Comment 2 Lucas Forschler 2017-09-28 12:45:28 PDT
Created attachment 322110 [details]
v1 patch for review
Comment 3 Lucas Forschler 2017-09-28 12:46:38 PDT
Note: AWS will not allow a lambda function to be used for multiple triggers. Therefore, these files are pretty similar, only differing in a few variables/constant values.
Comment 4 Alexey Proskuryakov 2017-09-28 14:29:10 PDT
Comment on attachment 322110 [details]
v1 patch for review

View in context: https://bugs.webkit.org/attachment.cgi?id=322110&action=review

rs=me, as this code is in production already.

Would be nice to have regression tests for it.

> Tools/WebKitArchiveSupport/lambda/delete-minified-s3-archive-from-dynamodb.py:32
> +        print('Error deleting item: {}\nfrom database: {}.'.format(item, table_name))
> +        raise e

Where do these errors go? How will we notice them?

> Tools/WebKitArchiveSupport/lambda/register-minified-s3-archive-in-dynamodb.py:26
> +    split_key = key.split('/')
> +    identifier = split_key[0]           # mac-sierra-x86_64-debug
> +    filename = split_key[1]             # 218331.zip
> +    revision = filename.split('.')[0]   # 218331

Is there a way to avoid duplicating code?
Comment 5 Alexey Proskuryakov 2017-09-28 14:30:52 PDT
Comment on attachment 322110 [details]
v1 patch for review

It may be useful to have comments explaining what event triggers these lambdas.
Comment 6 Lucas Forschler 2017-09-28 14:42:32 PDT
Comment on attachment 322110 [details]
v1 patch for review

View in context: https://bugs.webkit.org/attachment.cgi?id=322110&action=review

>> Tools/WebKitArchiveSupport/lambda/delete-minified-s3-archive-from-dynamodb.py:32
>> +        raise e
> 
> Where do these errors go? How will we notice them?

These errors are sent to Amazon CloudWatch. We would never see them unless we noticed problems and went back through logs to figure out what is going wrong.
It might be possible to configure events to watch for specific errors, I can investigate.

>> Tools/WebKitArchiveSupport/lambda/register-minified-s3-archive-in-dynamodb.py:26
>> +    revision = filename.split('.')[0]   # 218331
> 
> Is there a way to avoid duplicating code?

I've read several articles on this, and haven't found any solutions that look reasonable. It's something I will watch out for as we grow our infrastructure.
Comment 7 Lucas Forschler 2017-09-28 14:43:57 PDT
Committed revision 222634.