Bug 204806 – results.webkit.org: Decrease ttl for archives

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Jonathan Bedard]

Our test archives take up a lot of space. We should change the default ttl for these archives from 5 years to 8 weeks (~2 months)

[Jonathan Bedard]

<rdar://problem/57527644>

[Jonathan Bedard]

Created attachment 384735 [details]
Patch

[Aakash Jain]

(In reply to Jonathan Bedard from comment #0)
> Our test archives take up a lot of space. We should change the default ttl for these archives from 5 years to 8 weeks (~2 months)
What are the implications of not storing these archives after 2 months? How would end-users be impacted?

[Jonathan Bedard]

(In reply to Aakash Jain from comment #3)
> (In reply to Jonathan Bedard from comment #0)
> > Our test archives take up a lot of space. We should change the default ttl for these archives from 5 years to 8 weeks (~2 months)
> What are the implications of not storing these archives after 2 months? How
> would end-users be impacted?

Right now, since we don't visualize the archives, the impact is nothing.

In the future, when I finish the code to visualize the archives, results more than 2 months old will result in dead links.

[Aakash Jain]

Comment on attachment 384735 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=384735&action=review

> Tools/resultsdbpy/resultsdbpy/model/archive_context.py:55
> +        __table_name__ = 'archive_metadata_by_commit_01'

why do we need to delete the old table and create a new one with different name? Can't you simply login to production instance and manually delete the data from the table?

> Tools/resultsdbpy/resultsdbpy/model/archive_context.py:75
> +        __table_name__ = 'archive_chunks_01'

Ditto.

> Tools/resultsdbpy/resultsdbpy/model/model.py:43
>              raise ValueError('TTL must be at least 4 weeks')

should we have a similar check for minimum value of archive TTL?

[Jonathan Bedard]

Comment on attachment 384735 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=384735&action=review

>> Tools/resultsdbpy/resultsdbpy/model/archive_context.py:55
>> +        __table_name__ = 'archive_metadata_by_commit_01'
> 
> why do we need to delete the old table and create a new one with different name? Can't you simply login to production instance and manually delete the data from the table?

We could. But this is a terabyte of data, so I would need to write something to do the deletion. It's much simpler (and safer) to just drop the table.

>> Tools/resultsdbpy/resultsdbpy/model/model.py:43
>>              raise ValueError('TTL must be at least 4 weeks')
> 
> should we have a similar check for minimum value of archive TTL?

Seems reasonable, I'll add it.

[Jonathan Bedard]

Comment on attachment 384735 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=384735&action=review

>>> Tools/resultsdbpy/resultsdbpy/model/archive_context.py:55
>>> +        __table_name__ = 'archive_metadata_by_commit_01'
>> 
>> why do we need to delete the old table and create a new one with different name? Can't you simply login to production instance and manually delete the data from the table?
> 
> We could. But this is a terabyte of data, so I would need to write something to do the deletion. It's much simpler (and safer) to just drop the table.

Also worth noting that I think there is another bug in these tables that I'm trying to fix before deploying this change around archive duplication.

[Jonathan Bedard]

Created attachment 384743 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 384743 [details]
Patch

Clearing flags on attachment: 384743

Committed r253066: <https://trac.webkit.org/changeset/253066>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Jonathan Bedard]

Committed r253134: <https://trac.webkit.org/changeset/253134>