Bug 174487 - Better deal with changes to the ResourceLoadStatisticsStore file on disk
Summary: Better deal with changes to the ResourceLoadStatisticsStore file on disk
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKit2 (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Chris Dumez
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2017-07-13 19:51 PDT by Chris Dumez
Modified: 2017-07-15 10:13 PDT (History)
3 users (show)

See Also:


Attachments
Patch (11.49 KB, patch)
2017-07-13 20:25 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff
Patch (11.41 KB, patch)
2017-07-13 21:15 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Chris Dumez 2017-07-13 19:51:40 PDT
Better deal with changes to the ResourceLoadStatisticsStore file on disk. We should merge the data on disk with the one we have in memory instead of merely replacing our in-memory data. This avoids data loss when re-syncing from disk.
Comment 1 Chris Dumez 2017-07-13 20:25:45 PDT
Created attachment 315396 [details]
Patch
Comment 2 Chris Dumez 2017-07-13 21:15:01 PDT
Created attachment 315398 [details]
Patch
Comment 3 Brent Fulgham 2017-07-13 22:56:48 PDT
Comment on attachment 315398 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=315398&action=review

Looks good! R=me

> Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.cpp:130
> +    int m_monthDay { 0 }; // [1, 31].

Is this really 1-based? If we are, why is t it initialized with a 1?
Comment 4 Chris Dumez 2017-07-13 23:08:07 PDT
Comment on attachment 315398 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=315398&action=review

>> Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.cpp:130
>> +    int m_monthDay { 0 }; // [1, 31].
> 
> Is this really 1-based? If we are, why is t it initialized with a 1?

Yes, dayInMonthFromDayInYear() returns a value that is 1-based. The default constructor does not construct a valid date so I do not think it matters that we initialize all members to 0 though. The default constructor is only used for the KeyedDecoding and we would not use the date if the decoding failed.
Comment 5 WebKit Commit Bot 2017-07-13 23:39:11 PDT
Comment on attachment 315398 [details]
Patch

Clearing flags on attachment: 315398

Committed r219499: <http://trac.webkit.org/changeset/219499>
Comment 6 WebKit Commit Bot 2017-07-13 23:39:12 PDT
All reviewed patches have been landed.  Closing bug.
Comment 7 Darin Adler 2017-07-15 10:13:29 PDT
Comment on attachment 315398 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=315398&action=review

> Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.cpp:141
> +    std::merge(existingDates.begin(), existingDates.end(), newDates.begin(), newDates.end(), mergedDates.begin());

Iā€™d be tempted to assert std::is_sorted before calling std::merge.

> Source/WebKit/UIProcess/WebResourceLoadStatisticsStore.cpp:147
> +    // Remove duplicate dates.
> +    removeRepeatedElements(mergedDates);
> +
> +    // Drop old dates until the Vector size reaches operatingDatesWindow.
> +    while (mergedDates.size() > operatingDatesWindow)
> +        mergedDates.remove(0);

(When researching this, I discovered that Vector::removeRepeatedElements calls resize instead of calling shrink. It should call shrink.)

This code is unnecessarily inefficient. We should resize only once, not multiple times, and we should reduce the amount we move elements in the vector. Here is the almost identical code I would write that resizes the vector at most once and moves items in the vector at most twice:

    // Remove duplicate dates, then throw away all but the newest dates that fit in the window size.
    auto begin = mergedDates.begin();
    auto end = std::unique(begin, mergedDates.end());
    if (end - begin > operatingDatesWindow)
        end = std::move(end - operatingDatesWindow, end, begin);
    mergedDates.shrink(end - begin);