Bug 195952 – REGRESSION (r242911?): High Sierra Release WK2 Perf bot timing out while running IndexedDB/large-number-of-inserts.html

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Ryan Haddad]

The High Sierra Release WK2 Perf bot hitting a Buildbot step timeout of 20m without output while running IndexedDB/large-number-of-inserts.html

https://build.webkit.org/builders/Apple%20High%20Sierra%20Release%20WK2%20%28Perf%29/builds/612/steps/perf-test/logs/stdio

Based on the history of the queue, it looks like the last successful run was with r242891 and the first failing was r 242912

[Ryan Haddad]

"Check IDB quota usage through QuotaManager" seems like a potential culprit:
https://trac.webkit.org/changeset/242911/webkit

[youenn fablet]

Will look at it tonight

[youenn fablet]

So, a test that is doing about the same thing (PerformanceTests/IndexedDB/large-number-of-inserts-responsiveness.html) is taking twice the time between run 610 and 612.
The median test run remains of the same order though.

It might be that cache quota is hit. This triggers some IPC to the UIProcess which triggers some additional overhead.
We might be able to fix that by increasing the default quota for these tests.

[youenn fablet]

Additionally, we could also increase the build bot timeout above 20 minutes for perf tests and/or decrease the test iteration from 20 down to 10.

[youenn fablet]

Created attachment 365376 [details]
Patch

[EWS Watchlist]

Comment on attachment 365376 [details]
Patch

Attachment 365376 [details] did not pass mac-wk2-ews (mac-wk2):
Output: https://webkit-queues.webkit.org/results/11585671

New failing tests:
http/wpt/mediarecorder/MediaRecorder-AV-audio-video-dataavailable.html

[EWS Watchlist]

Created attachment 365417 [details]
Archive of layout-test-results from ews105 for mac-highsierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews105  Port: mac-highsierra-wk2  Platform: Mac OS X 10.13.6

[EWS Watchlist]

Comment on attachment 365376 [details]
Patch

Attachment 365376 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: https://webkit-queues.webkit.org/results/11586009

New failing tests:
fast/visual-viewport/ios/min-scale-greater-than-one.html

[EWS Watchlist]

Created attachment 365432 [details]
Archive of layout-test-results from ews124 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews124  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.6

[youenn fablet]

Comment on attachment 365376 [details]
Patch

Both errors unrelated

[Brady Eidson]

Comment on attachment 365376 [details]
Patch

242911 slowed down IDB enough to make this test timeout, and our solution is to reduce the magnitude of the test?

[youenn fablet]

(In reply to Brady Eidson from comment #11)
> Comment on attachment 365376 [details]
> Patch
> 
> 242911 slowed down IDB enough to make this test timeout, and our solution is
> to reduce the magnitude of the test?

The first step is to make the test run again instead of timing out, hence reducing the magnitude of the test. This patch is only doing this.

The second step is to add an API to increase the default quota to a bigger value so that we do not hit the quota-check IPC cost from NetworkProcess to UIProcess.
My guess is that this eats time for the first test.

[Ryosuke Niwa]

I feel like the latest time Sihui fixed this test, the issue was that our IDB limit was too small? Do you recall that, Sihui?

[youenn fablet]

(In reply to Ryosuke Niwa from comment #13)
> I feel like the latest time Sihui fixed this test, the issue was that our
> IDB limit was too small? Do you recall that, Sihui?

I think that something similar to setIDBPerOriginQuota with a big enough value should do the trick. Step 2 is probably just renaming setIDBPerOriginQuota to setPerOriginQuota, piping the provided value to StorageQuotaManager(s) and updating that perf test to set the quota accordingly.

[Radar WebKit Bug Importer]

<rdar://problem/49137688>

[Sihui Liu]

(In reply to youenn fablet from comment #14)
> (In reply to Ryosuke Niwa from comment #13)
> > I feel like the latest time Sihui fixed this test, the issue was that our
> > IDB limit was too small? Do you recall that, Sihui?
> 
> I think that something similar to setIDBPerOriginQuota with a big enough
> value should do the trick. Step 2 is probably just renaming
> setIDBPerOriginQuota to setPerOriginQuota, piping the provided value to
> StorageQuotaManager(s) and updating that perf test to set the quota
> accordingly.

At that time we had a lower default IDB quota for layout tests. Default quota for single layout test is 50MB and for non-test is 500MB. So increasing the quota for some tests still showed us what to expect when opening test page in browser.

If we were about to change the default quota for some test now with presence of StorageQuotaManager, it seems we are ignoring the performance issue? Could we still wait for UI process to respond and return a high quota in the first response?

[Ryan Haddad]

I skipped the test in https://trac.webkit.org/changeset/243604/webkit so that the bot would finish its test runs.

[youenn fablet]

Sihui, given the improvements you made, do you think we can reenable the test?

[Sihui Liu]

Created attachment 386918 [details]
Patch

[WebKit Commit Bot]

Comment on attachment 386918 [details]
Patch

Clearing flags on attachment: 386918

Committed r254240: <https://trac.webkit.org/changeset/254240>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.