Bug 167616 – Performance test IndexedDB/large-binary-keys.html creates a DB of more than 6GB and takes more than 10 minutes to run

• Status: NEW
• Component: Tools / Tests

[Carlos Alberto Lopez Perez]

The Performance test IndexedDB/large-binary-keys.html was timing out on the GTK+ performance bot since it was added on https://trac.webkit.org/changeset/209499.

The main reason was that our Performance bot was using a dedicated /tmp volume with 2GB of space, and this test requires at least ~ 6.5 GB of free disk space on /tmp to successfully run.

$ du -hs /tmp/WebKitTestRunner-pGQSwk
6.6G	/tmp/WebKitTestRunner-pGQSwk

$ find /tmp/WebKitTestRunner-pGQSwk
/tmp/WebKitTestRunner-pGQSwk
/tmp/WebKitTestRunner-pGQSwk/LocalStorage
/tmp/WebKitTestRunner-pGQSwk/Databases
/tmp/WebKitTestRunner-pGQSwk/Databases/IndexedDB
/tmp/WebKitTestRunner-pGQSwk/Databases/IndexedDB/file__0
/tmp/WebKitTestRunner-pGQSwk/Databases/IndexedDB/file__0/large-string-keys-DB-0
/tmp/WebKitTestRunner-pGQSwk/Databases/IndexedDB/file__0/large-string-keys-DB-0/IndexedDB.sqlite3-wal
/tmp/WebKitTestRunner-pGQSwk/Databases/IndexedDB/file__0/large-string-keys-DB-0/IndexedDB.sqlite3-shm
/tmp/WebKitTestRunner-pGQSwk/Databases/IndexedDB/file__0/large-string-keys-DB-0/IndexedDB.sqlite3
/tmp/WebKitTestRunner-pGQSwk/Cache
/tmp/WebKitTestRunner-pGQSwk/Cache/Version 11
/tmp/WebKitTestRunner-pGQSwk/Cache/Version 11/Blobs
/tmp/WebKitTestRunner-pGQSwk/Cache/Version 11/salt


$ ls -1sh /tmp/WebKitTestRunner-pGQSwk/Databases/IndexedDB/file__0/large-string-keys-DB-0
total 6.6G
201M IndexedDB.sqlite3
 50M IndexedDB.sqlite3-shm
6.3G IndexedDB.sqlite3-wal


I have already fixed the issue with getting out of space on the GTK+ perf bot. It has now 30GB for /tmp that should be enough.

So, I open this bug mainly to ask if testing the performance on such monster databases (6.6GB) is something representative of real world use cases or it is something worth testing?

The test also takes a lot to complete, between 30 and 45 minutes.

[Brady Eidson]

I think it's worth testing (It's provided huge benefit already)

[Carlos Alberto Lopez Perez]

(In reply to comment #1)
> I think it's worth testing (It's provided huge benefit already)

On the GTK+ perf bot this test is taking ~1620 seconds, which is beyond the 1200 global timeout for any step to be working without any output. So it causes the whole step to abort.

I will have to skip it on the GTK+ Perf bot until the test is changed to run faster, or some optimization is done on the WebKit/WebKitGTK+ code that makes it run faster.

Note also that I'm also proposing a patch on bug 167626 to bring back the 600 timeout value for any perf test. I don't think a unique test should take more than this (10 minutes) to run. We are already having 3+ hours long perf test steps, and if individual tests are allowed to run for more than 10 minutes we will be soon having much longer perf test steps.

[Carlos Alberto Lopez Perez]

On the Apple port this test takes ~17 minutes https://build.webkit.org/builders/Apple%20El%20Capitan%20Release%20WK2%20%28Perf%29/builds/4138/steps/perf-test/logs/stdio


Running IndexedDB/large-binary-keys.html (78 of 162)
RESULT IndexedDB: large-binary-keys: Time= 12710.29875 ms
median= 12767.6 ms, stdev= 192.085687956 ms, min= 12049.8 ms, max= 12873.7 ms
RESULT IndexedDB: large-binary-keys: JSHeap= 690339155.65 bytes
median= 663655063.0 bytes, stdev= 248329564.206 bytes, min= 248397774.0 bytes, max= 1206834295.0 bytes
RESULT IndexedDB: large-binary-keys: Malloc= 847334707.2 bytes
median= 872071168.0 bytes, stdev= 258591471.735 bytes, min= 369184768.0 bytes, max= 1255882752.0 bytes
Finished: 1048.817416 s


But, on the EFL port however this test only takes 5 minutes!

Running IndexedDB/large-binary-keys.html (78 of 162)
RESULT IndexedDB: large-binary-keys: Time= 6281.455 ms
median= 6316.0 ms, stdev= 255.338836634 ms, min= 5291.3 ms, max= 6786.8 ms
RESULT IndexedDB: large-binary-keys: JSHeap= 469837447.4 bytes
median= 416166533.0 bytes, stdev= 153900199.697 bytes, min= 235825501.0 bytes, max= 835616357.0 bytes
RESULT IndexedDB: large-binary-keys: Malloc= 896776601.6 bytes
median= 873631744.0 bytes, stdev= 193327106.186 bytes, min= 486801408.0 bytes, max= 1116913664.0 bytes
Finished: 531.608296 s

https://build.webkit.org/builders/EFL%20Linux%2064-bit%20Release%20WK2%20%28Perf%29/builds/11181/steps/perf-test/logs/stdio


I wonder what can explain such massive performance improvement on that port?

[Carlos Alberto Lopez Perez]

This test was skipped for the GTK+ Perf bot in https://trac.webkit.org/changeset/211430/trunk/PerformanceTests/Skipped