Bug 165222 – REGRESSION (r209144): imported/w3c/web-platform-tests/IndexedDB/idbindex-* tests are flaky in debug builds

• Component: New Bugs
• Status: RESOLVED DUPLICATE

[Ryan Haddad]

The following tests have been flaky since https://trac.webkit.org/changeset/209144 landed.

imported/w3c/web-platform-tests/IndexedDB/idbindex-multientry.htm
imported/w3c/web-platform-tests/IndexedDB/idbindex-rename-abort.html
imported/w3c/web-platform-tests/IndexedDB/idbindex-rename-errors.html

https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r209145%20(9666)/results.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2FIndexedDB%2Fidbindex-multientry.htm

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2FIndexedDB%2Fidbindex-rename-abort.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=imported%2Fw3c%2Fweb-platform-tests%2FIndexedDB%2Fidbindex-rename-errors.html

--- /Volumes/Data/slave/elcapitan-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/IndexedDB/idbindex-multientry-expected.txt
+++ /Volumes/Data/slave/elcapitan-debug-tests-wk2/build/layout-test-results/imported/w3c/web-platform-tests/IndexedDB/idbindex-multientry-actual.txt
@@ -1,3 +1,5 @@
 
-PASS IDBIndex.multiEntry - adding keys 
+Harness Error (TIMEOUT), message = null
 
+NOTRUN IDBIndex.multiEntry - adding keys 
+

[Ryan Haddad]

This affects macOS Debug WK2

[Brady Eidson]

Hammering these in WK2 Debug locally, all sorts of configurations of run-webkit-tests.

Cannot reproduce.

[Brady Eidson]

Wonder if there's any Database process crashes and they're just not coming up in the results?

[Ryan Haddad]

(In reply to comment #3)
> Wonder if there's any Database process crashes and they're just not coming
> up in the results?

I'll see if I can find anything.

[Alexey Proskuryakov]

Brady, how long do these tests take when you run them? If they are slow locally, timing out on the bots wouldn't be surprising.

[Ryan Haddad]

(In reply to comment #4)
> (In reply to comment #3)
> > Wonder if there's any Database process crashes and they're just not coming
> > up in the results?
> 
> I'll see if I can find anything.

I don't see any database process crashlogs on the bot that most recently had these failures.

[Brady Eidson]

(In reply to comment #5)
> Brady, how long do these tests take when you run them? If they are slow
> locally, timing out on the bots wouldn't be surprising.

Extremely quickly.

With webkit-httpd running, the command:
`DYLD_FRAMEWORK_PATH=~/build/Debug/ ~/build/Debug/WebKitTestRunner http://127.0.0.1:8800/IndexedDB/idbindex-multientry.htm http://127.0.0.1:8800/IndexedDB/idbindex-rename-abort.html http://127.0.0.1:8800/IndexedDB/idbindex-rename-errors.html`

...takes well under a second to successfully complete.

[Brady Eidson]

According to the flakiness dashboard, these literally never timeout on Release builds.
But on Debug builds they take *a long* time.

As mentioned in comment 7, I can't reproduce these locally even with a debug build. They are very fast.

[Brady Eidson]

I'm also trying to reason through why they could possibly be related to this patch, and I got nothin'

[Brady Eidson]

Even completes quickly under gmalloc (locally)

[Brady Eidson]

We should mark them flakey for debug only.

[Ryan Haddad]

(In reply to comment #11)
> We should mark them flakey for debug only.

Done in http://trac.webkit.org/projects/webkit/changeset/209272

[Matt Lewis]

These have recently started being flaky on El Capitan Release:

https://build.webkit.org/results/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/r224231%20(5639)/results.html
https://build.webkit.org/builders/Apple%20El%20Capitan%20Release%20WK2%20(Tests)/builds/5639

[Sihui Liu]

Expectations updated at https://bugs.webkit.org/show_bug.cgi?id=228241 as the test is no longer flaky.

*** This bug has been marked as a duplicate of bug 228241 ***