Bug 164797 – js/dom/domjit-function-get-element-by-id-licm.html and js/dom/domjit-function-get-element-by-id-changed.html are flaky timeouts

• Component: New Bugs
• Status: REOPENED

[Ryan Haddad]

js/dom/domjit-function-get-element-by-id-licm.html and js/dom/domjit-function-get-element-by-id-changed.html are flaky timeouts

https://build.webkit.org/results/Apple%20Yosemite%20Debug%20WK2%20(Tests)/r208747%20(16237)/results.html

http://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=js%2Fdom%2Fdomjit-function-get-element-by-id-changed.html
http://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=js%2Fdom%2Fdomjit-function-get-element-by-id-licm.html

[Ryan Haddad]

Flakiness dashboard isn't updating at the moment, so I'll try to gather some data on this manually.

[Ryan Haddad]

js/dom/domjit-function-get-element-by-id-changed.html timeout on El Capitan Debug WK2:

https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r208746%20(9448)/results.html

[Ryan Haddad]

js/dom/domjit-function-get-element-by-id-changed.html timeout on El Capitan Debug WK2:

https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r208746%20(9448)/results.html

[Ryan Haddad]

Tests were added with https://trac.webkit.org/changeset/208412

[Yusuke Suzuki]

I guess this becomes flaky after r208588 is landed, correct?
If so, I think this is due to the following reason.

The performance of the both tests rely on PureGetById.
But PureGetById is reverted recently in r208588.
Then, this revert makes the both tests flaky.
Once PureGetById patch is relanded OR this change[1] is landed, I believe the both tests becomes unflaky.

Until PureGetById patch is relanded or impure object patch[1] is landed, I think making the both tests TIMEOUT or SKIP is better.

What do you think of?

[1]: https://bugs.webkit.org/show_bug.cgi?id=164175

[Ryan Haddad]

I'm alright with marking them as flaky for the time being while we wait for the patch referenced above is landed.

[Yusuke Suzuki]

Created attachment 294886 [details]
mark tests as flaky

[WebKit Commit Bot]

Comment on attachment 294886 [details]
mark tests as flaky

Rejecting attachment 294886 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-02', 'apply-attachment', '--no-update', '--non-interactive', 294886, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Last 500 characters of output:
y', '--force', '--reviewer', u'Saam Barati']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Parsed 2 diffs from patch file(s).
patching file LayoutTests/ChangeLog
Hunk #1 succeeded at 1 with fuzz 3.
patching file LayoutTests/TestExpectations
Hunk #1 FAILED at 985.
1 out of 1 hunk FAILED -- saving rejects to file LayoutTests/TestExpectations.rej

Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force', '--reviewer', u'Saam Barati']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Full output: http://webkit-queues.webkit.org/results/2527044

[Yusuke Suzuki]

Committed r208807: <http://trac.webkit.org/changeset/208807>

[Ryan Haddad]

Reopening because the flaky expectation for these tests was removed, but they are still flaky.

[Ryan Haddad]

(In reply to comment #10)
> Reopening because the flaky expectation for these tests was removed, but
> they are still flaky.

https://trac.webkit.org/changeset/208824/trunk/LayoutTests/TestExpectations

[Yusuke Suzuki]

(In reply to comment #11)
> (In reply to comment #10)
> > Reopening because the flaky expectation for these tests was removed, but
> > they are still flaky.
> 
> https://trac.webkit.org/changeset/208824/trunk/LayoutTests/TestExpectations

Oops, I accidentally removed it. I'll land it soon.

[Ryan Haddad]

Marked as flaky again in http://trac.webkit.org/projects/webkit/changeset/208900

[Yusuke Suzuki]

(In reply to comment #13)
> Marked as flaky again in
> http://trac.webkit.org/projects/webkit/changeset/208900

Thanks so much.

[Alexey Proskuryakov]

This still happens on the bots very frequently. It seems like the tests are just ultra slow in debug - they take around 30 seconds consistently.