Bug 164797 - js/dom/domjit-function-get-element-by-id-licm.html and js/dom/domjit-function-get-element-by-id-changed.html are flaky timeouts
Summary: js/dom/domjit-function-get-element-by-id-licm.html and js/dom/domjit-function...
Status: REOPENED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Yusuke Suzuki
URL:
Keywords:
Depends on: 163305 164175
Blocks:
  Show dependency treegraph
 
Reported: 2016-11-15 14:48 PST by Ryan Haddad
Modified: 2016-12-08 13:33 PST (History)
5 users (show)

See Also:


Attachments
mark tests as flaky (1.84 KB, patch)
2016-11-15 15:19 PST, Yusuke Suzuki
sbarati: review+
commit-queue: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Comment 1 Ryan Haddad 2016-11-15 14:50:05 PST
Flakiness dashboard isn't updating at the moment, so I'll try to gather some data on this manually.
Comment 2 Ryan Haddad 2016-11-15 14:50:32 PST
js/dom/domjit-function-get-element-by-id-changed.html timeout on El Capitan Debug WK2:

https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r208746%20(9448)/results.html
Comment 3 Ryan Haddad 2016-11-15 14:50:32 PST
js/dom/domjit-function-get-element-by-id-changed.html timeout on El Capitan Debug WK2:

https://build.webkit.org/results/Apple%20El%20Capitan%20Debug%20WK2%20(Tests)/r208746%20(9448)/results.html
Comment 4 Ryan Haddad 2016-11-15 14:50:54 PST
Tests were added with https://trac.webkit.org/changeset/208412
Comment 5 Yusuke Suzuki 2016-11-15 15:00:00 PST
I guess this becomes flaky after r208588 is landed, correct?
If so, I think this is due to the following reason.

The performance of the both tests rely on PureGetById.
But PureGetById is reverted recently in r208588.
Then, this revert makes the both tests flaky.
Once PureGetById patch is relanded OR this change[1] is landed, I believe the both tests becomes unflaky.

Until PureGetById patch is relanded or impure object patch[1] is landed, I think making the both tests TIMEOUT or SKIP is better.

What do you think of?

[1]: https://bugs.webkit.org/show_bug.cgi?id=164175
Comment 6 Ryan Haddad 2016-11-15 15:05:27 PST
I'm alright with marking them as flaky for the time being while we wait for the patch referenced above is landed.
Comment 7 Yusuke Suzuki 2016-11-15 15:19:05 PST
Created attachment 294886 [details]
mark tests as flaky
Comment 8 WebKit Commit Bot 2016-11-16 12:39:47 PST
Comment on attachment 294886 [details]
mark tests as flaky

Rejecting attachment 294886 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-02', 'apply-attachment', '--no-update', '--non-interactive', 294886, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Last 500 characters of output:
y', '--force', '--reviewer', u'Saam Barati']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Parsed 2 diffs from patch file(s).
patching file LayoutTests/ChangeLog
Hunk #1 succeeded at 1 with fuzz 3.
patching file LayoutTests/TestExpectations
Hunk #1 FAILED at 985.
1 out of 1 hunk FAILED -- saving rejects to file LayoutTests/TestExpectations.rej

Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force', '--reviewer', u'Saam Barati']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Full output: http://webkit-queues.webkit.org/results/2527044
Comment 9 Yusuke Suzuki 2016-11-16 13:02:22 PST
Committed r208807: <http://trac.webkit.org/changeset/208807>
Comment 10 Ryan Haddad 2016-11-18 14:27:26 PST
Reopening because the flaky expectation for these tests was removed, but they are still flaky.
Comment 11 Ryan Haddad 2016-11-18 14:28:02 PST
(In reply to comment #10)
> Reopening because the flaky expectation for these tests was removed, but
> they are still flaky.

https://trac.webkit.org/changeset/208824/trunk/LayoutTests/TestExpectations
Comment 12 Yusuke Suzuki 2016-11-18 14:28:34 PST
(In reply to comment #11)
> (In reply to comment #10)
> > Reopening because the flaky expectation for these tests was removed, but
> > they are still flaky.
> 
> https://trac.webkit.org/changeset/208824/trunk/LayoutTests/TestExpectations

Oops, I accidentally removed it. I'll land it soon.
Comment 13 Ryan Haddad 2016-11-18 14:32:10 PST
Marked as flaky again in http://trac.webkit.org/projects/webkit/changeset/208900
Comment 14 Yusuke Suzuki 2016-11-18 14:32:59 PST
(In reply to comment #13)
> Marked as flaky again in
> http://trac.webkit.org/projects/webkit/changeset/208900

Thanks so much.
Comment 15 Alexey Proskuryakov 2016-12-08 13:33:53 PST
This still happens on the bots very frequently. It seems like the tests are just ultra slow in debug - they take around 30 seconds consistently.