Bug 159272 - REGRESSION (r202529?) inspector/debugger/break-on-* tests are flaky
Summary: REGRESSION (r202529?) inspector/debugger/break-on-* tests are flaky
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Inspector (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
Keywords: InRadar
Depends on:
Reported: 2016-06-29 13:46 PDT by Ryan Haddad
Modified: 2016-07-05 11:37 PDT (History)
7 users (show)

See Also:

[PATCH] Proposed Fix (104.80 KB, patch)
2016-07-01 17:33 PDT, Joseph Pecoraro
no flags Details | Formatted Diff | Diff
[PATCH] Proposed Fix (109.00 KB, patch)
2016-07-01 17:36 PDT, Joseph Pecoraro
no flags Details | Formatted Diff | Diff
[PATCH] Proposed Fix (104.83 KB, patch)
2016-07-01 17:41 PDT, Joseph Pecoraro
timothy: review+
commit-queue: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ryan Haddad 2016-06-29 13:46:32 PDT
inspector/debugger/break-on-* tests are flaky

These tests are flaky and seem to fail together on mac-wk1 with a similar diff



--- /Volumes/Data/slave/elcapitan-debug-tests-wk1/build/layout-test-results/inspector/debugger/break-on-exception-finally-expected.txt
+++ /Volumes/Data/slave/elcapitan-debug-tests-wk1/build/layout-test-results/inspector/debugger/break-on-exception-finally-actual.txt
@@ -4,6 +4,5 @@
 CONSOLE MESSAGE: line 8: TestError
 Checking pause locations when pausing on all exceptions.
-PAUSE #1 AT: doThrow:8:22
 Uncaught exception in test page: TestError [break-on-exception-tests.js:8]
Comment 1 Radar WebKit Bug Importer 2016-06-29 13:46:56 PDT
Comment 2 Ryan Haddad 2016-06-29 13:47:30 PDT
Seems to have started ~r202537, but I do not see an obviously related change.
Comment 3 Alexey Proskuryakov 2016-06-29 21:21:16 PDT
The first failures that I see are in r202531, in a test run that covered revisions r202531-r202529.

While <http://trac.webkit.org/projects/webkit/changeset/202529> does not look like an obvious culprit, it has to be it with fairly high probability.

This happens in WK2 too.
Comment 4 Joseph Pecoraro 2016-06-29 21:46:10 PDT
One possibility is merely adding a new inspector test causing other to break.

Also, these tests all have InspectorTest.reloadPage, which I suspect has always been the source of some instability given how inspector tests work.

I'd be fine with having these tests just be skipped while I investigate. Just running the inspector/debugger directory I encounter some sporadic issues that will require more involved investigation.
Comment 5 Ryan Haddad 2016-06-30 17:09:04 PDT
Skipped tests to get bots back to green while Joe investigates http://trac.webkit.org/projects/webkit/changeset/202720
Comment 6 Joseph Pecoraro 2016-07-01 17:29:19 PDT
I just completely rewrote all the break-on-* tests for a number of reasons:

    - easier to read and understand using new libraries
    - avoid reload, should make them individually faster
    - merge tests, instead of 1-file-per-test
      - eliminates a ton of dead and stale code in these tests

That said, I think an issue is that the bots might be running them slowly, so I'm going to mark them as [ Slow ] when first enabling them.
Comment 7 Joseph Pecoraro 2016-07-01 17:33:07 PDT
Created attachment 282613 [details]
[PATCH] Proposed Fix
Comment 8 Joseph Pecoraro 2016-07-01 17:36:58 PDT
Created attachment 282614 [details]
[PATCH] Proposed Fix

Comment 9 Joseph Pecoraro 2016-07-01 17:41:07 PDT
Created attachment 282616 [details]
[PATCH] Proposed Fix
Comment 10 WebKit Commit Bot 2016-07-02 20:33:45 PDT
Comment on attachment 282616 [details]
[PATCH] Proposed Fix

Rejecting attachment 282616 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-01', 'apply-attachment', '--no-update', '--non-interactive', 282616, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Last 500 characters of output:
patching file LayoutTests/platform/mac-wk2/TestExpectations
Hunk #1 succeeded at 294 (offset -2 lines).
patching file LayoutTests/platform/mac/TestExpectations
Hunk #1 FAILED at 737.
Hunk #2 succeeded at 1234 (offset -1 lines).
1 out of 2 hunks FAILED -- saving rejects to file LayoutTests/platform/mac/TestExpectations.rej

Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force', '--reviewer', u'Timothy Hatcher']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Full output: http://webkit-queues.webkit.org/results/1616548
Comment 11 Joseph Pecoraro 2016-07-05 11:37:16 PDT