Bug 159272 – REGRESSION (r202529?) inspector/debugger/break-on-* tests are flaky

• Status: RESOLVED
• Resolution: FIXED
• Component: Web Inspector

[Ryan Haddad]

inspector/debugger/break-on-* tests are flaky

These tests are flaky and seem to fail together on mac-wk1 with a similar diff

inspector/debugger/break-on-exception-finally.html
inspector/debugger/break-on-exception-native.html
inspector/debugger/break-on-exception.html
inspector/debugger/break-on-uncaught-exception-finally.html
inspector/debugger/break-on-uncaught-exception-native.html
inspector/debugger/break-on-uncaught-exception.html

https://build.webkit.org/builders/Apple%20El%20Capitan%20Debug%20WK1%20(Tests)/builds/6287


--- /Volumes/Data/slave/elcapitan-debug-tests-wk1/build/layout-test-results/inspector/debugger/break-on-exception-finally-expected.txt
+++ /Volumes/Data/slave/elcapitan-debug-tests-wk1/build/layout-test-results/inspector/debugger/break-on-exception-finally-actual.txt
@@ -4,6 +4,5 @@
 CONSOLE MESSAGE: line 8: TestError
 Checking pause locations when pausing on all exceptions.
 
-PAUSE #1 AT: doThrow:8:22
 Uncaught exception in test page: TestError [break-on-exception-tests.js:8]

[Radar WebKit Bug Importer]

<rdar://problem/27087938>

[Ryan Haddad]

Seems to have started ~r202537, but I do not see an obviously related change.

[Alexey Proskuryakov]

The first failures that I see are in r202531, in a test run that covered revisions r202531-r202529.

While <http://trac.webkit.org/projects/webkit/changeset/202529> does not look like an obvious culprit, it has to be it with fairly high probability.

This happens in WK2 too.

[Joseph Pecoraro]

One possibility is merely adding a new inspector test causing other to break.

Also, these tests all have InspectorTest.reloadPage, which I suspect has always been the source of some instability given how inspector tests work.

I'd be fine with having these tests just be skipped while I investigate. Just running the inspector/debugger directory I encounter some sporadic issues that will require more involved investigation.

[Ryan Haddad]

Skipped tests to get bots back to green while Joe investigates http://trac.webkit.org/projects/webkit/changeset/202720

[Joseph Pecoraro]

I just completely rewrote all the break-on-* tests for a number of reasons:

    - easier to read and understand using new libraries
    - avoid reload, should make them individually faster
    - merge tests, instead of 1-file-per-test
      - eliminates a ton of dead and stale code in these tests

That said, I think an issue is that the bots might be running them slowly, so I'm going to mark them as [ Slow ] when first enabling them.

[Joseph Pecoraro]

Created attachment 282613 [details]
[PATCH] Proposed Fix

[Joseph Pecoraro]

Created attachment 282614 [details]
[PATCH] Proposed Fix

Rebaselined.

[Joseph Pecoraro]

Created attachment 282616 [details]
[PATCH] Proposed Fix

[WebKit Commit Bot]

Comment on attachment 282616 [details]
[PATCH] Proposed Fix

Rejecting attachment 282616 [details] from commit-queue.

Failed to run "['/Volumes/Data/EWS/WebKit/Tools/Scripts/webkit-patch', '--status-host=webkit-queues.webkit.org', '--bot-id=webkit-cq-01', 'apply-attachment', '--no-update', '--non-interactive', 282616, '--port=mac']" exit_code: 2 cwd: /Volumes/Data/EWS/WebKit

Last 500 characters of output:
t-exception.html
patching file LayoutTests/platform/mac-wk2/TestExpectations
Hunk #1 succeeded at 294 (offset -2 lines).
patching file LayoutTests/platform/mac/TestExpectations
Hunk #1 FAILED at 737.
Hunk #2 succeeded at 1234 (offset -1 lines).
1 out of 2 hunks FAILED -- saving rejects to file LayoutTests/platform/mac/TestExpectations.rej

Failed to run "[u'/Volumes/Data/EWS/WebKit/Tools/Scripts/svn-apply', '--force', '--reviewer', u'Timothy Hatcher']" exit_code: 1 cwd: /Volumes/Data/EWS/WebKit

Full output: http://webkit-queues.webkit.org/results/1616548

[Joseph Pecoraro]

<https://trac.webkit.org/changeset/202825>