Bug 156133

Summary:

DumpRenderTree, WebKitTestRunner should be more strict about waitUntilDone()/notifyDone()

Product:

WebKit

Reporter:

David Kilzer (:ddkilzer) <ddkilzer>

Component:

Tools / Tests

Assignee:

Nobody <webkit-unassigned>

Status:

NEW

Severity:

Normal

CC:

ap, buildbot, joepeck, lforschler, rniwa, ryanhaddad, simon.fraser, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

WebKit Nightly Build

Hardware:

Unspecified

OS:

Unspecified

Bug Depends on:

149319, 167598

Bug Blocks:

Attachments:

Description	Flags
Patch v1 (proof-of-concept; not for landing)	none
Patch v2 (proof-of-concept; not for landing)	buildbot: commit-queue-
Archive of layout-test-results from ews104 for mac-yosemite-wk2	none
Archive of layout-test-results from ews125 for ios-simulator-wk2	none
Archive of layout-test-results from ews115 for mac-yosemite	none
Archive of layout-test-results from ews103 for mac-yosemite	none

David Kilzer (:ddkilzer)

Reported 2016-04-02 07:53:54 PDT

The fix for Bug 149319 (r198963) included a broken test that only called notifyDone(), but never called waitUntilDone(): Bug 149319: Backdrop Filter should repaint when changed via script <https://bugs.webkit.org/show_bug.cgi?id=149319> <http://trac.webkit.org/changeset/198963> It was fixed in r198971: <http://trac.webkit.org/changeset/198971> However, this got me to thinking about whether we could detect such a bad test before it lands by being stricter (meaning crash DumpRenderTree or WebKitTestRunner with an RELEASE_ASSERT) about two conditions: 1. Calling notifyDone() before calling waitUntilDone(). 2. Calling waitUntilDone() after calling notifyDone().

Attachments
Patch v1 (proof-of-concept; not for landing) (6.22 KB, patch) 2016-04-02 08:22 PDT, David Kilzer (:ddkilzer)	no flags	Details Formatted Diff Diff
Patch v2 (proof-of-concept; not for landing) (5.97 KB, patch) 2016-04-02 13:10 PDT, David Kilzer (:ddkilzer)	buildbot: commit-queue-	Details Formatted Diff Diff
Archive of layout-test-results from ews104 for mac-yosemite-wk2 (798.89 KB, application/zip) 2016-04-02 13:36 PDT, Build Bot	no flags	Details
Archive of layout-test-results from ews125 for ios-simulator-wk2 (195.85 KB, application/zip) 2016-04-02 13:52 PDT, Build Bot	no flags	Details
Archive of layout-test-results from ews115 for mac-yosemite (541.26 KB, application/zip) 2016-04-02 13:52 PDT, Build Bot	no flags	Details
Archive of layout-test-results from ews103 for mac-yosemite (421.44 KB, application/zip) 2016-04-02 15:22 PDT, Build Bot	no flags	Details
Show Obsolete (1) View All Add attachment proposed patch, testcase, etc.

David Kilzer (:ddkilzer)

Comment 1 2016-04-02 08:22:30 PDT

Created attachment 275467 [details] Patch v1 (proof-of-concept; not for landing)

David Kilzer (:ddkilzer)

Comment 2 2016-04-02 08:24:13 PDT

(In reply to comment #1) > Created attachment 275467 [details] > Patch v1 (proof-of-concept; not for landing) If there are any DumpRenderTree or WebKitTestRunner crashes with this patch, that means a layout test exists that calls waitUntilDone()/notifyDone() in the wrong order.

David Kilzer (:ddkilzer)

Comment 3 2016-04-02 13:10:15 PDT

Created attachment 275478 [details] Patch v2 (proof-of-concept; not for landing)

Build Bot

Comment 4 2016-04-02 13:36:24 PDT

Comment on attachment 275478 [details] Patch v2 (proof-of-concept; not for landing) Attachment 275478 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.webkit.org/results/1088391 Number of test failures exceeded the failure limit.

Build Bot

Comment 5 2016-04-02 13:36:27 PDT

Created attachment 275481 [details] Archive of layout-test-results from ews104 for mac-yosemite-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews104 Port: mac-yosemite-wk2 Platform: Mac OS X 10.10.5

David Kilzer (:ddkilzer)

Comment 6 2016-04-02 13:49:26 PDT

Okay, this proof-of-concept indicates there might be some work to do here, but I don't have time to spend on it immediately.

Radar WebKit Bug Importer

Comment 7 2016-04-02 13:49:42 PDT

<rdar://problem/25511581>

Build Bot

Comment 8 2016-04-02 13:51:59 PDT

Comment on attachment 275478 [details] Patch v2 (proof-of-concept; not for landing) Attachment 275478 [details] did not pass ios-sim-ews (ios-simulator-wk2): Output: http://webkit-queues.webkit.org/results/1088437 Number of test failures exceeded the failure limit.

Build Bot

Comment 9 2016-04-02 13:52:03 PDT

Created attachment 275482 [details] Archive of layout-test-results from ews125 for ios-simulator-wk2 The attached test failures were seen while running run-webkit-tests on the ios-sim-ews. Bot: ews125 Port: ios-simulator-wk2 Platform: Mac OS X 10.10.5

Build Bot

Comment 10 2016-04-02 13:52:07 PDT

Comment on attachment 275478 [details] Patch v2 (proof-of-concept; not for landing) Attachment 275478 [details] did not pass mac-debug-ews (mac): Output: http://webkit-queues.webkit.org/results/1088435 Number of test failures exceeded the failure limit.

Build Bot

Comment 11 2016-04-02 13:52:10 PDT

Created attachment 275483 [details] Archive of layout-test-results from ews115 for mac-yosemite The attached test failures were seen while running run-webkit-tests on the mac-debug-ews. Bot: ews115 Port: mac-yosemite Platform: Mac OS X 10.10.5

Build Bot

Comment 12 2016-04-02 15:22:11 PDT

Comment on attachment 275478 [details] Patch v2 (proof-of-concept; not for landing) Attachment 275478 [details] did not pass mac-ews (mac): Output: http://webkit-queues.webkit.org/results/1088818 Number of test failures exceeded the failure limit.

Build Bot

Comment 13 2016-04-02 15:22:15 PDT

Created attachment 275486 [details] Archive of layout-test-results from ews103 for mac-yosemite The attached test failures were seen while running run-webkit-tests on the mac-ews. Bot: ews103 Port: mac-yosemite Platform: Mac OS X 10.10.5

Note You need to log in before you can comment on or make changes to this bug.