The fix for Bug 149319 (r198963) included a broken test that only called notifyDone(), but never called waitUntilDone(): Bug 149319: Backdrop Filter should repaint when changed via script <https://bugs.webkit.org/show_bug.cgi?id=149319> <http://trac.webkit.org/changeset/198963> It was fixed in r198971: <http://trac.webkit.org/changeset/198971> However, this got me to thinking about whether we could detect such a bad test before it lands by being stricter (meaning crash DumpRenderTree or WebKitTestRunner with an RELEASE_ASSERT) about two conditions: 1. Calling notifyDone() before calling waitUntilDone(). 2. Calling waitUntilDone() after calling notifyDone().
Created attachment 275467 [details] Patch v1 (proof-of-concept; not for landing)
(In reply to comment #1) > Created attachment 275467 [details] > Patch v1 (proof-of-concept; not for landing) If there are any DumpRenderTree or WebKitTestRunner crashes with this patch, that means a layout test exists that calls waitUntilDone()/notifyDone() in the wrong order.
Created attachment 275478 [details] Patch v2 (proof-of-concept; not for landing)
Comment on attachment 275478 [details] Patch v2 (proof-of-concept; not for landing) Attachment 275478 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.webkit.org/results/1088391 Number of test failures exceeded the failure limit.
Created attachment 275481 [details] Archive of layout-test-results from ews104 for mac-yosemite-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews104 Port: mac-yosemite-wk2 Platform: Mac OS X 10.10.5
Okay, this proof-of-concept indicates there might be some work to do here, but I don't have time to spend on it immediately.
<rdar://problem/25511581>
Comment on attachment 275478 [details] Patch v2 (proof-of-concept; not for landing) Attachment 275478 [details] did not pass ios-sim-ews (ios-simulator-wk2): Output: http://webkit-queues.webkit.org/results/1088437 Number of test failures exceeded the failure limit.
Created attachment 275482 [details] Archive of layout-test-results from ews125 for ios-simulator-wk2 The attached test failures were seen while running run-webkit-tests on the ios-sim-ews. Bot: ews125 Port: ios-simulator-wk2 Platform: Mac OS X 10.10.5
Comment on attachment 275478 [details] Patch v2 (proof-of-concept; not for landing) Attachment 275478 [details] did not pass mac-debug-ews (mac): Output: http://webkit-queues.webkit.org/results/1088435 Number of test failures exceeded the failure limit.
Created attachment 275483 [details] Archive of layout-test-results from ews115 for mac-yosemite The attached test failures were seen while running run-webkit-tests on the mac-debug-ews. Bot: ews115 Port: mac-yosemite Platform: Mac OS X 10.10.5
Comment on attachment 275478 [details] Patch v2 (proof-of-concept; not for landing) Attachment 275478 [details] did not pass mac-ews (mac): Output: http://webkit-queues.webkit.org/results/1088818 Number of test failures exceeded the failure limit.
Created attachment 275486 [details] Archive of layout-test-results from ews103 for mac-yosemite The attached test failures were seen while running run-webkit-tests on the mac-ews. Bot: ews103 Port: mac-yosemite Platform: Mac OS X 10.10.5