Bug 85941 - nrwt: --skip-failing-tests should also skip flaky tests
Summary: nrwt: --skip-failing-tests should also skip flaky tests
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Dirk Pranke
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-05-08 18:27 PDT by Dirk Pranke
Modified: 2012-05-09 12:59 PDT (History)
4 users (show)

See Also:

Attachments
Patch (3.16 KB, patch)
2012-05-08 18:28 PDT, Dirk Pranke 		no flags Details | Formatted Diff | Diff
write more tests (7.41 KB, patch)
2012-05-08 19:35 PDT, Dirk Pranke 		ojan: review+
webkit.review.bot: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ec2-cr-linux-02 (742.35 KB, application/zip)
2012-05-08 20:16 PDT, WebKit Review Bot 		no flags Details
Show Obsolete (1) View All Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this bug.
Description Dirk Pranke 2012-05-08 18:27:51 PDT 
nrwt: --skip-failing-tests should also skip flaky tests
Comment 1 Dirk Pranke 2012-05-08 18:28:22 PDT 
Created attachment 140838 [details]
Patch
Comment 2 Dirk Pranke 2012-05-08 18:33:25 PDT 
this should keep the chromium ews bots from running ~450 tests that are currently marked as flaky, and the only results you should see are from the tests that are expected to reliably PASS but actually fail.

Note that if you run a test and it fails, and then you retry and it passes, we should still see the failure, I think. I don't currently, and am investigating ...
Comment 3 Dirk Pranke 2012-05-08 19:35:08 PDT 
Created attachment 140855 [details]
write more tests
Comment 4 WebKit Review Bot 2012-05-08 20:16:19 PDT 
Comment on attachment 140855 [details]
write more tests

Attachment 140855 [details] did not pass chromium-ews (chromium-xvfb):
Output: http://queues.webkit.org/results/12656501

New failing tests:
http/tests/loading/post-in-iframe-with-back-navigation.html
Comment 5 WebKit Review Bot 2012-05-08 20:16:23 PDT 
Created attachment 140858 [details]
Archive of layout-test-results from ec2-cr-linux-02

The attached test failures were seen while running run-webkit-tests on the chromium-ews.
Bot: ec2-cr-linux-02  Port: <class 'webkitpy.common.config.ports.ChromiumXVFBPort'>  Platform: Linux-2.6.35-28-virtual-x86_64-with-Ubuntu-10.10-maverick
Comment 6 Dirk Pranke 2012-05-09 12:59:03 PDT 
Committed r116552: <http://trac.webkit.org/changeset/116552>