Bug 89368 - Mark failing tests as failing
Summary: Mark failing tests as failing
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Tony Payne
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2012-06-18 11:48 PDT by Tony Payne
Modified: 2012-06-18 13:50 PDT (History)
2 users (show)

See Also:


Attachments
Patch (1.69 KB, patch)
2012-06-18 11:49 PDT, Tony Payne
no flags Details | Formatted Diff | Diff
Patch (1.70 KB, patch)
2012-06-18 11:52 PDT, Tony Payne
tony: review-
tony: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Tony Payne 2012-06-18 11:48:40 PDT
Mark failing tests as failing
Comment 1 Tony Payne 2012-06-18 11:49:38 PDT
Created attachment 148144 [details]
Patch
Comment 2 Tony Payne 2012-06-18 11:52:05 PDT
Created attachment 148146 [details]
Patch
Comment 3 Adam Barth 2012-06-18 12:05:30 PDT
Tony might have some thoughts about this patch.
Comment 4 Tony Chang 2012-06-18 13:45:32 PDT
Comment on attachment 148146 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=148146&action=review

> LayoutTests/platform/chromium/TestExpectations:1448
> +BUGWK89367 LINUX : platform/chromium-linux/fast/text/international/complex-joining-using-gpos.html = IMAGE
> +BUGWK89367 LINUX : fast/text/international/thai-line-breaks.html = IMAGE

They pass on the waterfall, right?  IIRC, the EWS bots run a different version of Ubuntu than the waterfall bots, so these tests only fail on the EWS bots.  I think these are always going to fail until we switch to Precise in a year or so.
Comment 5 Tony Payne 2012-06-18 13:47:54 PDT
(In reply to comment #4)
> (From update of attachment 148146 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=148146&action=review
> 
> > LayoutTests/platform/chromium/TestExpectations:1448
> > +BUGWK89367 LINUX : platform/chromium-linux/fast/text/international/complex-joining-using-gpos.html = IMAGE
> > +BUGWK89367 LINUX : fast/text/international/thai-line-breaks.html = IMAGE
> 
> They pass on the waterfall, right?  IIRC, the EWS bots run a different version of Ubuntu than the waterfall bots, so these tests only fail on the EWS bots.  I think these are always going to fail until we switch to Precise in a year or so.

These failures really hurt EWS performance since they guarantee the tests will be run 3 times, and they make it impossible to use the CQ when there are red tests, even if the CL is fixing those red tests. I don't think this is a situation that's tenable for a year. It impacts developer productivity significantly.
Comment 6 Adam Barth 2012-06-18 13:49:40 PDT
That's not really accurate.  The EWS remembers the set of failing tests from run to run and tends to ignore these tests anyway.
Comment 7 Tony Payne 2012-06-18 13:50:34 PDT
(In reply to comment #6)
> That's not really accurate.  The EWS remembers the set of failing tests from run to run and tends to ignore these tests anyway.

That doesn't match my experience at all.