Bug 137615 - REGRESSION(r174535): Updated expectations for the layout tests have wrong color (or wrong message)
Summary: REGRESSION(r174535): Updated expectations for the layout tests have wrong col...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2014-10-10 10:49 PDT by Carlos Alberto Lopez Perez
Modified: 2014-10-14 10:24 PDT (History)
2 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Carlos Alberto Lopez Perez 2014-10-10 10:49:15 PDT
r174535 <http://trac.webkit.org/r174535> has changed the background color of the expectations to red. But the message stills keeps beeing "This anchor should have a green background"

This are the changes on the expectations from r174535: http://sprunge.us/JQPG?diff

As you can see, the color of the anchor is changed to red (bgcolor=#FF0000), but the message inside still says "This anchor should have a green background"

One of the two (the color or the message) is wrong, isn't it?

The affected tests are:

Regressions: Unexpected text-only failures (8)
  css3/selectors3/html/css3-modsel-61.html [ Failure ]
  css3/selectors3/html/css3-modsel-83.html [ Failure ]
  css3/selectors3/xhtml/css3-modsel-61.xml [ Failure ]
  css3/selectors3/xhtml/css3-modsel-83.xml [ Failure ]
  css3/selectors3/xml/css3-modsel-61.xml [ Failure ]
  css3/selectors3/xml/css3-modsel-83.xml [ Failure ]
  fast/selectors/061.html [ Failure ]
  fast/selectors/083.html [ Failure ]
Comment 1 Benjamin Poulain 2014-10-10 11:11:36 PDT
It is intended. The commit r174535 intentionally breaks all support of :not(:link) and :not(:visited).

Eventually we should fix the regression, but in the short term, the benefits outweigh the risks.

Help is welcome if you have time :)
Comment 2 Carlos Alberto Lopez Perez 2014-10-10 11:15:23 PDT
(In reply to comment #1)
> It is intended. The commit r174535 intentionally breaks all support of :not(:link) and :not(:visited).
> 
> Eventually we should fix the regression, but in the short term, the benefits outweigh the risks.
> 
> Help is welcome if you have time :)

Shouldn't then be a better idea to mark the test as failing?

Or at least update the text saying something like:

"This anchor should have a red background until bug #XXXX is fixed. Once that is done, this anchor should have a green background."

???

Otherwise is very confusing and misleading.
Comment 3 Benjamin Poulain 2014-10-10 11:29:42 PDT
(In reply to comment #2)
> (In reply to comment #1)
> > It is intended. The commit r174535 intentionally breaks all support of :not(:link) and :not(:visited).
> > 
> > Eventually we should fix the regression, but in the short term, the benefits outweigh the risks.
> > 
> > Help is welcome if you have time :)
> 
> Shouldn't then be a better idea to mark the test as failing?
> 
> Or at least update the text saying something like:
> 
> "This anchor should have a red background until bug #XXXX is fixed. Once that is done, this anchor should have a green background."
> 
> ???
> 
> Otherwise is very confusing and misleading.

Checking in failing test is a common practice. The reason is those test can still catch unintended changes. If we were to mark them as failing, an intended change would be ignored by the bots.
Comment 4 Carlos Alberto Lopez Perez 2014-10-10 11:33:29 PDT
(In reply to comment #3)
> (In reply to comment #2)
> > (In reply to comment #1)
> > > It is intended. The commit r174535 intentionally breaks all support of :not(:link) and :not(:visited).
> > > 
> > > Eventually we should fix the regression, but in the short term, the benefits outweigh the risks.
> > > 
> > > Help is welcome if you have time :)
> > 
> > Shouldn't then be a better idea to mark the test as failing?
> > 
> > Or at least update the text saying something like:
> > 
> > "This anchor should have a red background until bug #XXXX is fixed. Once that is done, this anchor should have a green background."
> > 
> > ???
> > 
> > Otherwise is very confusing and misleading.
> 
> Checking in failing test is a common practice. The reason is those test can still catch unintended changes. If we were to mark them as failing, an intended change would be ignored by the bots.

And what about to updating the text of the anchor to say that it is red now because of a tracked bug but that it should be green after that bug is fixed?
Comment 5 Benjamin Poulain 2014-10-10 12:49:25 PDT
(In reply to comment #4)
> > Checking in failing test is a common practice. The reason is those test can still catch unintended changes. If we were to mark them as failing, an intended change would be ignored by the bots.
> 
> And what about to updating the text of the anchor to say that it is red now because of a tracked bug but that it should be green after that bug is fixed?

The tests  your listed are standards tests. I don't think we should change them.
Comment 6 Carlos Alberto Lopez Perez 2014-10-14 10:24:51 PDT
(In reply to comment #5)
> (In reply to comment #4)
> > > Checking in failing test is a common practice. The reason is those test can still catch unintended changes. If we were to mark them as failing, an intended change would be ignored by the bots.
> > 
> > And what about to updating the text of the anchor to say that it is red now because of a tracked bug but that it should be green after that bug is fixed?
> 
> The tests  your listed are standards tests. I don't think we should change them.

Ok.

I have rebaselined the expectations for the GTK port for this tests on r174685 <http://trac.webkit.org/r174685> as also removed them from the expected failures for GTK.

Closing