RESOLVED DUPLICATE of bug 104597 104435
TestExpectations knows tests fail but NRWT reports a regression 
https://bugs.webkit.org/show_bug.cgi?id=104435
Summary TestExpectations knows tests fail but NRWT reports a regression
noel gordon
Reported 2012-12-08 00:33:44 PST
http://build.webkit.org/builders/Chromium%20Win%20Release%20%28Tests%29/builds/31815/steps/layout-test/logs/stdio Regressions: Unexpected audio failures (4) webaudio/audiobuffersource-loop-points.html [ Failure ] webaudio/codec-tests/vorbis/vbr-128kbps-44khz.html [ Failure ] webaudio/codec-tests/vorbis/vbr-70kbps-44khz.html [ Failure ] webaudio/codec-tests/vorbis/vbr-128kbps-44khz.html [ Failure ] However, chromium TestExpectations contains: % ack "buffersource-loop" LayoutTests/platform/chromium/TestExpectations webkit.org/b/100647 [ Win ] webaudio/audiobuffersource-loop-points.html [ Failure ] % ack "vorbis/vbr" LayoutTests/platform/chromium/TestExpectations Bug(dpranke) [ Win ] webaudio/codec-tests/vorbis/vbr-70kbps-44khz.html [ Failure ] Bug(dpranke) [ Win ] webaudio/codec-tests/vorbis/vbr-96kbps-44khz.html [ Failure ] Bug(dpranke) [ Win ] webaudio/codec-tests/vorbis/vbr-128kbps-44khz.html [ Failure ] Seems these are expected failures, not regressions?
Attachments
Add attachment proposed patch, testcase, etc.
Dirk Pranke
Comment 1 2012-12-08 12:09:36 PST
Ah, yeah, looks like I broke this case (mapping AUDIO onto FAIL) in my recent refactoring.
noel gordon
Comment 2 2012-12-09 16:27:49 PST
Currently on chrome linux. http://build.webkit.org/builders/Chromium%20Linux%20Release%20%28Tests%29/builds/42173/steps/layout-test/logs/stdio Regressions: Unexpected image and text failures (5) fast/text/hyphenate-character.html [ Failure ] fast/text/hyphenate-first-word.html [ Failure ] fast/text/hyphenate-limit-lines.html [ Failure ] fast/text/hyphenate-locale.html [ Failure ] fast/text/hyphens.html [ Failure ] % ack "text/hyphen" LayoutTests/platform/chromium/TestExpectations crbug.com/47083 fast/text/hyphens.html [ Failure ImageOnlyFailure ] crbug.com/47083 fast/text/hyphenate-character.html [ Failure ImageOnlyFailure ] crbug.com/47083 fast/text/hyphenate-first-word.html [ Failure ImageOnlyFailure ] crbug.com/47083 fast/text/hyphenate-limit-lines.html [ Failure ImageOnlyFailure ] crbug.com/47083 fast/text/hyphenate-limit-before-after.html [ Failure ImageOnlyFailure ] crbug.com/47083 fast/text/hyphenate-locale.html [ Failure ImageOnlyFailure ] Not just mapping AUDIO into FAIL it seems.
Dirk Pranke
Comment 3 2012-12-10 11:26:20 PST
Hm. That's puzzling ... if mapping IMAGE+TEXT onto Failure was broken, I'd expect a lot more than 5 failures. Looking further ...
Dirk Pranke
Comment 4 2012-12-10 14:17:59 PST
(In reply to comment #3) > Hm. That's puzzling ... if mapping IMAGE+TEXT onto Failure was broken, I'd expect a lot more than 5 failures. Looking further ... I can't reproduce this now ... I think maybe you happened to see a failure on the bots after r137009 but hadn't updated your local checkout yet, and tkent added the suppressions back in r137144.
Stephen White
Comment 5 2012-12-10 14:22:39 PST
(In reply to comment #4) > (In reply to comment #3) > > Hm. That's puzzling ... if mapping IMAGE+TEXT onto Failure was broken, I'd expect a lot more than 5 failures. Looking further ... > > I can't reproduce this now ... I think maybe you happened to see a failure on the bots after r137009 but hadn't updated your local checkout yet, and tkent added the suppressions back in r137144. What's weird is, AFAICT they're only failing on the build.webkit.org bots. They're passing on the canaries. Might need a Chromium DEPS roll into WebKit; I'm not sure.
Dirk Pranke
Comment 6 2012-12-10 14:39:28 PST
bug fix posted in bug 104597 (I forgot that this was a dedicated bug for this issue). I'm going to close this as a dup of 104597 for now, but if we find other problems with other results showing up incorrectly, we can re-open this to reuse it. *** This bug has been marked as a duplicate of bug 104597 ***
Note You need to log in before you can comment on or make changes to this bug.