Bug 50471 - media/controls-without-preload.html failing on Windows
Summary: media/controls-without-preload.html failing on Windows
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: 528+ (Nightly build)
Hardware: All Windows 7
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar, LayoutTestFailure, PlatformOnly
Depends on:
Blocks:
 
Reported: 2010-12-03 12:21 PST by Jessie Berlin
Modified: 2011-02-25 12:47 PST (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Jessie Berlin 2010-12-03 12:21:32 PST
It has been failing on both Windows 7 and XP:
http://build.webkit.org/results/Windows%207%20Release%20(Tests)/r73278%20(7105)/media/controls-without-preload-actual.txt

The failures have been happening since the test results were committed:
http://trac.webkit.org/changeset/73278

Which were separate from the tests being added:
http://trac.webkit.org/changeset/73257
Comment 1 Jessie Berlin 2010-12-03 12:53:40 PST
Committed the windows-specific results in r73288
http://trac.webkit.org/changeset/73288
Comment 2 Philippe Normand 2010-12-03 16:48:15 PST
Can r73278 please be reverted? The test expectations are platform-specific.
Comment 3 Jessie Berlin 2011-02-25 11:43:03 PST
Still failing on XP. I think the XP failures started with r78690:

http://build.webkit.org/old-results/Windows%20XP%20Debug%20(Tests)/r78690%20(25314)/results.html
http://build.webkit.org/old-results/Windows%20XP%20Debug%20(Tests)/r78689%20(25313)/results.html

I will commit the expected failing results for XP soon.
Comment 4 Eric Carlson 2011-02-25 12:08:02 PST
(In reply to comment #3)
> Still failing on XP. I think the XP failures started with r78690:
> 
> http://build.webkit.org/old-results/Windows%20XP%20Debug%20(Tests)/r78690%20(25314)/results.html
> http://build.webkit.org/old-results/Windows%20XP%20Debug%20(Tests)/r78689%20(25313)/results.html
> 
> I will commit the expected failing results for XP soon.

I think it would be better to add the test to the skipped list and file a bug instead of committing failing result.s
Comment 5 Jessie Berlin 2011-02-25 12:38:55 PST
Hrm, well I comitted the failing results in http://trac.webkit.org/changeset/79716 before your comment.

They don't seem to be flakey, just consistently failing with those results. Would it really be more useful to remove these expected results and just add it to the skip lists?
Comment 6 Eric Carlson 2011-02-25 12:42:57 PST
(In reply to comment #5)
> Hrm, well I comitted the failing results in http://trac.webkit.org/changeset/79716 before your comment.
> 
> They don't seem to be flakey, just consistently failing with those results. Would it really be more useful to remove these expected results and just add it to the skip lists?

Yes, I do think it is better to have a failing file in the skipped list rather than using incorrect results.
Comment 7 Adam Roben (:aroben) 2011-02-25 12:47:48 PST
(In reply to comment #4)
> (In reply to comment #3)
> > Still failing on XP. I think the XP failures started with r78690:
> > 
> > http://build.webkit.org/old-results/Windows%20XP%20Debug%20(Tests)/r78690%20(25314)/results.html
> > http://build.webkit.org/old-results/Windows%20XP%20Debug%20(Tests)/r78689%20(25313)/results.html
> > 
> > I will commit the expected failing results for XP soon.
> 
> I think it would be better to add the test to the skipped list and file a bug instead of committing failing result.s

A bug has been filed: this one.

Committing expected failure results has been our policy for a while now. We do this when a test is failing in a consistent way and is not crashing or timing out. The advantage of committing expected failure results is that we will notice if someone causes a change in the test's behavior (including fixing it!).

A number of people say that it's better to have the test in the Skipped file because then there's a record that it's failing. However, this bug records that the test is failing, and people don't seem to look through the Skipped file to see what tests are failing much in practice.
Comment 8 Jessie Berlin 2011-02-25 12:47:55 PST
<rdar://problem/9055504>