Bug 46439 - Unskip remaining canvas layout tests so that they can track progressions/regressions
Summary: Unskip remaining canvas layout tests so that they can track progressions/regr...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Canvas (show other bugs)
Version: 528+ (Nightly build)
Hardware: PC OS X 10.5
: P2 Normal
Assignee: Matthew Delaney
URL:
Keywords:
: 46830 (view as bug list)
Depends on:
Blocks:
 
Reported: 2010-09-23 17:43 PDT by Matthew Delaney
Modified: 2010-10-28 13:30 PDT (History)
9 users (show)

See Also:


Attachments
Patch (55.29 KB, patch)
2010-09-23 17:47 PDT, Matthew Delaney
no flags Details | Formatted Diff | Diff
Unskip canvas layout tests / update platform expectations (62.16 KB, patch)
2010-09-30 16:44 PDT, Matthew Delaney
no flags Details | Formatted Diff | Diff
Unskip canvas layout tests / update platform expectations (64.77 KB, patch)
2010-09-30 17:18 PDT, Matthew Delaney
no flags Details | Formatted Diff | Diff
Patch (52.88 KB, patch)
2010-10-28 12:49 PDT, Matthew Delaney
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Matthew Delaney 2010-09-23 17:43:28 PDT
There are a handful of *mostly* failing canvas layout tests that are still skipped. Some are now passing. These all should be unskipped and their expectations reset for them to be currently useful.
Comment 1 Matthew Delaney 2010-09-23 17:47:20 PDT
Created attachment 68632 [details]
Patch
Comment 2 Csaba Osztrogonác 2010-09-23 23:37:39 PDT
Are there opened bugs for fixing these failing?
Comment 3 Matthew Delaney 2010-09-24 11:48:57 PDT
Yes and no. There is one master bug opened to "pass them all" and then at least a few tracking individual failing tests. We should definitely open bugs for each though.
Comment 4 Adele Peterson 2010-09-24 13:20:57 PDT
Comment on attachment 68632 [details]
Patch

I agree this makes sense to unskip these.  Please file individual bugs for the tests that have failing expected results, so we don't lose track of them.
Comment 5 Matthew Delaney 2010-09-27 10:49:18 PDT
I created this master bug (https://bugs.webkit.org/show_bug.cgi?id=46506) that will depend on all of the individual bugs filed per test. In the process of filing all those.
Comment 6 WebKit Commit Bot 2010-09-28 15:10:29 PDT
Comment on attachment 68632 [details]
Patch

Clearing flags on attachment: 68632

Committed r68568: <http://trac.webkit.org/changeset/68568>
Comment 7 WebKit Commit Bot 2010-09-28 15:10:35 PDT
All reviewed patches have been landed.  Closing bug.
Comment 8 WebKit Review Bot 2010-09-28 15:28:40 PDT
http://trac.webkit.org/changeset/68568 might have broken Qt Linux Release
The following changes are on the blame list:
http://trac.webkit.org/changeset/68568
http://trac.webkit.org/changeset/68566
http://trac.webkit.org/changeset/68567
Comment 9 Csaba Osztrogonác 2010-09-28 23:08:07 PDT
(In reply to comment #8)
> http://trac.webkit.org/changeset/68568 might have broken Qt Linux Release
> The following changes are on the blame list:
> http://trac.webkit.org/changeset/68568
> http://trac.webkit.org/changeset/68566
> http://trac.webkit.org/changeset/68567

Qt buildbot don't like this patch, because the following tests pass:

canvas/philip/tests/2d.gradient.radial.cone.front.html
canvas/philip/tests/2d.gradient.radial.inside2.html
canvas/philip/tests/2d.gradient.radial.inside3.html
canvas/philip/tests/2d.gradient.radial.outside1.html
canvas/philip/tests/2d.line.cap.closed.html
canvas/philip/tests/2d.line.join.parallel.html
canvas/philip/tests/2d.shadow.enable.blur.html
canvas/philip/tests/2d.shadow.enable.x.html
canvas/philip/tests/2d.shadow.enable.y.html
canvas/philip/tests/2d.shadow.gradient.alpha.html
canvas/philip/tests/2d.shadow.gradient.basic.html
canvas/philip/tests/2d.shadow.gradient.transparent.2.html
canvas/philip/tests/2d.strokeRect.zero.4.html
canvas/philip/tests/toDataURL.jpeg.alpha.html
Comment 10 Csaba Osztrogonác 2010-09-28 23:56:40 PDT
Qt specific expected files landed for passing tests in http://trac.webkit.org/changeset/68633
Comment 11 Csaba Osztrogonác 2010-09-28 23:59:37 PDT
(In reply to comment #8)
> http://trac.webkit.org/changeset/68568 might have broken Qt Linux Release
> The following changes are on the blame list:
> http://trac.webkit.org/changeset/68568
> http://trac.webkit.org/changeset/68566
> http://trac.webkit.org/changeset/68567

Dear Matthew and Adele, please don't ignore this kind of notices.
We usually fix what we break and don't leave the tree burning.
Comment 12 Eric Seidel (no email) 2010-09-29 19:05:58 PDT
It looks like this broke canvas/philip/tests/toDataURL.jpeg.alpha.html on Snow Leopard.
Comment 13 Eric Seidel (no email) 2010-09-29 19:07:09 PDT
This was a very Mac-centric patch.  Why not check in mac-specific results for these?  I expect a number of these tests are related to canvas on CG systems.
Comment 14 Eric Seidel (no email) 2010-09-29 19:11:44 PDT
We could just roll this out, since the landing was actually done by the commit-queue.

I think the failure in this patch was not considering all platforms.  Canvas is sadly rather graphics-library specific.
Comment 15 Eric Seidel (no email) 2010-09-29 19:16:56 PDT
canvas/philip/tests/2d.imageData.get.source.outside.html is failing on Leopard too.
Comment 16 Eric Seidel (no email) 2010-09-29 19:18:09 PDT
If the commit-bot landed this, then all the tests must have passed for it on either leopard or snow leopard (we have nodes of both OSes and its' random which one you get).  So one of these failures must be flaky.
Comment 17 Eric Seidel (no email) 2010-09-29 19:20:18 PDT
It was landed by a Snow Leopard bot:
https://webkit-commit-queue.appspot.com/results/4210003
(you can tell by the python version).

https://webkit-commit-queue.appspot.com/patch/68632

also shows the the tests flaked once before the land completed correctly.  So it seems this added a flaky test too.

I'm going to roll this out, especially since I feel partially responsible as the cq-admin.  We can roll this back in when someone is around to deal with the tree fires.
Comment 18 Eric Seidel (no email) 2010-09-29 19:24:27 PDT
Reverted r68568 for reason:

Failing tests on both Leopard and Snow Leopard

Committed r68737: <http://trac.webkit.org/changeset/68737>
Comment 19 WebKit Review Bot 2010-09-29 20:14:37 PDT
http://trac.webkit.org/changeset/68737 might have broken SnowLeopard Intel Release (Tests)
Comment 20 Matthew Delaney 2010-09-30 14:20:26 PDT
Currently preparing a patch to add the expected results to the proper locations and unskip the appropriate tests - so as to not mess up the other ports. Sorry about that!
Comment 21 Matthew Delaney 2010-09-30 14:44:32 PDT
*** Bug 46830 has been marked as a duplicate of this bug. ***
Comment 22 Matthew Delaney 2010-09-30 16:44:33 PDT
Created attachment 69395 [details]
Unskip canvas layout tests / update platform expectations

Unskips all but two of the remaining philip canvas layout tests and adds in the mac platform specific results.
Comment 23 Matthew Delaney 2010-09-30 17:18:56 PDT
Created attachment 69400 [details]
Unskip canvas layout tests / update platform expectations

Unskips all but two of the remaining philip canvas layout tests and adds in the mac platform specific results.
Comment 24 Geoffrey Garen 2010-10-14 15:13:02 PDT
Comment on attachment 69400 [details]
Unskip canvas layout tests / update platform expectations

Let's give the commit queue another go at this.
Comment 25 WebKit Commit Bot 2010-10-14 15:14:55 PDT
Comment on attachment 69400 [details]
Unskip canvas layout tests / update platform expectations

Rejecting patch 69400 from commit-queue.

Failed to run "['./WebKitTools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=abarth-cq-sl', 'apply-attachment', '--force-clean', '--non-interactive', '--quiet', 69400]" exit_code: 2
Cleaning working directory
Updating working directory
Logging in as commit-queue@webkit.org...
Fetching: https://bugs.webkit.org/attachment.cgi?id=69400&action=edit
Fetching: https://bugs.webkit.org/show_bug.cgi?id=46439&ctype=xml
Processing 1 patch from 1 bug.
Processing patch 69400 from bug 46439.
Failed to run "[u'/Users/abarth/git/webkit-queue/WebKitTools/Scripts/svn-apply', u'--reviewer', u'Geoffrey Garen', u'--force']" exit_code: 1

Full output: http://queues.webkit.org/results/4439036
Comment 26 Eric Seidel (no email) 2010-10-14 16:02:09 PDT
The patch doesn't apply.  Sorry the output is kinda lame.
Comment 27 Matthew Delaney 2010-10-28 12:49:33 PDT
Created attachment 72221 [details]
Patch
Comment 28 WebKit Commit Bot 2010-10-28 13:30:17 PDT
Comment on attachment 72221 [details]
Patch

Clearing flags on attachment: 72221

Committed r70803: <http://trac.webkit.org/changeset/70803>
Comment 29 WebKit Commit Bot 2010-10-28 13:30:26 PDT
All reviewed patches have been landed.  Closing bug.