Bug 50233 – Remove unnecessary pixel results, use platform-independent text results, re Changeset 72802

• Status: RESOLVED
• Resolution: FIXED
• Component: New Bugs

[W. James MacLean]

Remove unnecessary pixel results, use platform-independent text results, re Changeset 72802

[W. James MacLean]

Created attachment 75137 [details]
Patch

[W. James MacLean]

Marking as "Platform All"

[Ojan Vafai]

Comment on attachment 75137 [details]
Patch

Thanks for doing this. In order to avoid needing pixel results, you need to explicitly mark each test as dumpAsText (see http://webkit.org/quality/testwriting.html). Examples: http://codesearch.google.com/codesearch?as_q=dumpAsText&vert=chromium&as_lang=javascript

Also, please delete the chromium-mac and chromium-win pixel results for these tests as well.

[W. James MacLean]

Created attachment 75153 [details]
Patch

[Ojan Vafai]

Comment on attachment 75153 [details]
Patch

Please delete the platform/mac results too.

[W. James MacLean]

Created attachment 75160 [details]
Patch

[Ojan Vafai]

Comment on attachment 75160 [details]
Patch

Yay! Thanks again for following up.

[W. James MacLean]

(In reply to comment #7)
> (From update of attachment 75160 [details])
> Yay! Thanks again for following up.

My pleasure! :-)

[WebKit Commit Bot]

Comment on attachment 75160 [details]
Patch

Rejecting patch 75160 from commit-queue.

Failed to run "['./WebKitTools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=eseidel-cq-sl', 'build-and-test', '--no-clean', '--no-update', '--test', '--non-interactive']" exit_code: 2
Last 500 characters of output:
s/websocket/tests/workers ......
http/tests/workers .....
http/tests/xhtmlmp .
http/tests/xmlhttprequest ............................................................................................................................................................................
http/tests/xmlhttprequest/web-apps ...............
http/tests/xmlhttprequest/workers .........
582.95s total testing time

22077 test cases (99%) succeeded
5 test cases (<1%) were new
12 test cases (<1%) had stderr output

Full output: http://queues.webkit.org/results/6610002

[Csaba Osztrogonác]

Comment on attachment 75160 [details]
Patch

try again

[WebKit Commit Bot]

The commit-queue encountered the following flaky tests while processing attachment 75160 [details]:

compositing/iframes/overlapped-nested-iframes.html

Please file bugs against the tests.  These tests were authored by [e-mail].  The commit-queue is continuing to process your patch.

[WebKit Commit Bot]

The commit-queue encountered the following flaky tests while processing attachment 75160 [details]:

media/event-attributes.html

Please file bugs against the tests.  These tests were authored by [e-mail].  The commit-queue is continuing to process your patch.

[WebKit Commit Bot]

The commit-queue encountered the following flaky tests while processing attachment 75160 [details]:

fast/workers/storage/use-same-database-in-page-and-workers.html

Please file bugs against the tests.  These tests were authored by [e-mail].  The commit-queue is continuing to process your patch.

[WebKit Commit Bot]

Comment on attachment 75160 [details]
Patch

Rejecting patch 75160 from commit-queue.

Failed to run "['./WebKitTools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=abarth-cq-sl', 'build-and-test', '--no-clean', '--no-update', '--test', '--non-interactive']" exit_code: 2
Last 500 characters of output:
s/websocket/tests/workers ......
http/tests/workers .....
http/tests/xhtmlmp .
http/tests/xmlhttprequest ............................................................................................................................................................................
http/tests/xmlhttprequest/web-apps ...............
http/tests/xmlhttprequest/workers .........
538.85s total testing time

22083 test cases (99%) succeeded
5 test cases (<1%) were new
12 test cases (<1%) had stderr output

Full output: http://queues.webkit.org/results/6774004

[Ojan Vafai]

I think the reason this is failing the commit queue is that the tests are showing up as "new". I don't know exactly what that means, but I think it means something is not right. James, do you not seeing it claiming these tests are "new" when you run this locally?

[W. James MacLean]

(In reply to comment #15)
> I think the reason this is failing the commit queue is that the tests are showing up as "new". I don't know exactly what that means, but I think it means something is not right. James, do you not seeing it claiming these tests are "new" when you run this locally?

I don't recall seeing this. When I created the tests, the first thing I did was generate baselines, so after that there was no mention of 'new'. Perhaps it thinks the expectation files are missing?

[Eric Seidel (no email)]

Yes.  5 "new" tests, means 5 tests missing expectations.  Are you sure you have Mac or Mac Snow Leopard expectations?

[W. James MacLean]

(In reply to comment #17)
> Yes.  5 "new" tests, means 5 tests missing expectations.  Are you sure you have Mac or Mac Snow Leopard expectations?

The point of this patch was to remove unnecessary pixel expectations and use a single set of -expected.txt files. All the tests use dumpAsText() to facilitate this, and none of them produce anything other than a blank screen.

So this patch deletes all the png and checksum files checked in earlier and uses a single set of text expectations in LayoutTests/svg/custom/.

Do we have to have platform-specific expectations for Mac? In my testing on linux and snow leopard the test harness seemed happy with this setup.

[Ojan Vafai]

I looked carefully at the patch. Everything looks fine. I wonder if webkit-patch is not handling the file renames correctly?

[WebKit Commit Bot]

Comment on attachment 75160 [details]
Patch

Rejecting patch 75160 from commit-queue.

Failed to run "['./WebKitTools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=abarth-cq-sl', 'build-and-test', '--no-clean', '--no-update', '--test', '--non-interactive']" exit_code: 2
Last 500 characters of output:
s/websocket/tests/workers ......
http/tests/workers .....
http/tests/xhtmlmp .
http/tests/xmlhttprequest ............................................................................................................................................................................
http/tests/xmlhttprequest/web-apps ...............
http/tests/xmlhttprequest/workers .........
538.18s total testing time

22084 test cases (99%) succeeded
5 test cases (<1%) were new
12 test cases (<1%) had stderr output

Full output: http://queues.webkit.org/results/6836007

[Ojan Vafai]

I tried patching this locally (using webkit-patch). The expected results don't come in. Looking at the raw diff of this patch, I don't see the rename anywhere in the patch, just in the changelog. :( How did you do the rename? Are you on git or svn?

[W. James MacLean]

(In reply to comment #21)
> I tried patching this locally (using webkit-patch). The expected results don't come in. Looking at the raw diff of this patch, I don't see the rename anywhere in the patch, just in the changelog. :( How did you do the rename? Are you on git or svn?

I 'renamed' using 'mv' followed by 'git add'. My git repo certainly sees the files as moved ...

[Eric Seidel (no email)]

You hit bug 49154. :(

[W. James MacLean]

(In reply to comment #23)
> You hit bug 49154. :(

Ahhh ....

What's the best way to get around this, given that we can't change the content of the moved file. Two patches, one to delete and one to restore? Applying the one that deletes will be difficult, as the tests will have no expected output ...

[Tony Chang]

(In reply to comment #24)

> What's the best way to get around this

Use svn.

[W. James MacLean]

(In reply to comment #25)
> (In reply to comment #24)
> 
> > What's the best way to get around this
> 
> Use svn.

I'd rather not have to build an SVN repo just to submit this patch. Surely there's another way.

[Eric Seidel (no email)]

I suspect there is a way to get git patches to reflect renames  Maybe git format-patch --binary?  donno.

[W. James MacLean]

(In reply to comment #27)
> I suspect there is a way to get git patches to reflect renames  Maybe git format-patch --binary?  donno.

I don't think so ... webkit-patch must already be using --binary as the png images in the patch appear OK.

[W. James MacLean]

Created attachment 75697 [details]
Patch

[W. James MacLean]

Comment on attachment 75697 [details]
Patch

I've split the patch into two parts to get around the git issue. This part deletes all pixel tests and adds the platform-independent text expectations, the next part of the patch will delete the platform-specific expectations.

[Ojan Vafai]

Comment on attachment 75697 [details]
Patch

Doing it this way is fine, but I think maybe you left the html files out of this patch by accident?

[W. James MacLean]

Created attachment 75899 [details]
Patch

[W. James MacLean]

Comment on attachment 75899 [details]
Patch

Sorry about forgetting the actual test files :-)

This patch should be good to go.

[WebKit Commit Bot]

Comment on attachment 75899 [details]
Patch

Rejecting patch 75899 from commit-queue.

Failed to run "['./WebKitTools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=eseidel-cq-sl', 'build', '--no-clean', '--no-update', '--build-style=both']" exit_code: 2
Last 500 characters of output:
86_64 objective-c++ com.apple.compilers.gcc.4_2
	CompileC /Projects/CommitQueue/WebKitBuild/WebCore.build/Debug/WebCore.build/Objects-normal/x86_64/ScriptExecutionContext.o /Projects/CommitQueue/WebCore/dom/ScriptExecutionContext.cpp normal x86_64 c++ com.apple.compilers.gcc.4_2
	CompileC /Projects/CommitQueue/WebKitBuild/WebCore.build/Debug/WebCore.build/Objects-normal/x86_64/Settings.o /Projects/CommitQueue/WebCore/page/Settings.cpp normal x86_64 c++ com.apple.compilers.gcc.4_2
(28 failures)


Full output: http://queues.webkit.org/results/6717110

[W. James MacLean]

(In reply to comment #34)
> (From update of attachment 75899 [details])
> Rejecting patch 75899 from commit-queue.
> 
> Failed to run "['./WebKitTools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=eseidel-cq-sl', 'build', '--no-clean', '--no-update', '--build-style=both']" exit_code: 2
> Last 500 characters of output:
> 86_64 objective-c++ com.apple.compilers.gcc.4_2
>     CompileC /Projects/CommitQueue/WebKitBuild/WebCore.build/Debug/WebCore.build/Objects-normal/x86_64/ScriptExecutionContext.o /Projects/CommitQueue/WebCore/dom/ScriptExecutionContext.cpp normal x86_64 c++ com.apple.compilers.gcc.4_2
>     CompileC /Projects/CommitQueue/WebKitBuild/WebCore.build/Debug/WebCore.build/Objects-normal/x86_64/Settings.o /Projects/CommitQueue/WebCore/page/Settings.cpp normal x86_64 c++ com.apple.compilers.gcc.4_2
> (28 failures)
> 
> 
> Full output: http://queues.webkit.org/results/6717110

Apparently this failed to build, which I don't think is related to the patch. Can we try committing again?

[WebKit Commit Bot]

Comment on attachment 75899 [details]
Patch

Rejecting patch 75899 from commit-queue.

Failed to run "['./WebKitTools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=abarth-cq-sl', 'apply-attachment', '--force-clean', '--non-interactive', 75899]" exit_code: 2
Last 500 characters of output:
g OpenSource
From git://git.webkit.org/WebKit
   20980e9..32fd619  master     -> origin/master
	M	WebKitLibraries/libWebKitSystemInterfaceSnowLeopard.a
	M	WebKitLibraries/WebKitSystemInterface.h
	M	WebKitLibraries/ChangeLog
	M	WebKitLibraries/libWebKitSystemInterfaceLeopard.a
Incomplete data: Delta source ended unexpectedly at /usr/local/git/libexec/git-core/git-svn line 4618

Died at WebKitTools/Scripts/update-webkit line 132.

Failed to run "['WebKitTools/Scripts/update-webkit']" exit_code: 2

Full output: http://queues.webkit.org/results/6966008

[WebKit Commit Bot]

Comment on attachment 75899 [details]
Patch

Rejecting patch 75899 from commit-queue.

Failed to run "['./WebKitTools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=eseidel-cq-sl', 'build', '--no-clean', '--no-update', '--build-style=both']" exit_code: 2
Last 500 characters of output:
leC /Projects/CommitQueue/WebKitBuild/WebCore.build/Debug/WebCore.build/Objects-normal/x86_64/JSPositionError.o /Projects/CommitQueue/WebKitBuild/Debug/DerivedSources/WebCore/JSPositionError.cpp normal x86_64 c++ com.apple.compilers.gcc.4_2
	CompileC /Projects/CommitQueue/WebKitBuild/WebCore.build/Debug/WebCore.build/Objects-normal/x86_64/JSPluginElementFunctions.o /Projects/CommitQueue/WebCore/bindings/js/JSPluginElementFunctions.cpp normal x86_64 c++ com.apple.compilers.gcc.4_2
(41 failures)


Full output: http://queues.webkit.org/results/7008012

[W. James MacLean]

(In reply to comment #37)
> (From update of attachment 75899 [details])
> Rejecting patch 75899 from commit-queue.
> 
> Failed to run "['./WebKitTools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=eseidel-cq-sl', 'build', '--no-clean', '--no-update', '--build-style=both']" exit_code: 2
> Last 500 characters of output:
> leC /Projects/CommitQueue/WebKitBuild/WebCore.build/Debug/WebCore.build/Objects-normal/x86_64/JSPositionError.o /Projects/CommitQueue/WebKitBuild/Debug/DerivedSources/WebCore/JSPositionError.cpp normal x86_64 c++ com.apple.compilers.gcc.4_2
>     CompileC /Projects/CommitQueue/WebKitBuild/WebCore.build/Debug/WebCore.build/Objects-normal/x86_64/JSPluginElementFunctions.o /Projects/CommitQueue/WebCore/bindings/js/JSPluginElementFunctions.cpp normal x86_64 c++ com.apple.compilers.gcc.4_2
> (41 failures)
> 
> 
> Full output: http://queues.webkit.org/results/7008012

Again, the failure appears to have nothing to do with the content of the patch - please let me know if I'm missing something :-)

Can we try again and, if it fails, put it through manually?

[Eric Seidel (no email)]

Comment on attachment 75899 [details]
Patch

Sure.  Maybe this machine is just running too many bots.

[WebKit Commit Bot]

Comment on attachment 75899 [details]
Patch

Clearing flags on attachment: 75899

Committed r73824: <http://trac.webkit.org/changeset/73824>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[W. James MacLean]

Created attachment 76376 [details]
Patch

[W. James MacLean]

Comment on attachment 76376 [details]
Patch

This is the second part of the cleanup patch ... nearly done!

[Csaba Osztrogonác]

Reopen to make CQ land https://bugs.webkit.org/attachment.cgi?id=76376

[WebKit Review Bot]

Comment on attachment 76376 [details]
Patch

Clearing flags on attachment: 76376

Committed r73911: <http://trac.webkit.org/changeset/73911>

[WebKit Review Bot]

All reviewed patches have been landed.  Closing bug.