Bug 51236 - Make java/argument-to-object-type.html use shouldBeEqualToString to make it easier to debug
Summary: Make java/argument-to-object-type.html use shouldBeEqualToString to make it e...
Status: RESOLVED INVALID
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Other OS X 10.5
: P2 Normal
Assignee: Eric Seidel (no email)
URL:
Keywords:
Depends on:
Blocks: 51102
  Show dependency treegraph
 
Reported: 2010-12-17 00:40 PST by Eric Seidel (no email)
Modified: 2010-12-17 12:57 PST (History)
5 users (show)

See Also:


Attachments
Patch (1.74 KB, patch)
2010-12-17 00:41 PST, Eric Seidel (no email)
abarth: review+
commit-queue: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Eric Seidel (no email) 2010-12-17 00:40:11 PST
Make java/argument-to-object-type.html use shouldBeEqualToString to make it easier to debug
Comment 1 Eric Seidel (no email) 2010-12-17 00:41:53 PST
Created attachment 76854 [details]
Patch
Comment 2 Eric Seidel (no email) 2010-12-17 00:42:34 PST
I've not actually run this (since I don't have a copy of webkit built), but the cq will build and run the tests for me.  This should make it easier to solve bug 51102.
Comment 3 WebKit Commit Bot 2010-12-17 02:56:51 PST
The commit-queue encountered the following flaky tests while processing attachment 76854 [details]:

fast/workers/storage/use-same-database-in-page-and-workers.html bug 50995 (author: dumi@chromium.org)
java/argument-to-object-type.html bug 51102 (author: ap@webkit.org)
The commit-queue is continuing to process your patch.
Comment 4 WebKit Commit Bot 2010-12-17 03:23:06 PST
Comment on attachment 76854 [details]
Patch

Rejecting attachment 76854 [details] from commit-queue.

Failed to run "['./WebKitTools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '--bot-id=abarth-cq-sl', 'build-and-test', '--no-clean', '--no-update', '--test', '--non-interactive']" exit_code: 2
Last 500 characters of output:
.........................................................................................................................................................................
inspector .................................................................................
java .
java/argument-to-object-type.html -> failed

Exiting early after 1 failures. 17897 tests run.
292.72s total testing time

17896 test cases (99%) succeeded
1 test case (<1%) had incorrect layout
8 test cases (<1%) had stderr output

Full output: http://queues.webkit.org/results/7290011
Comment 5 Eric Seidel (no email) 2010-12-17 03:52:37 PST
The failure can be seen here:
https://bugs.webkit.org/attachment.cgi?id=76861
I guess that's why we didn't shouldBeEqualToSTring.
Comment 6 Alexey Proskuryakov 2010-12-17 12:04:01 PST
Eric, has this been rolled out?
Comment 7 Eric Seidel (no email) 2010-12-17 12:30:10 PST
It was never rolled in. :)
Comment 8 Alexey Proskuryakov 2010-12-17 12:43:42 PST
Oh, ok. I was looking at <https://bug-51102-attachments.webkit.org/attachment.cgi?id=76861>, and didn't realize that it was from a failed attempt to commit this change.
Comment 9 Eric Seidel (no email) 2010-12-17 12:53:15 PST
Yeah.  It's slightly concerning that the queue thought that the failed attempt was a flaky test.  Calls our flaky test detection into question.  But as far as I know this was never committed.

Looks like our flaky test detection code will report a constant failure as a flaky test if another test happens to flake at the same time:
http://trac.webkit.org/browser/trunk/WebKitTools/Scripts/webkitpy/tool/bot/commitqueuetask.py#L171

Notice how if first_failing_tests is non-empty, we run the tests again, if a different set of tests fail, we report them all as flaky (which is wrong!).

I'll file a separate bug to fix.
Comment 10 Eric Seidel (no email) 2010-12-17 12:57:54 PST
Filed bug 51272 about the flaky test detection code being wrong.