WebKit Bugzilla
New
Browse
Log In
×
Sign in with GitHub
or
Remember my login
Create Account
·
Forgot Password
Forgotten password account recovery
RESOLVED FIXED
55909
new-run-webkit-tests --run-singly option is busted
https://bugs.webkit.org/show_bug.cgi?id=55909
Summary
new-run-webkit-tests --run-singly option is busted
James Robinson
Reported
2011-03-07 15:33:56 PST
When running with --run-singly option tests that fail and are expected to fail are producing 'unexpected pass' output. Looking at the actual output from the test they seem to be really failing. For example, canvas/philip/tests/2d.composite.operation.clear.html is marked as an expected TEXT diff and actually does fail with a TEXT diff. When run with --run-singly --verbose on chromium linux I get this output: ... 2011-03-07 15:32:45,836 13188 executive.py:365 DEBUG "/usr/bin/wdiff --start-delete=##WDIFF_DEL## --end-delete=##WDIFF_END## --start-insert=##WDIFF_ADD## --end-insert=##WDIFF_END## /usr/local/google/home/jamesr/chrome/src/webkit/Release/layout-test-results/canvas/philip/tests/2d.composite.operation.clear-expected.txt /usr/local/google/home/jamesr/chrome/src/webkit/Release/layout-test-results/canvas/philip/tests/2d.composite.operation.clear-actual.txt" took 0.01s 2011-03-07 15:32:45,883 13188 executive.py:365 DEBUG "ruby -I /usr/local/google/home/jamesr/chrome/src/third_party/WebKit/Websites/bugs.webkit.org/PrettyPatch /usr/local/google/home/jamesr/chrome/src/third_party/WebKit/Websites/bugs.webkit.org/PrettyPatch/prettify.rb /usr/local/google/home/jamesr/chrome/src/webkit/Release/layout-test-results/canvas/philip/tests/2d.composite.operation.clear-diff.txt" took 0.05s 2011-03-07 15:32:45,989 13188 worker_mixin.py:141 DEBUG worker/0 canvas/philip/tests/2d.composite.operation.clear.html passed 2011-03-07 15:32:45,990 13184 printing.py:541 INFO canvas/philip/tests/2d.composite.operation.clear.html -> unexpected pass 2011-03-07 15:32:45,990 13188 worker.py:83 DEBUG worker/0 done ... but if I diff the output on disk locally... $ diff /usr/local/google/home/jamesr/chrome/src/webkit/Release/layout-test-results/canvas/philip/tests/2d.composite.operation.clear-expected.txt /usr/local/google/home/jamesr/chrome/src/webkit/Release/layout-test-results/canvas/philip/tests/2d.composite.operation.clear-actual.txt 1c1 < Passed ---
> Failed assertion ctx.globalCompositeOperation == 'xor' (got clear[string], expected xor[string])
Attachments
Patch
(3.28 KB, patch)
2011-04-12 14:54 PDT
,
Dirk Pranke
no flags
Details
Formatted Diff
Diff
simplify by reading an attribute on SingleTestThread() instead
(2.76 KB, patch)
2011-04-13 14:17 PDT
,
Dirk Pranke
no flags
Details
Formatted Diff
Diff
mask out unused args
(2.75 KB, patch)
2011-04-13 14:18 PDT
,
Dirk Pranke
tony
: review+
Details
Formatted Diff
Diff
Show Obsolete
(2)
View All
Add attachment
proposed patch, testcase, etc.
Dirk Pranke
Comment 1
2011-04-06 19:45:11 PDT
I don't think it should block
bug 34984
and I plan to clear the blocking flag on Mon 4/11. If you disagree, now would be a good time to speak up :).
Dirk Pranke
Comment 2
2011-04-12 14:54:57 PDT
Created
attachment 89278
[details]
Patch
Dirk Pranke
Comment 3
2011-04-12 18:04:38 PDT
clearing blocking flag.
Tony Chang
Comment 4
2011-04-13 10:28:07 PDT
Comment on
attachment 89278
[details]
Patch View in context:
https://bugs.webkit.org/attachment.cgi?id=89278&action=review
> Tools/Scripts/webkitpy/layout_tests/layout_package/worker_mixin.py:170 > + def run(other): > + self._result = worker._run_single_test(driver, test_input)
I think it's more common to do something like: def run(self, worker_mixin=self): worker_mixin._result = ...
> Tools/Scripts/webkitpy/layout_tests/run_webkit_tests_unittest.py:328 > + res, out, err, user = logging_run(['--run-singly', 'failures/unexpected'])
Nit: maybe replace the unused outputs with _
Dirk Pranke
Comment 5
2011-04-13 14:17:29 PDT
Created
attachment 89453
[details]
simplify by reading an attribute on SingleTestThread() instead
Dirk Pranke
Comment 6
2011-04-13 14:18:30 PDT
Created
attachment 89456
[details]
mask out unused args
Dirk Pranke
Comment 7
2011-04-13 18:37:48 PDT
Committed
r83801
Note
You need to
log in
before you can comment on or make changes to this bug.
Top of Page
Format For Printing
XML
Clone This Bug