WebKit Bugzilla
New
Browse
Log In
×
Sign in with GitHub
or
Remember my login
Create Account
·
Forgot Password
Forgotten password account recovery
RESOLVED FIXED
97155
run-perf-tests should record individual value instead of statistics
https://bugs.webkit.org/show_bug.cgi?id=97155
Summary
run-perf-tests should record individual value instead of statistics
Ryosuke Niwa
Reported
2012-09-19 17:58:50 PDT
run-perf-tests should record indivisual value instead of statistics
Attachments
Patch
(16.65 KB, patch)
2012-09-19 18:05 PDT
,
Ryosuke Niwa
morrita
: review+
webkit.review.bot
: commit-queue-
Details
Formatted Diff
Diff
View All
Add attachment
proposed patch, testcase, etc.
Ryosuke Niwa
Comment 1
2012-09-19 18:05:34 PDT
Created
attachment 164811
[details]
Patch
Ryosuke Niwa
Comment 2
2012-09-19 18:06:54 PDT
I don't have access to perf-o-matic code this week so I'll update it next week.
WebKit Review Bot
Comment 3
2012-09-19 18:57:23 PDT
Comment on
attachment 164811
[details]
Patch
Attachment 164811
[details]
did not pass chromium-ews (chromium-xvfb): Output:
http://queues.webkit.org/results/13954001
New failing tests: http/tests/css/link-css-disabled-value-with-slow-loading-sheet.html
Hajime Morrita
Comment 4
2012-09-19 18:59:45 PDT
Comment on
attachment 164811
[details]
Patch The code looks good considering there are some usecases for this value so... what is this for?
Ryosuke Niwa
Comment 5
2012-09-19 19:32:53 PDT
(In reply to
comment #4
)
> (From update of
attachment 164811
[details]
) > The code looks good considering there are some usecases for this value so... what is this for?
In the long term, perf-o-matic should be able to store these values and do some statistical analysis on them. e.g. Once we ported Datazilla, it has a built-in student's t-test and other things. For now, we can just report them on results page so that we can use them to analyze test results locally.
Hajime Morrita
Comment 6
2012-09-19 19:43:14 PDT
(In reply to
comment #5
)
> In the long term, perf-o-matic should be able to store these values and do some statistical analysis on them. e.g. Once we ported Datazilla, it has a built-in student's t-test and other things. For now, we can just report them on results page so that we can use them to analyze test results locally.
Got it. Thanks.
WebKit Review Bot
Comment 7
2012-09-19 21:24:47 PDT
Comment on
attachment 164811
[details]
Patch Rejecting
attachment 164811
[details]
from commit-queue. Failed to run "['/mnt/git/webkit-commit-queue/Tools/Scripts/webkit-patch', '--status-host=queues.webkit.org', '-..." exit_code: 1 Last 500 characters of output: ueue/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 70, in run_and_handle_errors self._run(tool, options, state) File "/mnt/git/webkit-commit-queue/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 64, in _run step(tool, options).run(state) File "/mnt/git/webkit-commit-queue/Tools/Scripts/webkitpy/tool/steps/validatereviewer.py", line 50, in run if changelog_entry.has_valid_reviewer(): AttributeError: 'NoneType' object has no attribute 'has_valid_reviewer' Full output:
http://queues.webkit.org/results/13916045
Ryosuke Niwa
Comment 8
2012-09-19 22:13:44 PDT
Committed
r129091
: <
http://trac.webkit.org/changeset/129091
>
WebKit Review Bot
Comment 9
2012-09-20 06:08:07 PDT
Re-opened since this is blocked by 97205
Csaba Osztrogonác
Comment 10
2012-09-20 06:15:58 PDT
(In reply to
comment #9
)
> Re-opened since this is blocked by 97205
Rolled out by
https://trac.webkit.org/changeset/129123
, because it broke all perf bots. Apple bot before the patch:
http://build.webkit.org/builders/Apple%20Lion%20Release%20%28Perf%29/builds/5748
Apple bot after the patch:
http://build.webkit.org/builders/Apple%20Lion%20Release%20%28Perf%29/builds/5749
Qt bot before the patch:
http://build.webkit.org/builders/Qt%20Linux%2064-bit%20Release%20%28Perf%29/builds/5002
Qt bot after the patch:
http://build.webkit.org/builders/Qt%20Linux%2064-bit%20Release%20%28Perf%29/builds/5003
Ryosuke Niwa
Comment 11
2012-09-20 10:41:24 PDT
Could you not roll out these perf-test patches? In practice, nobody is looking at the results of these bots (webkit-perf.appspot.com) and bots don't report whether performance has regressed or not anyway. It causes too much svn commit churn if we were rolling out every single patch like this.
Ryosuke Niwa
Comment 12
2012-09-20 10:42:55 PDT
Perf. bots are still in development, and rolling out patches just because tests started to fail will slow down the development process than anything else. It's not helpful.
Csaba Osztrogonác
Comment 13
2012-09-20 12:35:14 PDT
Sure. But I don't understand why we should run broken perf bots for days. It is wasting the CPU resources ...
Ryosuke Niwa
Comment 14
2012-09-20 13:04:01 PDT
Committed
r129158
: <
http://trac.webkit.org/changeset/129158
>
Hajime Morrita
Comment 15
2012-09-20 16:47:58 PDT
(In reply to
comment #12
)
> Perf. bots are still in development, and rolling out patches just because tests started to fail will slow down the development process than anything else. It's not helpful.
Having red in
http://build.webkit.org/console
hurts the perception of the project health, even though current situation is embracing constant redness. If we expect long-running redness, we probably should hide it or make it clear by givin them a separate group IMO. It doesn't seems polite behavior for me to blame people who revert the patch which made bots red.
Ryosuke Niwa
Comment 16
2012-09-20 16:59:25 PDT
(In reply to
comment #15
)
> (In reply to
comment #12
) > > Perf. bots are still in development, and rolling out patches just because tests started to fail will slow down the development process than anything else. It's not helpful. > > If we expect long-running redness, we probably should hide it or make it clear by givin them a separate group IMO.
Yeah, it might make sense for us to add a new category for Perf bots and put them there. We might have added prematurely. It has helped us catching some regressions in the past, but the entire framework is very immature compared to layout tests and other test frameworks we have.
Note
You need to
log in
before you can comment on or make changes to this bug.
Top of Page
Format For Printing
XML
Clone This Bug