Bug 156827 - Add content animation tests to benchmark_runner, and allow the runner to collect device data as part of the results
Summary: Add content animation tests to benchmark_runner, and allow the runner to coll...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Simon Fraser (smfr)
URL:
Keywords:
Depends on: 160057
Blocks:
  Show dependency treegraph
 
Reported: 2016-04-20 17:30 PDT by Simon Fraser (smfr)
Modified: 2016-07-21 15:54 PDT (History)
9 users (show)

See Also:


Attachments
Patch (41.03 KB, patch)
2016-04-20 17:46 PDT, Simon Fraser (smfr)
slewis: review+
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Simon Fraser (smfr) 2016-04-20 17:30:31 PDT
Add content animation tests to benchmark_runner, and allow the runner to collect device data as part of the results
Comment 1 Simon Fraser (smfr) 2016-04-20 17:46:17 PDT
Created attachment 276877 [details]
Patch
Comment 2 Stephanie Lewis 2016-04-20 20:30:01 PDT
Comment on attachment 276877 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=276877&action=review

> Tools/ChangeLog:-640
> - 

logs of changelog changes

> Tools/Scripts/webkitpy/benchmark_runner/benchmark_runner.py:81
> +

We still need to kill the server If something goes wrong serving the driver or launching the test.
Comment 3 Csaba Osztrogonác 2016-04-21 09:34:18 PDT
just to document, patches landed in 
- https://trac.webkit.org/changeset/199805
- https://trac.webkit.org/changeset/199806

But unfortunately the new tests fail on all performance bots:

Running Animation/css-animation.html (2 of 153)
ERROR: layer at (0,0) size 800x600
FAILED
Finished: 0.449525 s

Running Animation/raf-animation.html (3 of 153)
ERROR: layer at (0,0) size 800x600
FAILED
Finished: 0.449265 s
Comment 4 Simon Fraser (smfr) 2016-04-21 11:02:11 PDT
(In reply to comment #3)
> just to document, patches landed in 
> - https://trac.webkit.org/changeset/199805
> - https://trac.webkit.org/changeset/199806
> 
> But unfortunately the new tests fail on all performance bots:
> 
> Running Animation/css-animation.html (2 of 153)
> ERROR: layer at (0,0) size 800x600
> FAILED
> Finished: 0.449525 s
> 
> Running Animation/raf-animation.html (3 of 153)
> ERROR: layer at (0,0) size 800x600
> FAILED
> Finished: 0.449265 s

These tests only really make sense for iOS. We should not run them for other platforms.
Comment 5 Csaba Osztrogonác 2016-04-22 01:48:45 PDT
(In reply to comment #4)
> (In reply to comment #3)
> > just to document, patches landed in 
> > - https://trac.webkit.org/changeset/199805
> > - https://trac.webkit.org/changeset/199806
> > 
> > But unfortunately the new tests fail on all performance bots:
> > 
> > Running Animation/css-animation.html (2 of 153)
> > ERROR: layer at (0,0) size 800x600
> > FAILED
> > Finished: 0.449525 s
> > 
> > Running Animation/raf-animation.html (3 of 153)
> > ERROR: layer at (0,0) size 800x600
> > FAILED
> > Finished: 0.449265 s
> 
> These tests only really make sense for iOS. We should not run them for other
> platforms.

In this case please skip them on other platforms. It's not OK leaving bots red.