Bug 150450 - Add graphics benchmark to performance bot
Summary: Add graphics benchmark to performance bot
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Said Abou-Hallawa
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2015-10-22 10:44 PDT by Said Abou-Hallawa
Modified: 2015-11-03 17:59 PST (History)
5 users (show)

See Also:


Attachments
Patch (4.36 KB, patch)
2015-10-22 10:49 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch (4.36 KB, patch)
2015-10-22 13:55 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Output file (4.07 KB, application/octet-stream)
2015-10-26 17:49 PDT, Said Abou-Hallawa
no flags Details
Patch (5.23 KB, patch)
2015-10-26 17:53 PDT, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff
Patch (5.46 KB, patch)
2015-11-03 17:00 PST, Said Abou-Hallawa
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Said Abou-Hallawa 2015-10-22 10:44:23 PDT
Need to automate running and reporting the results of the graphics benchmark.
Comment 1 Said Abou-Hallawa 2015-10-22 10:49:41 PDT
Created attachment 263835 [details]
Patch
Comment 2 Said Abou-Hallawa 2015-10-22 11:28:13 PDT
<rdar://problem/22882966>
Comment 3 dewei_zhu 2015-10-22 11:37:57 PDT
Comment on attachment 263835 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=263835&action=review

> Tools/Scripts/webkitpy/benchmark_runner/data/plans/animometer.plan:2
> +    "timeout": 30000,

I would like to choose shorter timeout for this test. 500min seems too long for a test.
Comment 4 Said Abou-Hallawa 2015-10-22 13:55:04 PDT
Created attachment 263855 [details]
Patch
Comment 5 Said Abou-Hallawa 2015-10-22 13:57:47 PDT
Comment on attachment 263835 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=263835&action=review

>> Tools/Scripts/webkitpy/benchmark_runner/data/plans/animometer.plan:2
>> +    "timeout": 30000,
> 
> I would like to choose shorter timeout for this test. 500min seems too long for a test.

Yes you are right. I changed the timeout to be 1200s. Currently the benchmark takes exactly 1080s. But soon I am going to lower this number even to be less than 500s.
Comment 6 Said Abou-Hallawa 2015-10-26 17:49:25 PDT
Created attachment 264110 [details]
Output file
Comment 7 Said Abou-Hallawa 2015-10-26 17:50:01 PDT
Here is the output of running the benchmark after applying the attached patch using the command:

python Tools/Scripts/run-benchmark --plan animometer --platform osx --browser safari --output-file animometer --count 1

Animometer:Score:Geometric: 584.4pt stdev=0.9%
          Basic canvas path suite:Score:Geometric: 1.3377Kpt stdev=0.0%
                                 Canvas arc segments:Score: 855.4pt stdev=0.8%
                                 Canvas arc segments, fill:Score: 1.510Kpt stdev=0.2%
                                 Canvas arcTo segments:Score: 948pt stdev=4.5%
                                 Canvas arcTo segments, fill:Score: 1.454Kpt stdev=0.5%
                                 Canvas bezier path:Score: 435.2pt stdev=0.1%
                                 Canvas bezier path, fill:Score: 1.0943Kpt stdev=0.1%
                                 Canvas bezier segments:Score: 854pt stdev=3.0%
                                 Canvas line path with dash pattern:Score: 1.096Kpt stdev=0.3%
                                 Canvas line path, bevel join:Score: 865pt stdev=4.9%
                                 Canvas line path, fill:Score: 1.871Kpt stdev=1.0%
                                 Canvas line path, miter join:Score: 855pt stdev=3.7%
                                 Canvas line path, round join:Score: 852pt stdev=2.6%
                                 Canvas line segments, butt caps:Score: 4.158Kpt stdev=0.3%
                                 Canvas line segments, round caps:Score: 4.14Kpt stdev=1.5%
                                 Canvas line segments, square caps:Score: 4.18Kpt stdev=1.2%
                                 Canvas quadratic path:Score: 566pt stdev=1.5%
                                 Canvas quadratic path, fill:Score: 1.30Kpt stdev=1.5%
                                 Canvas quadratic segments:Score: 1.1998Kpt stdev=0.1%
                                 Canvas rects:Score: 1.76Kpt stdev=1.9%
                                 Canvas rects, fill:Score: 3.217Kpt stdev=0.6%
          Canvas suite:Score:Geometric: 545.5pt stdev=0.5%
                      canvas bouncing PNG images:Score: 2.82Kpt stdev=1.2%
                      canvas bouncing SVG images:Score: 59.08pt stdev=0.1%
                      canvas bouncing circles:Score: 2.03Kpt stdev=3.3%
                      canvas bouncing clipped rects:Score: 453pt stdev=1.4%
                      canvas bouncing gradient circles:Score: 315.9pt stdev=0.6%
          HTML suite:Score:Geometric: 281pt stdev=1.5%
                    CSS bouncing PNG images:Score: 839pt stdev=2.2%
                    CSS bouncing SVG images:Score: 41.0pt stdev=2.4%
                    CSS bouncing circles:Score: 778.2pt stdev=0.6%
                    CSS bouncing clipped rects:Score: 279pt stdev=4.7%
                    CSS bouncing gradient circles:Score: 162pt stdev=1.6%
                    CSS layering text:Score: 411pt stdev=1.8%
          SVG suite:Score:Geometric: 568pt stdev=2.6%
                   SVG bouncing PNG images:Score: 2.70Kpt stdev=8.2%
                   SVG bouncing SVG images:Score: 46.9pt stdev=2.7%
                   SVG bouncing circles:Score: 4.81Kpt stdev=2.0%
                   SVG bouncing clipped rects:Score: 364pt stdev=2.0%
                   SVG bouncing gradient circles:Score: 268pt stdev=2.0%

Also the output file is attached.
Comment 8 Said Abou-Hallawa 2015-10-26 17:53:56 PDT
Created attachment 264112 [details]
Patch
Comment 9 dewei_zhu 2015-10-26 18:01:27 PDT
Comment on attachment 264112 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=264112&action=review

> Tools/Scripts/webkitpy/benchmark_runner/data/patches/Animometer.patch:95
> ++});

I do not have review privilege. I've invited Ryosuke to take a look.
Comment 10 dewei_zhu 2015-11-03 14:12:07 PST
Comment on attachment 264112 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=264112&action=review

>> Tools/Scripts/webkitpy/benchmark_runner/data/patches/Animometer.patch:95
>> ++});
> 
> I do not have review privilege. I've invited Ryosuke to take a look.

The format is verified by submitting to my locally instance.

> Tools/Scripts/webkitpy/benchmark_runner/data/plans/animometer.plan:2
> +    "timeout": 1200,

I run the test locally. It takes almost 19mins. I will suggest 1.5x / 2x the time the test usually takes.
Comment 11 Chris Dumez 2015-11-03 14:15:23 PST
Comment on attachment 264112 [details]
Patch

rs=me based on Dewei's informal review. Please address his minor comment before landing.
Comment 12 Said Abou-Hallawa 2015-11-03 17:00:03 PST
Created attachment 264757 [details]
Patch
Comment 13 Said Abou-Hallawa 2015-11-03 17:01:36 PST
Comment on attachment 264112 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=264112&action=review

>> Tools/Scripts/webkitpy/benchmark_runner/data/plans/animometer.plan:2
>> +    "timeout": 1200,
> 
> I run the test locally. It takes almost 19mins. I will suggest 1.5x / 2x the time the test usually takes.

I increased the timeout to be 1800 seconds.
Comment 14 WebKit Commit Bot 2015-11-03 17:59:28 PST
Comment on attachment 264757 [details]
Patch

Clearing flags on attachment: 264757

Committed r192006: <http://trac.webkit.org/changeset/192006>
Comment 15 WebKit Commit Bot 2015-11-03 17:59:32 PST
All reviewed patches have been landed.  Closing bug.