Bug 150450 – Add graphics benchmark to performance bot

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Said Abou-Hallawa]

Need to automate running and reporting the results of the graphics benchmark.

[Said Abou-Hallawa]

Created attachment 263835 [details]
Patch

[Said Abou-Hallawa]

<rdar://problem/22882966>

Comment on attachment 263835 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=263835&action=review

> Tools/Scripts/webkitpy/benchmark_runner/data/plans/animometer.plan:2
> +    "timeout": 30000,

I would like to choose shorter timeout for this test. 500min seems too long for a test.

[Said Abou-Hallawa]

Created attachment 263855 [details]
Patch

[Said Abou-Hallawa]

Comment on attachment 263835 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=263835&action=review

>> Tools/Scripts/webkitpy/benchmark_runner/data/plans/animometer.plan:2
>> +    "timeout": 30000,
> 
> I would like to choose shorter timeout for this test. 500min seems too long for a test.

Yes you are right. I changed the timeout to be 1200s. Currently the benchmark takes exactly 1080s. But soon I am going to lower this number even to be less than 500s.

[Said Abou-Hallawa]

Created attachment 264110 [details]
Output file

[Said Abou-Hallawa]

Here is the output of running the benchmark after applying the attached patch using the command:

python Tools/Scripts/run-benchmark --plan animometer --platform osx --browser safari --output-file animometer --count 1

Animometer:Score:Geometric: 584.4pt stdev=0.9%
          Basic canvas path suite:Score:Geometric: 1.3377Kpt stdev=0.0%
                                 Canvas arc segments:Score: 855.4pt stdev=0.8%
                                 Canvas arc segments, fill:Score: 1.510Kpt stdev=0.2%
                                 Canvas arcTo segments:Score: 948pt stdev=4.5%
                                 Canvas arcTo segments, fill:Score: 1.454Kpt stdev=0.5%
                                 Canvas bezier path:Score: 435.2pt stdev=0.1%
                                 Canvas bezier path, fill:Score: 1.0943Kpt stdev=0.1%
                                 Canvas bezier segments:Score: 854pt stdev=3.0%
                                 Canvas line path with dash pattern:Score: 1.096Kpt stdev=0.3%
                                 Canvas line path, bevel join:Score: 865pt stdev=4.9%
                                 Canvas line path, fill:Score: 1.871Kpt stdev=1.0%
                                 Canvas line path, miter join:Score: 855pt stdev=3.7%
                                 Canvas line path, round join:Score: 852pt stdev=2.6%
                                 Canvas line segments, butt caps:Score: 4.158Kpt stdev=0.3%
                                 Canvas line segments, round caps:Score: 4.14Kpt stdev=1.5%
                                 Canvas line segments, square caps:Score: 4.18Kpt stdev=1.2%
                                 Canvas quadratic path:Score: 566pt stdev=1.5%
                                 Canvas quadratic path, fill:Score: 1.30Kpt stdev=1.5%
                                 Canvas quadratic segments:Score: 1.1998Kpt stdev=0.1%
                                 Canvas rects:Score: 1.76Kpt stdev=1.9%
                                 Canvas rects, fill:Score: 3.217Kpt stdev=0.6%
          Canvas suite:Score:Geometric: 545.5pt stdev=0.5%
                      canvas bouncing PNG images:Score: 2.82Kpt stdev=1.2%
                      canvas bouncing SVG images:Score: 59.08pt stdev=0.1%
                      canvas bouncing circles:Score: 2.03Kpt stdev=3.3%
                      canvas bouncing clipped rects:Score: 453pt stdev=1.4%
                      canvas bouncing gradient circles:Score: 315.9pt stdev=0.6%
          HTML suite:Score:Geometric: 281pt stdev=1.5%
                    CSS bouncing PNG images:Score: 839pt stdev=2.2%
                    CSS bouncing SVG images:Score: 41.0pt stdev=2.4%
                    CSS bouncing circles:Score: 778.2pt stdev=0.6%
                    CSS bouncing clipped rects:Score: 279pt stdev=4.7%
                    CSS bouncing gradient circles:Score: 162pt stdev=1.6%
                    CSS layering text:Score: 411pt stdev=1.8%
          SVG suite:Score:Geometric: 568pt stdev=2.6%
                   SVG bouncing PNG images:Score: 2.70Kpt stdev=8.2%
                   SVG bouncing SVG images:Score: 46.9pt stdev=2.7%
                   SVG bouncing circles:Score: 4.81Kpt stdev=2.0%
                   SVG bouncing clipped rects:Score: 364pt stdev=2.0%
                   SVG bouncing gradient circles:Score: 268pt stdev=2.0%

Also the output file is attached.

[Said Abou-Hallawa]

Created attachment 264112 [details]
Patch

Comment on attachment 264112 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=264112&action=review

> Tools/Scripts/webkitpy/benchmark_runner/data/patches/Animometer.patch:95
> ++});

I do not have review privilege. I've invited Ryosuke to take a look.

Comment on attachment 264112 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=264112&action=review

>> Tools/Scripts/webkitpy/benchmark_runner/data/patches/Animometer.patch:95
>> ++});
> 
> I do not have review privilege. I've invited Ryosuke to take a look.

The format is verified by submitting to my locally instance.

> Tools/Scripts/webkitpy/benchmark_runner/data/plans/animometer.plan:2
> +    "timeout": 1200,

I run the test locally. It takes almost 19mins. I will suggest 1.5x / 2x the time the test usually takes.

[Chris Dumez]

Comment on attachment 264112 [details]
Patch

rs=me based on Dewei's informal review. Please address his minor comment before landing.

[Said Abou-Hallawa]

Created attachment 264757 [details]
Patch

[Said Abou-Hallawa]

Comment on attachment 264112 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=264112&action=review

>> Tools/Scripts/webkitpy/benchmark_runner/data/plans/animometer.plan:2
>> +    "timeout": 1200,
> 
> I run the test locally. It takes almost 19mins. I will suggest 1.5x / 2x the time the test usually takes.

I increased the timeout to be 1800 seconds.

[WebKit Commit Bot]

Comment on attachment 264757 [details]
Patch

Clearing flags on attachment: 264757

Committed r192006: <http://trac.webkit.org/changeset/192006>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.