Bug 105513 - Add a performance test for Accelerated Compositing.
Summary: Add a performance test for Accelerated Compositing.
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
Depends on:
Blocks: 77037
  Show dependency treegraph
Reported: 2012-12-20 01:49 PST by Hanyee Kim
Modified: 2013-01-13 21:53 PST (History)
7 users (show)

See Also:

Patch (222.12 KB, patch)
2012-12-20 01:58 PST, Hanyee Kim
no flags Details | Formatted Diff | Diff
This is sample results. (24.02 KB, text/html)
2012-12-20 03:08 PST, Hanyee Kim
no flags Details
Patch (221.56 KB, patch)
2012-12-20 03:11 PST, Hanyee Kim
simon.fraser: review-
simon.fraser: commit-queue-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Hanyee Kim 2012-12-20 01:49:29 PST
I would like to measure performance of Accelerated Compositing.
So, I add a performance test for Accelerated Compositing.
This test case is a modified version of Falling Leaves.
Comment 1 Hanyee Kim 2012-12-20 01:58:07 PST
Created attachment 180303 [details]
Comment 2 Ryosuke Niwa 2012-12-20 02:07:12 PST
Comment on attachment 180303 [details]

View in context: https://bugs.webkit.org/attachment.cgi?id=180303&action=review

> PerformanceTests/Animation/leaves/leaves.css:1
> +/*

What is the license of this file?

> PerformanceTests/ChangeLog:12
> +        The original version does not measure fps, so I added code to measure fps using
> +        WebKitAnimationFrame.

Does this test generate stable results? Could you upload sample results (PerformaneTestResults.html) by running run-perf-tests PerformanceTests/Animation/Leaves.html 4-5 times?

> PerformanceTests/ChangeLog:37
> +        (body):
> +        (#container):
> +        (#leafContainer):
> +        (#message):
> +        (p):
> +        (a):
> +        (em):
> +        (.phone):
> +        (#leafContainer > div):
> +        (#leafContainer > div > img):
> +        (@-webkit-keyframes fade):
> +        (95%):
> +        (100%):
> +        (@-webkit-keyframes drop):
> +        (@-webkit-keyframes clockwiseSpin):
> +        (@-webkit-keyframes counterclockwiseSpinAndFlip):

Please remove this.

> PerformanceTests/ChangeLog:39
> +        * Animation/leaves/leaves.html: Added.
> +        * Animation/leaves/leaves.js: Added.

Please add this as Animation/Leaves.html (notice the capitalization!).
You can add css and png files under Animation/resources/

> PerformanceTests/ChangeLog:48
> +        (init):
> +        (randomInteger):
> +        (randomFloat):
> +        (pixelValue):
> +        (durationValue):
> +        (createALeaf):
> +        (window.onload):
> +        (onCompletedRun):
> +        (onAnimationFrame):

Comment 3 Ryosuke Niwa 2012-12-20 02:11:26 PST
Also, is accelerated compositing even enabled in DRT?
Comment 4 Hanyee Kim 2012-12-20 03:08:18 PST
Created attachment 180312 [details]
This is sample results.
Comment 5 Hanyee Kim 2012-12-20 03:11:13 PST
Created attachment 180313 [details]
Comment 6 Hanyee Kim 2012-12-20 03:18:23 PST
(In reply to comment #3)
Thank you for reviewing!!

I uploaded new patch and sample result html file.
The license of this case is in each file. I think that this case can upload in WebKit. I will check this license to Falling Leaves author and comment again.
This test generate stable results.
I know that accelerated compositing is enabled in DRT.
Comment 7 Hanyee Kim 2012-12-26 00:53:33 PST
I asked Dean Jackson, the Author of Falling Leaves about the license and he told me that there is no problem.
Comment 8 Simon Fraser (smfr) 2013-01-02 20:52:34 PST
Comment on attachment 180313 [details]

View in context: https://bugs.webkit.org/attachment.cgi?id=180313&action=review

> PerformanceTests/Animation/resources/leaves.js:178
> +window.onload = function () {
> +    PerfTestRunner.prepareToMeasureValuesAsync({iterationCount: 10, done: onCompletedRun, unit: 'fps'});
> +    window.webkitRequestAnimationFrame(onAnimationFrame);
> +}

window.addEventListener('load', ...) is better style.

> PerformanceTests/Animation/resources/leaves.js:201
> +function onAnimationFrame() {
> +    if ((iteration++ % FRAMES_PER_TIMER_READING) == 0) {
> +        var currTime = new Date();
> +
> +        // Calculate the framerate based upon the difference between the absolute times of the oldest and newest frames, subdivided by how macany frames were drawn inbetween
> +        var frameRateVal = (FRAMES_PER_TIMER_READING * 1000) / (currTime - lastMeasuredTime);
> +        if (!isNaN(frameRateVal))
> +            PerfTestRunner.measureValueAsync(frameRateVal);
> +
> +        lastMeasuredTime = currTime;
> +    }
> +    if (!measureCompleted) {
> +        window.webkitRequestAnimationFrame(onAnimationFrame);
> +    }
> +}

You're assuming that RAF frequency is related to the frequency with which CSS animations are rendered. This is not true on some platforms, so I don't think this is really a useful way to measure CSS animation performance.

There are other interesting things to measure: how fast this animations runs when driven via RAF, or how expensive it is to do layout on a page with lots of accelerated compositing layers. But I don't think this test useful for all platforms.
Comment 9 Hanyee Kim 2013-01-08 23:42:58 PST
(In reply to comment #8)
Thanks for the feedback.

Could you tell me which platform is problematic? I understand RAF is not exact to measure AC rendering performance on some platforms, but I think that it is the most proper method at this moment. I will check the other methods that you mentioned.

Please tell me if you have another ideas that can measure AC performance more exactly.
Comment 10 Simon Fraser (smfr) 2013-01-09 09:10:16 PST
On Mac (and I think Chromium), compositing itself happens in another thread or indeed another process. This is totally separate from RAF.
Comment 11 Hanyee Kim 2013-01-09 21:41:44 PST
(In reply to comment #10)
According to your comment #10, on Mac (or Chromium), I assume that the other methods you mentioned will not be able to measure the AC performance exactly.

So I propose a new idea: Add implementation to measure performance in each platforms and PerformanceTest gets the measured value through window.internals. I think that it will be able to measure a correct value, but it need platform-specific implementation. What do you think of this?
Comment 12 Ryosuke Niwa 2013-01-09 21:50:37 PST
+jochen, +senorblanco from Chromium.
Comment 13 Simon Fraser (smfr) 2013-01-09 22:00:07 PST
I don't see any value in a WebKit test that measures compositing performance of some other framework on Mac (and, unless the content is crazy, it's pretty much always going to tell you 60fps).

As I said above, the more interesting performance issues with compositing happen when the page content is being modified dynamically.
Comment 14 Hanyee Kim 2013-01-13 20:20:09 PST
(In reply to comment #13)
I understand that this test case is not useful on desktop (mac, chromium).
But, it is helpful on embedded platforms (mobile device, TV, ...), because the performance of CSS animations does not reach 60fps on many devices.

Why do you think that the page content being modified dynamically is more interesting to measure? Is it related to performance of CSS animation?
Comment 15 Ryosuke Niwa 2013-01-13 21:53:35 PST
(In reply to comment #14)
> (In reply to comment #13)
> I understand that this test case is not useful on desktop (mac, chromium).
> But, it is helpful on embedded platforms (mobile device, TV, ...), because the performance of CSS animations does not reach 60fps on many devices.

It appears to me that this should be handled by browser-level performance tests. WebKit's performance tests are designed to measure the performance of WebCore, not a browser that embeds WebKit.