Bug 105513 – Add a performance test for Accelerated Compositing.

• Component: Tools / Tests
• Status: NEW

[Hanyee Kim]

I would like to measure performance of Accelerated Compositing.
So, I add a performance test for Accelerated Compositing.
This test case is a modified version of Falling Leaves.
(http://www.webkit.org/blog-files/leaves/)

[Hanyee Kim]

Created attachment 180303 [details]
Patch

[Ryosuke Niwa]

Comment on attachment 180303 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=180303&action=review

> PerformanceTests/Animation/leaves/leaves.css:1
> +/*

What is the license of this file?

> PerformanceTests/ChangeLog:12
> +        The original version does not measure fps, so I added code to measure fps using
> +        WebKitAnimationFrame.

Does this test generate stable results? Could you upload sample results (PerformaneTestResults.html) by running run-perf-tests PerformanceTests/Animation/Leaves.html 4-5 times?

> PerformanceTests/ChangeLog:37
> +        (body):
> +        (#container):
> +        (#leafContainer):
> +        (#message):
> +        (p):
> +        (a):
> +        (em):
> +        (.phone):
> +        (#leafContainer > div):
> +        (#leafContainer > div > img):
> +        (@-webkit-keyframes fade):
> +        (95%):
> +        (100%):
> +        (@-webkit-keyframes drop):
> +        (@-webkit-keyframes clockwiseSpin):
> +        (@-webkit-keyframes counterclockwiseSpinAndFlip):

Please remove this.

> PerformanceTests/ChangeLog:39
> +        * Animation/leaves/leaves.html: Added.
> +        * Animation/leaves/leaves.js: Added.

Please add this as Animation/Leaves.html (notice the capitalization!).
You can add css and png files under Animation/resources/

> PerformanceTests/ChangeLog:48
> +        (init):
> +        (randomInteger):
> +        (randomFloat):
> +        (pixelValue):
> +        (durationValue):
> +        (createALeaf):
> +        (window.onload):
> +        (onCompletedRun):
> +        (onAnimationFrame):

Ditto.

[Ryosuke Niwa]

Also, is accelerated compositing even enabled in DRT?

[Hanyee Kim]

Created attachment 180312 [details]
This is sample results.

[Hanyee Kim]

Created attachment 180313 [details]
Patch

[Hanyee Kim]

(In reply to comment #3)
Thank you for reviewing!!

I uploaded new patch and sample result html file.
The license of this case is in each file. I think that this case can upload in WebKit. I will check this license to Falling Leaves author and comment again.
This test generate stable results.
I know that accelerated compositing is enabled in DRT.

[Hanyee Kim]

I asked Dean Jackson, the Author of Falling Leaves about the license and he told me that there is no problem.

[Simon Fraser (smfr)]

Comment on attachment 180313 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=180313&action=review

> PerformanceTests/Animation/resources/leaves.js:178
> +window.onload = function () {
> +    PerfTestRunner.prepareToMeasureValuesAsync({iterationCount: 10, done: onCompletedRun, unit: 'fps'});
> +    window.webkitRequestAnimationFrame(onAnimationFrame);
> +}

window.addEventListener('load', ...) is better style.

> PerformanceTests/Animation/resources/leaves.js:201
> +function onAnimationFrame() {
> +    if ((iteration++ % FRAMES_PER_TIMER_READING) == 0) {
> +        var currTime = new Date();
> +
> +        // Calculate the framerate based upon the difference between the absolute times of the oldest and newest frames, subdivided by how macany frames were drawn inbetween
> +        var frameRateVal = (FRAMES_PER_TIMER_READING * 1000) / (currTime - lastMeasuredTime);
> +        if (!isNaN(frameRateVal))
> +            PerfTestRunner.measureValueAsync(frameRateVal);
> +
> +        lastMeasuredTime = currTime;
> +    }
> +    if (!measureCompleted) {
> +        window.webkitRequestAnimationFrame(onAnimationFrame);
> +    }
> +}

You're assuming that RAF frequency is related to the frequency with which CSS animations are rendered. This is not true on some platforms, so I don't think this is really a useful way to measure CSS animation performance.

There are other interesting things to measure: how fast this animations runs when driven via RAF, or how expensive it is to do layout on a page with lots of accelerated compositing layers. But I don't think this test useful for all platforms.

[Hanyee Kim]

(In reply to comment #8)
Thanks for the feedback.

Could you tell me which platform is problematic? I understand RAF is not exact to measure AC rendering performance on some platforms, but I think that it is the most proper method at this moment. I will check the other methods that you mentioned.

Please tell me if you have another ideas that can measure AC performance more exactly.

[Simon Fraser (smfr)]

On Mac (and I think Chromium), compositing itself happens in another thread or indeed another process. This is totally separate from RAF.

[Hanyee Kim]

(In reply to comment #10)
According to your comment #10, on Mac (or Chromium), I assume that the other methods you mentioned will not be able to measure the AC performance exactly.

So I propose a new idea: Add implementation to measure performance in each platforms and PerformanceTest gets the measured value through window.internals. I think that it will be able to measure a correct value, but it need platform-specific implementation. What do you think of this?

[Ryosuke Niwa]

+jochen, +senorblanco from Chromium.

[Simon Fraser (smfr)]

I don't see any value in a WebKit test that measures compositing performance of some other framework on Mac (and, unless the content is crazy, it's pretty much always going to tell you 60fps).

As I said above, the more interesting performance issues with compositing happen when the page content is being modified dynamically.

[Hanyee Kim]

(In reply to comment #13)
I understand that this test case is not useful on desktop (mac, chromium).
But, it is helpful on embedded platforms (mobile device, TV, ...), because the performance of CSS animations does not reach 60fps on many devices.

Why do you think that the page content being modified dynamically is more interesting to measure? Is it related to performance of CSS animation?

[Ryosuke Niwa]

(In reply to comment #14)
> (In reply to comment #13)
> I understand that this test case is not useful on desktop (mac, chromium).
> But, it is helpful on embedded platforms (mobile device, TV, ...), because the performance of CSS animations does not reach 60fps on many devices.

It appears to me that this should be handled by browser-level performance tests. WebKit's performance tests are designed to measure the performance of WebCore, not a browser that embeds WebKit.