Bug 167867 – Beef up the ARES-6 explainer text

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Filip Pizlo]

Patch forthcoming.

[Filip Pizlo]

Created attachment 300684 [details]
the patch

[Filip Pizlo]

Created attachment 300685 [details]
the patch

[Filip Pizlo]

Created attachment 300734 [details]
more

[Saam Barati]

Comment on attachment 300734 [details]
more

View in context: https://bugs.webkit.org/attachment.cgi?id=300734&action=review

r=me

> PerformanceTests/ARES-6/index.html:111
> +        <p>ARES-6 rewards browsers that start up quickly and run smoothly. It's not enough to just measure the total running time of a workload. Browsers may perform differently for the same workload depending on how many times it has run. Garbage collection runs periodically, making some iterations take longer than others. Code that runs repeatedly gets optimized by the browser, so the first iteration of any workload is more expensive than the rest. ARES-6 runs these benchmarks for 200 iterations and reports the execution time of the first iteration, the average of the worst 2% iterations, and the overall average. Each of these values is given an equal weight when computing the ovarall time. ARES-6 equally rewards fast start-up, low jank, and sophisticated adaptive optimizations for long-running code.</p>

Maybe it's worth mentioning here that "the overall time" is the geomean of those three metrics?

[Filip Pizlo]

(In reply to comment #4)
> Comment on attachment 300734 [details]
> more
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=300734&action=review
> 
> r=me
> 
> > PerformanceTests/ARES-6/index.html:111
> > +        <p>ARES-6 rewards browsers that start up quickly and run smoothly. It's not enough to just measure the total running time of a workload. Browsers may perform differently for the same workload depending on how many times it has run. Garbage collection runs periodically, making some iterations take longer than others. Code that runs repeatedly gets optimized by the browser, so the first iteration of any workload is more expensive than the rest. ARES-6 runs these benchmarks for 200 iterations and reports the execution time of the first iteration, the average of the worst 2% iterations, and the overall average. Each of these values is given an equal weight when computing the ovarall time. ARES-6 equally rewards fast start-up, low jank, and sophisticated adaptive optimizations for long-running code.</p>
> 
> Maybe it's worth mentioning here that "the overall time" is the geomean of
> those three metrics?

I changed "overall average" to say "overall geometric mean".

[Filip Pizlo]

Landed in https://trac.webkit.org/changeset/212129