DRT tool needs to capture real "on screen" window content

• Component: Tools / Tests
• Status: RESOLVED WONTFIX

[Pierre-Olivier Latour]

For some of our pixel testing, we need to have the DRT tool support an "--onscreen" option that makes the window visible on screen while the tests are running and capture its contents by read from the screen, not the window backbuffer.

[Pierre-Olivier Latour]

I will provide a tentative patch soon.

[Mark Rowe (bdash)]

Why do we need this?

[Pierre-Olivier Latour]

This bug is dependent on https://bugs.webkit.org/show_bug.cgi?id=21322

[Pierre-Olivier Latour]

(In reply to comment #2)
> Why do we need this?

In some case, we really need to know what the window content really looks like on screen, and grabbing the backbuffer is not sufficient. For more details, we can talk directly.

[Pierre-Olivier Latour]

Created attachment 24246 [details]
First revision

[Pierre-Olivier Latour]

Created attachment 24251 [details]
2nd revision

[Eric Seidel (no email)]

(In reply to comment #4)
> (In reply to comment #2)
> > Why do we need this?
> 
> In some case, we really need to know what the window content really looks like
> on screen, and grabbing the backbuffer is not sufficient. For more details, we
> can talk directly.

I'd be very curious as well.  Could you maybe post some explanation in the bug?  Chromium's test_shell also renders onscreen, but I'd like them to move towards a model similar to DRT where the window is offscreen instead.  You sound like you have reasons not to move to such?  When is the backing store for a window going to be different from the contents captured some other way?  When OpenGL is in play?

[Pierre-Olivier Latour]

> I'd be very curious as well.  Could you maybe post some explanation in the bug?
>  Chromium's test_shell also renders onscreen, but I'd like them to move towards
> a model similar to DRT where the window is offscreen instead.  You sound like
> you have reasons not to move to such?  When is the backing store for a window
> going to be different from the contents captured some other way?  When OpenGL
> is in play?

You are correct: if there's a GL surface on the window, like for the Apple Quartz Composer webkit plug-in, or third party plug-ins that use GL, by definition the rendering of these plug-ins is not in the window backing. So to get a proper snapshot, you need to use the new APIs in CG introduced in 10.5 (or read directly from the framebuffer, but it's more complicated).

There are probably other cases I can't think of (maybe if the bits put on screen are "touched" in any way when blit from the window backing like color matching).

[Pierre-Olivier Latour]

More investigation showed that CGWindowListCreateImage() works correctly even if the window is not visible (i.e. outside of the screen bounds), as long as it's ordered in. This includes capturing the surface on top of the window.

One point of interest is that in case of multiple video cards configurations, windows out of the screens area are assumed to be on the primary video card (this matters for OpenGL virtual screen).

Note that CGWindowListCreateImage() is only available on Mac OS X 10.5 and later, and there's no satisfying replacement on earlier OS. You could read from the framebuffer directly, but this involves OpenGL and require the window to be both visible and un-occluded on screen.

Finally, the image you get through CGWindowListCreateImage() very slightly differs from the one you get by reading the window backing buffer as the window server compositing path gets involved. Practically, this means there are very small dithering artifacts (at least on my iMac).

[Pierre-Olivier Latour]

Created attachment 24279 [details]
3rd revision

After discussing with Darin, he pointed out this behavior, if better, shouldn't be opt-in, but the default. So here's a new patch with the old behavior gone and cleaned up code as well.

[Timothy Hatcher]

Comment on attachment 24279 [details]
3rd revision

This should just use the BUILDING_ON_TIGER macro defined in DumpRenderTreeMac.h.

#ifndef BUILDING_ON_TIGER
// new code
#else
// old code
#endif

Otherwise looks good.

[Pierre-Olivier Latour]

Because of interdependencies, this patch will be merged with the master one at
https://bugs.webkit.org/show_bug.cgi?id=21322