Bug 21495 - DRT tool needs to capture real "on screen" window content
Summary: DRT tool needs to capture real "on screen" window content
Status: RESOLVED WONTFIX
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Mac OS X 10.5
: P2 Normal
Assignee: Nobody
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2008-10-08 19:09 PDT by Pierre-Olivier Latour
Modified: 2008-10-27 12:25 PDT (History)
1 user (show)

See Also:


Attachments
First revision (8.47 KB, patch)
2008-10-09 17:01 PDT, Pierre-Olivier Latour
no flags Details | Formatted Diff | Diff
2nd revision (8.08 KB, patch)
2008-10-09 19:37 PDT, Pierre-Olivier Latour
no flags Details | Formatted Diff | Diff
3rd revision (2.25 KB, patch)
2008-10-10 16:29 PDT, Pierre-Olivier Latour
timothy: review-
Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Pierre-Olivier Latour 2008-10-08 19:09:08 PDT
For some of our pixel testing, we need to have the DRT tool support an "--onscreen" option that makes the window visible on screen while the tests are running and capture its contents by read from the screen, not the window backbuffer.
Comment 1 Pierre-Olivier Latour 2008-10-08 19:11:20 PDT
I will provide a tentative patch soon.
Comment 2 Mark Rowe (bdash) 2008-10-08 19:15:48 PDT
Why do we need this?
Comment 3 Pierre-Olivier Latour 2008-10-08 19:36:32 PDT
This bug is dependent on https://bugs.webkit.org/show_bug.cgi?id=21322
Comment 4 Pierre-Olivier Latour 2008-10-08 19:39:13 PDT
(In reply to comment #2)
> Why do we need this?

In some case, we really need to know what the window content really looks like on screen, and grabbing the backbuffer is not sufficient. For more details, we can talk directly.
Comment 5 Pierre-Olivier Latour 2008-10-09 17:01:31 PDT
Created attachment 24246 [details]
First revision
Comment 6 Pierre-Olivier Latour 2008-10-09 19:37:04 PDT
Created attachment 24251 [details]
2nd revision
Comment 7 Eric Seidel (no email) 2008-10-10 00:27:06 PDT
(In reply to comment #4)
> (In reply to comment #2)
> > Why do we need this?
> 
> In some case, we really need to know what the window content really looks like
> on screen, and grabbing the backbuffer is not sufficient. For more details, we
> can talk directly.

I'd be very curious as well.  Could you maybe post some explanation in the bug?  Chromium's test_shell also renders onscreen, but I'd like them to move towards a model similar to DRT where the window is offscreen instead.  You sound like you have reasons not to move to such?  When is the backing store for a window going to be different from the contents captured some other way?  When OpenGL is in play?
Comment 8 Pierre-Olivier Latour 2008-10-10 12:15:19 PDT
> I'd be very curious as well.  Could you maybe post some explanation in the bug?
>  Chromium's test_shell also renders onscreen, but I'd like them to move towards
> a model similar to DRT where the window is offscreen instead.  You sound like
> you have reasons not to move to such?  When is the backing store for a window
> going to be different from the contents captured some other way?  When OpenGL
> is in play?

You are correct: if there's a GL surface on the window, like for the Apple Quartz Composer webkit plug-in, or third party plug-ins that use GL, by definition the rendering of these plug-ins is not in the window backing. So to get a proper snapshot, you need to use the new APIs in CG introduced in 10.5 (or read directly from the framebuffer, but it's more complicated).

There are probably other cases I can't think of (maybe if the bits put on screen are "touched" in any way when blit from the window backing like color matching).
Comment 9 Pierre-Olivier Latour 2008-10-10 16:25:29 PDT
More investigation showed that CGWindowListCreateImage() works correctly even if the window is not visible (i.e. outside of the screen bounds), as long as it's ordered in. This includes capturing the surface on top of the window.

One point of interest is that in case of multiple video cards configurations, windows out of the screens area are assumed to be on the primary video card (this matters for OpenGL virtual screen).

Note that CGWindowListCreateImage() is only available on Mac OS X 10.5 and later, and there's no satisfying replacement on earlier OS. You could read from the framebuffer directly, but this involves OpenGL and require the window to be both visible and un-occluded on screen.

Finally, the image you get through CGWindowListCreateImage() very slightly differs from the one you get by reading the window backing buffer as the window server compositing path gets involved. Practically, this means there are very small dithering artifacts (at least on my iMac).
Comment 10 Pierre-Olivier Latour 2008-10-10 16:29:36 PDT
Created attachment 24279 [details]
3rd revision

After discussing with Darin, he pointed out this behavior, if better, shouldn't be opt-in, but the default. So here's a new patch with the old behavior gone and cleaned up code as well.
Comment 11 Timothy Hatcher 2008-10-17 04:27:43 PDT
Comment on attachment 24279 [details]
3rd revision

This should just use the BUILDING_ON_TIGER macro defined in DumpRenderTreeMac.h.

#ifndef BUILDING_ON_TIGER
// new code
#else
// old code
#endif

Otherwise looks good.
Comment 12 Pierre-Olivier Latour 2008-10-22 19:12:18 PDT
Because of interdependencies, this patch will be merged with the master one at
https://bugs.webkit.org/show_bug.cgi?id=21322