Bug 60277 - [Meta] Add a WebCore testing harness
Summary: [Meta] Add a WebCore testing harness
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: Tools / Tests (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords:
: 42612 (view as bug list)
Depends on: 60313
Blocks: 60211
  Show dependency treegraph
 
Reported: 2011-05-05 09:46 PDT by Dimitri Glazkov (Google)
Modified: 2011-05-27 06:17 PDT (History)
6 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Dimitri Glazkov (Google) 2011-05-05 09:46:43 PDT
Requirements:
* lives in WebCore namespace, allowing tests to use WebCore;
* in DRT only, exposes window.internals namespace-style object, which holds extra methods/properties to enable better testing;
* can be compiled out of the production builds.

Nice-to-haves:
* opens the path for implementing WebCore unit tests;

Benefits compared to layoutTestController:
* share more testing code among ports;
* less plumbing through the WebKit layer.
Comment 1 Dimitri Glazkov (Google) 2011-05-05 09:47:48 PDT
*** Bug 42612 has been marked as a duplicate of this bug. ***
Comment 2 Dimitri Glazkov (Google) 2011-05-05 09:58:50 PDT
So I have a simple/stupid approach mostly using existing machinery:

1) Add ENABLE(TESTING) guard and make sure that all of the harness code is conditionally compiled with this flag;

2) Expose a static isInTestMode flag, which is only true when we're running inside DRT. We already have something like this in Chromium (http://codesearch.google.com/codesearch/p#OAMlx_jo-ck/src/third_party/WebKit/Source/WebCore/platform/chromium/PlatformBridge.h&l=190)

3) Make it so that iff the isInTestMode flag is true, the window.internals object and all corresponding DOM plumbing are visible.

This way, the production builds should simply always set ENABLE(TESTING) to false and never worry about carrying the extra weight of the testing code.

WDYT?
Comment 3 Alexey Proskuryakov 2011-05-05 10:17:19 PDT
At the meeting, we've been thinking about putting the code into a separate project (presumably producing a static library to link testing tools against). That still seems reasonable to me.
Comment 4 Darin Adler 2011-05-05 10:20:49 PDT
A separate library, but not necessarily a separate project in the Xcode project sense; for Xcode it may simply be a separate target.
Comment 5 Dimitri Glazkov (Google) 2011-05-05 10:24:07 PDT
(In reply to comment #4)
> A separate library, but not necessarily a separate project in the Xcode project sense; for Xcode it may simply be a separate target.

A separate project brings all the trouble of exporting/forwarding headers, which I am explicitly trying steer away from. I am aiming to build a much better testing harness, not a slightly better one :)

Will I have less trouble with a separate library? I am Mac build system ignoramus.
Comment 6 Darin Adler 2011-05-05 10:28:43 PDT
(In reply to comment #5)
> Will I have less trouble with a separate library? I am Mac build system ignoramus.

Yes, in Xcode at least.
Comment 7 Dimitri Glazkov (Google) 2011-05-05 10:30:33 PDT
(In reply to comment #6)
> (In reply to comment #5)
> > Will I have less trouble with a separate library? I am Mac build system ignoramus.
> 
> Yes, in Xcode at least.

Ok, lemme play with that. Will report what I find.
Comment 8 Dimitri Glazkov (Google) 2011-05-05 11:37:11 PDT
(In reply to comment #6)
> (In reply to comment #5)
> > Will I have less trouble with a separate library? I am Mac build system ignoramus.
> 
> Yes, in Xcode at least.

Darin, can you help me close my mental loop? I apologize in advance, this is a whole new area for me:

1) With this library, we would still the ENABLE(TESTING) guard and isInTestMode flag, as outlined above. Right?

2) What does building a separate library give us over just plopping this in WebCore target?
Comment 9 Darin Adler 2011-05-05 11:58:44 PDT
(In reply to comment #8)
> 1) With this library, we would still the ENABLE(TESTING) guard and isInTestMode flag, as outlined above. Right?

I don’t know. The idea is that the testing code has access to WebCore code, but isn’t compiled into the WebCore framework/library/DLL.

> 2) What does building a separate library give us over just plopping this in WebCore target?

I’m sorry you missed our WebKit Contributors Meeting session on this. The static library would be linked into tools like DumpRenderTree. Outside the test tool, the testing code would not be present in the WebCore binary, so it would not increase code size or introduce code paths that could be exploited by bad guys.
Comment 10 Dimitri Glazkov (Google) 2011-05-05 13:36:36 PDT
> I’m sorry you missed our WebKit Contributors Meeting session on this. The static library would be linked into tools like DumpRenderTree. Outside the test tool, the testing code would not be present in the WebCore binary, so it would not increase code size or introduce code paths that could be exploited by bad guys.

I was there! :) I just couldn't remember what we decided.
Comment 11 Hajime Morrita 2011-05-05 18:02:42 PDT
In my understanding, WebCoreTestSupport is like a WebKit library/framework.
It accesses WebCore. It has its own binary.

Possible differences are:

- Namespace. It would be OK to put it into WebCore::
- Library type. It would be a static library
- something else that darin and ap mentioned in this thread...

I think restricted WebCore API access (by selective symbol exposure) is
a reasonable compromise and I thinks it's OK to export WebCore APIs only for 
WebCoreTestSupport: We'd need to export them anyway even using existing DRT.

Note that WebCoreTestSupport would need to access WebKit private APIs to retrieve
WebCore::Frame (and other WebCore internal object) from WebFrame-like wrapper objet.