Bug 238731 – Source/WebKit code should be testable with C++ unit tests

• Component: Tools / Tests
• Status: NEW

[Kimmo Kinnunen]

Source/WebKit code should be testable with C++ unit tests

Source/WebKit contains very untrivial code, and it should be possible to write C++ unit tests for this code.

[Kimmo Kinnunen]

Created attachment 456556 [details]
Strawman with BUILD_FOR_TESTING

[Kimmo Kinnunen]

Limiting the solutions to the current build system and to the logic of having tests in TestWebKitAPI:

The options I can come up with with:
1) Use BUILD_FOR_TESTING to select whether to export the tested symbols from WebKit or not

2) ??



The workaround options sometimes discussed but which I don't think work:

N1) Helper static library: "add some testing functionality to WebKitTestSupport.a".
The goal is to test the interfaces extensively. It is not feasible to duplicate the tested interfaces into another header. E.g. to test IPC::Connection, it is not feasible to re-type the whole of IPC::Connection interface to WebKitTestSupport.h to test each and every aspect of said interface.
   

N2) Helper static library: build the tests inside WebKitTestSupport.a
The tests need the tested symbols. IPC::Connection cannot exist in WebKit and WebKitTestSupport at the same time.


N3) Main dynamic library: build the tests inside WebKit.framework
The tests need to use gtest symbols to execute the gtest framework logic, assertions, etc. these cannot exist in WebKit.framework and TestWebKitAPI at the same time

The tests need to use the TestWebKitAPI helpers to print, setup, etc. misc functionality. These cannot exist in WebKit.framework and TestWebKitAPI at the same time

N4) Test via layout tests, expose each and all tested interfaces through JS
It is not feasible or possible to expose all nuances of the C++ interfaces through the JS bindings. Also, this is laboursome extra work to maintain JS bindings to C++ for the sole purpose of running C++ methods. See IPC_TESTING_API.

[Kimmo Kinnunen]

Created attachment 456557 [details]
Strawman with BUILD_FOR_TESTING

[Alexey Proskuryakov]

I appreciate the analysis of challenges around other approaches. 

I still think that this approach doesn't work either - we just can't have a "for testing" build that exports different symbols and headers. One of the reasons is shown by EWS, where tapi failed.

[Kimmo Kinnunen]

(In reply to Alexey Proskuryakov from comment #4)
> I appreciate the analysis of challenges around other approaches. 
> 
> I still think that this approach doesn't work either - we just can't have a
> "for testing" build that exports different symbols and headers. One of the
> reasons is shown by EWS, where tapi failed.

Can you expand? Did having "for testing" build break something?

I'll still try quickly to expand the patch tomorrow, as currently there are no proposed alternatives.

I'll try to workaround the header issue by introducing empty libWebKitTestSupport and have the implementation-private WebKit headers be associated with that project as project headers, so they don't mix with real WebKit framework headers.

[Alexey Proskuryakov]

> Can you expand? Did having "for testing" build break something?

Some of the EWS failures are caused by exports not matching API/SPI headers. It is an invariant that WebKit exports exactly the same symbols as what is exposed in API/SPI.

error: headers don't have symbol '__ZN3IPC10Connection22createClientConnectionENS0_10IdentifierERNS0_6ClientE'
error: headers don't have symbol '__ZN3IPC10Connection22createServerConnectionENS0_10IdentifierERNS0_6ClientE'
error: headers don't have symbol '__ZN3IPC10Connection30createConnectionIdentifierPairEv'

[Radar WebKit Bug Importer]

<rdar://problem/91556917>