In this function in WebKitPolicyClient test, void beforeAll() { kServer = new WebKitTestServer(); kServer->run(serverCallback); PolicyClientTest::add("WebKitPolicyClient", "navigation-policy", testNavigationPolicy); // 1 PolicyClientTest::add("WebKitPolicyClient", "response-policy", testResponsePolicy); // 2 PolicyClientTest::add("WebKitPolicyClient", "new-window-policy", testNewWindowPolicy); // 3 } If you run any permutations where 3 is not last, you will hit assertions. Placing 3 first gives it the best chance of failing. The issue is caused due to the piggy timeouts in the new window policy test, they come with a comment, // Using a short timeout is a bit ugly here, but it's hard to get around because if we block // the new window signal we cannot halt the main loop in the create callback. If we // halt the main loop in the policy decision, the create callback never executes.