Bug 145854

Summary: Move all content extension tests to same directory to fix flakiness
Product: WebKit Reporter: Alex Christensen <achristensen>
Component: Tools / TestsAssignee: Alex Christensen <achristensen>
Status: RESOLVED FIXED    
Severity: Normal    
Priority: P2    
Version: 528+ (Nightly build)   
Hardware: Unspecified   
OS: Unspecified   
Attachments:
Description Flags
Patch ap: review+

Description Alex Christensen 2015-06-10 14:09:39 PDT
Right now there are two directories with content extension tests, which causes them to be run with two WebKitTestRunners at the same time, and sometimes they delete each other's content extension when resetting the state between tests.  I'm pretty sure this is what is causing flakiness like http://webkit-test-results.appspot.com/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2Fcontentextensions&revision=185388
Comment 1 Alex Christensen 2015-06-10 14:12:16 PDT
Created attachment 254675 [details]
Patch
Comment 2 Alexey Proskuryakov 2015-06-10 14:17:11 PDT
Comment on attachment 254675 [details]
Patch

This is ok as a short term fix, however the requirement to have all these tests in one directory is not something that we should remember forever, and also we should have tests work with -f. 

How can we make parallel testing processes independent?
Comment 3 Alex Christensen 2015-06-10 14:36:57 PDT
http://trac.webkit.org/changeset/185430

The problem is related to the call to _removeAllContentExtensions in TestControllerMac.mm but I'm not sure how to safely make it work with different directories.  And I'm only about 90% sure this moving-all-tests-to-the-same-directory will work.