Bug 196017

Summary: (Test fix) Override minimumTimeBetweenDataRecordsRemoval in ResourceLoadStatisticsStore::shouldRemoveDataRecord() during layout tests
Product: WebKit Reporter: John Wilander <wilander>
Component: WebKit Misc.Assignee: John Wilander <wilander>
Status: RESOLVED FIXED    
Severity: Normal CC: commit-queue, sroberts, webkit-bug-importer
Priority: P2 Keywords: InRadar
Version: WebKit Nightly Build   
Hardware: Unspecified   
OS: Unspecified   
See Also: https://bugs.webkit.org/show_bug.cgi?id=195923
Attachments:
Description Flags
Patch for landing none

Description John Wilander 2019-03-20 10:55:43 PDT
The test added in https://bugs.webkit.org/show_bug.cgi?id=195923 is failing because ITP refuses (and should refuse) to remove website data repeatedly. We should add an exception to this rule when running under parameters().isRunningTest.
Comment 1 John Wilander 2019-03-20 10:57:19 PDT
Created attachment 365359 [details]
Patch for landing
Comment 2 WebKit Commit Bot 2019-03-20 12:01:37 PDT
Comment on attachment 365359 [details]
Patch for landing

Clearing flags on attachment: 365359

Committed r243222: <https://trac.webkit.org/changeset/243222>
Comment 3 WebKit Commit Bot 2019-03-20 12:01:39 PDT
All reviewed patches have been landed.  Closing bug.
Comment 4 Radar WebKit Bug Importer 2019-03-20 12:05:10 PDT
<rdar://problem/49072535>
Comment 5 Shawn Roberts 2019-03-22 11:36:29 PDT
http/tests/resourceLoadStatistics/website-data-removal-for-site-navigated-to-with-link-decoration.html started to become flaky again on the dashboard ~r243340

Dashboard :

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Ftests%2FresourceLoadStatistics%2Fwebsite-data-removal-for-site-navigated-to-with-link-decoration.html

I can reproduce it locally with :

run-webkit-tests http/tests/resourceLoadStatistics/website-data-removal-for-site-navigated-to-with-link-decoration.html --iterations 500 --child-processes 1

Usually only about 2/500 failures in revisions between r243351 to r243373. with 1 child process.

If I run it in full parallel (-f), it will fail a lot more consistently, but I don't know if that may be causing false positives.

It will fail in full parallel between r242231 and r243347. ~2/5000 iterations

Passes in full parallel with 5000 iterations from r243222 to r243230. 

Local Diff is the same as the bots, with 1 child process or full parallel :

--- /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/http/tests/resourceLoadStatistics/website-data-removal-for-site-navigated-to-with-link-decoration-expected.txt
+++ /Volumes/Data/slave/mojave-release-tests-wk2/build/layout-test-results/http/tests/resourceLoadStatistics/website-data-removal-for-site-navigated-to-with-link-decoration-actual.txt
@@ -32,4 +32,4 @@
         localhost
     gotLinkDecorationFromPrevalentResource: No    isPrevalentResource: No
     isVeryPrevalentResource: No
-    dataRecordsRemoved: 1
+    dataRecordsRemoved: 0
Comment 6 Shawn Roberts 2019-03-27 11:25:20 PDT
Regressed issue down to https://trac.webkit.org/changeset/243348/webkit , appears to be responsible for new behavior.