The tests: http/wpt/beacon/contentextensions/beacon-blocked.html http/wpt/beacon/contentextensions/beacon-redirect-blocked.html are flaky according to the flakiness dashboard on Sierra and High Sierra. They show signs of being flaky beyond the earliest run on the dashboard. Example build: https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r224405%20(5492)/results.html https://build.webkit.org/builders/Apple%20Sierra%20Release%20WK2%20(Tests)/builds/5492 History: https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Fwpt%2Fbeacon%2Fcontentextensions%2Fbeacon-blocked.html https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=http%2Fwpt%2Fbeacon%2Fcontentextensions%2Fbeacon-redirect-blocked.html http/wpt/beacon/contentextensions/beacon-blocked.html diff --- /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/http/wpt/beacon/contentextensions/beacon-blocked-expected.txt +++ /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/http/wpt/beacon/contentextensions/beacon-blocked-actual.txt @@ -1,5 +1,3 @@ -CONSOLE MESSAGE: line 9: Content blocker prevented frame displaying http://localhost:8800/WebKit/beacon/contentextensions/beacon-blocked.html from loading a resource from http://localhost:8800/WebKit/beacon/resources/beacon-preflight.py -CONSOLE MESSAGE: line 9: Beacon API cannot load http://localhost:8800/WebKit/beacon/resources/beacon-preflight.py. Resource blocked by content blocker -PASS Content extensions should be able to block beacons +FAIL Content extensions should be able to block beacons assert_false: sendBeacon should return false expected false got true http/wpt/beacon/contentextensions/beacon-redirect-blocked.html diff: --- /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/http/wpt/beacon/contentextensions/beacon-redirect-blocked-expected.txt +++ /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/http/wpt/beacon/contentextensions/beacon-redirect-blocked-actual.txt @@ -1,4 +1,3 @@ -CONSOLE MESSAGE: Beacon API cannot load http://127.0.0.1:8800/WebKit/beacon/resources/beacon-preflight.py?allowCors=1&cmd=put&id=f470f43c-258c-4c82-b880-ace3bcdb211c&redirect_status=307&location=http%3A%2F%2F127.0.0.1%3A8800%2FWebKit%2Fbeacon%2Fresources%2Fbeacon-preflight.py%3FallowCors%3D1%26cmd%3Dput%26id%3Df470f43c-258c-4c82-b880-ace3bcdb211c&count=1. Blocked by content extension -PASS Content extensions should be able to block beacon redirects +FAIL Content extensions should be able to block beacon redirects assert_equals: Did not receive beacon expected 0 but got 1
From stderr: 2017-11-03 13:37:34.083 WebKitTestRunner[89471:31874437] Rule list compilation failed: Unspecified error during compile.
<rdar://problem/35401978>
Marked tests as flaky in https://trac.webkit.org/r224690
This is apparently not specific to Beacon. All content extensions tests are currently flaky and sometimes give the following error on stderr: "Rule list compilation failed: Unspecified error during compile.".
(In reply to Chris Dumez from comment #4) > This is apparently not specific to Beacon. All content extensions tests are > currently flaky and sometimes give the following error on stderr: > "Rule list compilation failed: Unspecified error during compile.". Alex added more logging recently and on recent builds we see: Content Rule List compiling failed: Moving file failed. 2018-01-09 12:15:40.338 WebKitTestRunner[91903:69125462] Rule list compilation failed: Unspecified error during compile.
(In reply to Chris Dumez from comment #5) > (In reply to Chris Dumez from comment #4) > > This is apparently not specific to Beacon. All content extensions tests are > > currently flaky and sometimes give the following error on stderr: > > "Rule list compilation failed: Unspecified error during compile.". > > Alex added more logging recently and on recent builds we see: > Content Rule List compiling failed: Moving file failed. > 2018-01-09 12:15:40.338 WebKitTestRunner[91903:69125462] Rule list > compilation failed: Unspecified error during compile. I also see: 2018-01-08 18:34:52.233 WebKitTestRunner[40704:65555369] Error in moveFile: Error Domain=NSCocoaErrorDomain Code=4 "“ContentRuleListSTFKD0” couldn’t be moved to “ContentExtensions” because either the former doesn't exist, or the folder containing the latter doesn't exist." UserInfo={NSSourceFilePathErrorKey=/var/folders/sz/cwvl9y7149bc1xwf4f_x1_k00000gn/T/WebKitTestRunners-pkI04D/ContentRuleListSTFKD0, NSUserStringVariant=( Move ), NSDestinationFilePath=/Users/buildbot/Library/WebKit/WebKitTestRunner/ContentExtensions/ContentExtension-TestContentExtensions, NSFilePath=/var/folders/sz/cwvl9y7149bc1xwf4f_x1_k00000gn/T/WebKitTestRunners-pkI04D/ContentRuleListSTFKD0, NSUnderlyingError=0x7fddf7074c80 {Error Domain=NSPOSIXErrorDomain Code=2 "No such file or directory"}}
(In reply to Chris Dumez from comment #6) > (In reply to Chris Dumez from comment #5) > > (In reply to Chris Dumez from comment #4) > > > This is apparently not specific to Beacon. All content extensions tests are > > > currently flaky and sometimes give the following error on stderr: > > > "Rule list compilation failed: Unspecified error during compile.". > > > > Alex added more logging recently and on recent builds we see: > > Content Rule List compiling failed: Moving file failed. > > 2018-01-09 12:15:40.338 WebKitTestRunner[91903:69125462] Rule list > > compilation failed: Unspecified error during compile. > > I also see: > 2018-01-08 18:34:52.233 WebKitTestRunner[40704:65555369] Error in moveFile: > Error Domain=NSCocoaErrorDomain Code=4 "“ContentRuleListSTFKD0” couldn’t be > moved to “ContentExtensions” because either the former doesn't exist, or the > folder containing the latter doesn't exist." > UserInfo={NSSourceFilePathErrorKey=/var/folders/sz/ > cwvl9y7149bc1xwf4f_x1_k00000gn/T/WebKitTestRunners-pkI04D/ > ContentRuleListSTFKD0, NSUserStringVariant=( > Move > ), > NSDestinationFilePath=/Users/buildbot/Library/WebKit/WebKitTestRunner/ > ContentExtensions/ContentExtension-TestContentExtensions, > NSFilePath=/var/folders/sz/cwvl9y7149bc1xwf4f_x1_k00000gn/T/ > WebKitTestRunners-pkI04D/ContentRuleListSTFKD0, > NSUnderlyingError=0x7fddf7074c80 {Error Domain=NSPOSIXErrorDomain Code=2 "No > such file or directory"}} TestController::platformConfigureViewForTest(const TestInvocation& test) uses [_WKUserContentExtensionStore defaultStore] which apparently ends up using the default store path, which ends up being shared by all WKTR instances that are running in parallel :(
Created attachment 330922 [details] Patch
Comment on attachment 330922 [details] Patch Attachment 330922 [details] did not pass mac-wk2-ews (mac-wk2): Output: http://webkit-queues.webkit.org/results/6023313 New failing tests: http/tests/workers/service/postmessage-after-sw-process-crash.https.html
Created attachment 330941 [details] Archive of layout-test-results from ews106 for mac-sierra-wk2 The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews. Bot: ews106 Port: mac-sierra-wk2 Platform: Mac OS X 10.12.6
Comment on attachment 330941 [details] Archive of layout-test-results from ews106 for mac-sierra-wk2 This failure cannot be related.
Comment on attachment 330922 [details] Patch Clearing flags on attachment: 330922 Committed r226751: <https://trac.webkit.org/changeset/226751>
All reviewed patches have been landed. Closing bug.