225665 2021-05-11 11:21:32 -0700 [ BigSur Release wk2 arm64 & iOS] http/tests/appcache/fail-on-update-2.html (layout-test) is a flaky timeout 2022-02-15 17:19:27 -0800 1 1 1 Unclassified WebKit New Bugs WebKit Nightly Build Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=236680 InRadar P2 Normal --- 1 jenner webkit-unassigned ayumi_kojima ehutchison webkit-bot-watchers-bugzilla webkit-bug-importer oldest_to_newest 1759001 0 jenner 2021-05-11 11:21:32 -0700 http/tests/appcache/fail-on-update-2.html is a flaky timeout on BigSur Release wk2 on Apple Silicon Macs only. HISTORY: https://results.webkit.org/?suite=layout-tests&test=http%2Ftests%2Fappcache%2Ffail-on-update-2.html TIMEOUT TEXT: -SUCCESS: No crash. +#PID UNRESPONSIVE - WebKitTestRunner (pid 21811) +FAIL: Timed out waiting for notifyDone to be called + +#EOF +#EOF STDERR TEXT: CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. CONSOLE MESSAGE: Application Cache manifest could not be fetched, because the manifest had a 404 response. CONSOLE MESSAGE: ApplicationCache is deprecated. Please use ServiceWorkers instead. Wait on notifyDone timed out, process WebKitTestRunner pid = 21811#EOF 1759049 1 jenner 2021-05-11 13:46:26 -0700 There were some consistent timeouts on Intel Macs. But they stopped, and haven't occurred for over a month. I did attempt to reproduce it at BigSur Release ToT wk1, but was unable to. It doesn't appear this test is having a problem on Intel anymore, only Apple Silicon. Since this appears to only occur on Apple Silicon, I can't reproduce the timeout, as I don't have access to said system type. I have gone ahead and updated the test expectations to Pass Timeout here: https://trac.webkit.org/changeset/277339/webkit 1759052 2 webkit-bug-importer 2021-05-11 13:52:21 -0700 <rdar://problem/77861895> 1759466 3 jenner 2021-05-12 17:05:19 -0700 As it turns out, there is an expectation set for this test in: sources/OpenSource/LayoutTests/TestExpectations The expectation is set as: [ DumpJSConsoleLogInStdErr ] That expectation is set to hide the deprecation error from the stdout, and puts it as a stderr instead. When I updated my expectation it over-rode that and caused constant text failures. Updated test expectations to DumpJSConsoleLogInStdErr Slow here: https://trac.webkit.org/changeset/277410/webkit This should stop the constant text failures, and help confirm if these timeouts are actual timeouts, or just a test being slow. 1759935 4 jenner 2021-05-13 17:49:51 -0700 After marking the test is slow, it appears to have actually gotten significantly faster. Before marking as slow, the test would take anywhere between 3 to 19 seconds to pass, and would timeout at roughly one minute. Since marking as slow, the test has been passing only taking between .86 seconds and 3.26 seconds for the test to run and pass. I know we also were looking at the ram usage on this particular Mac mini, and found at the time that it was using approx. 10 gigs of its ram. We have addressed this issue, and the test is no longer timing out. 1778438 5 ayumi_kojima 2021-07-21 09:18:53 -0700 The same flaky timeout is seen on iOS 14 E Simulator wk2. I was not able to reproduce the timeout on simulator (iOS 15) on my local machine using run-webkit-tests --ios-simulator --iterations 500 --exit-after-n-crashes-or-timeouts 1 --exit-after-n-failures 1 http/tests/appcache/fail-on-update-2.html -f --release --no-build --clobber-old-results The flaky timeout on iOS is seen since 5/6/21 (No test results are available on iOS before 5/6/21) 1778467 6 ayumi_kojima 2021-07-21 10:32:42 -0700 Updated test expectations for ios-wk2 https://trac.webkit.org/changeset/280148/webkit 1811888 7 ehutchison 2021-11-04 09:47:19 -0700 Test is still flaking on iOS15; marking as timeout. https://trac.webkit.org/changeset/285266/webkit