| Summary: | [ BigSur Release wk2 arm64 & iOS] http/tests/appcache/fail-on-update-2.html (layout-test) is a flaky timeout | ||
|---|---|---|---|
| Product: | WebKit | Reporter: | Robert Jenner <jenner> |
| Component: | New Bugs | Assignee: | Nobody <webkit-unassigned> |
| Status: | NEW --- | ||
| Severity: | Normal | CC: | ayumi_kojima, ehutchison, webkit-bot-watchers-bugzilla, webkit-bug-importer |
| Priority: | P2 | Keywords: | InRadar |
| Version: | WebKit Nightly Build | ||
| Hardware: | Unspecified | ||
| OS: | Unspecified | ||
| See Also: | https://bugs.webkit.org/show_bug.cgi?id=236680 | ||
|
Description
Robert Jenner
2021-05-11 11:21:32 PDT
There were some consistent timeouts on Intel Macs. But they stopped, and haven't occurred for over a month. I did attempt to reproduce it at BigSur Release ToT wk1, but was unable to. It doesn't appear this test is having a problem on Intel anymore, only Apple Silicon. Since this appears to only occur on Apple Silicon, I can't reproduce the timeout, as I don't have access to said system type. I have gone ahead and updated the test expectations to Pass Timeout here: https://trac.webkit.org/changeset/277339/webkit As it turns out, there is an expectation set for this test in: sources/OpenSource/LayoutTests/TestExpectations The expectation is set as: [ DumpJSConsoleLogInStdErr ] That expectation is set to hide the deprecation error from the stdout, and puts it as a stderr instead. When I updated my expectation it over-rode that and caused constant text failures. Updated test expectations to DumpJSConsoleLogInStdErr Slow here: https://trac.webkit.org/changeset/277410/webkit This should stop the constant text failures, and help confirm if these timeouts are actual timeouts, or just a test being slow. After marking the test is slow, it appears to have actually gotten significantly faster. Before marking as slow, the test would take anywhere between 3 to 19 seconds to pass, and would timeout at roughly one minute. Since marking as slow, the test has been passing only taking between .86 seconds and 3.26 seconds for the test to run and pass. I know we also were looking at the ram usage on this particular Mac mini, and found at the time that it was using approx. 10 gigs of its ram. We have addressed this issue, and the test is no longer timing out. The same flaky timeout is seen on iOS 14 E Simulator wk2. I was not able to reproduce the timeout on simulator (iOS 15) on my local machine using run-webkit-tests --ios-simulator --iterations 500 --exit-after-n-crashes-or-timeouts 1 --exit-after-n-failures 1 http/tests/appcache/fail-on-update-2.html -f --release --no-build --clobber-old-results The flaky timeout on iOS is seen since 5/6/21 (No test results are available on iOS before 5/6/21) Updated test expectations for ios-wk2 https://trac.webkit.org/changeset/280148/webkit Test is still flaking on iOS15; marking as timeout. https://trac.webkit.org/changeset/285266/webkit |