Bug 172201 - [macOS WK2] LayoutTest webaudio/silent-audio-interrupted-in-background.html is a flaky timeout on older OSes (El Capitan / Sierra)
Summary: [macOS WK2] LayoutTest webaudio/silent-audio-interrupted-in-background.html i...
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: New Bugs (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2017-05-16 17:16 PDT by Matt Lewis
Modified: 2019-09-23 10:32 PDT (History)
5 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Matt Lewis 2017-05-16 17:16:26 PDT
The test webaudio/silent-audio-interrupted-in-background.html is a flaky timeout on macOS wk2.

The test was last edited in r214721
https://trac.webkit.org/changeset/214721/webkit

https://build.webkit.org/builders/Apple%20Sierra%20Release%20WK2%20(Tests)/builds/1525
https://build.webkit.org/results/Apple%20Sierra%20Release%20WK2%20(Tests)/r216951%20(1525)/results.html

https://webkit-test-results.webkit.org/dashboards/flakiness_dashboard.html#showAllRuns=true&tests=webaudio%2Fsilent-audio-interrupted-in-background.html

diff:
--- /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/webaudio/silent-audio-interrupted-in-background-expected.txt
+++ /Volumes/Data/slave/sierra-release-tests-wk2/build/layout-test-results/webaudio/silent-audio-interrupted-in-background-actual.txt
@@ -1,14 +1,5 @@
-Tests that silent WebAudio rendering gets interrupted in hidden pages.
+#PID UNRESPONSIVE - com.apple.WebKit.WebContent.Development (pid 95499)
+FAIL: Timed out waiting for notifyDone to be called
 
-On success, you will see a series of "PASS" messages, followed by "TEST COMPLETE".
-
-
-PASS context.state became 'running'
-* Setting page visibility to hidden
-PASS context.state became 'interrupted'
-* Setting page visibility to visible
-PASS context.state became 'running'
-PASS successfullyParsed is true
-
-TEST COMPLETE
-
+#EOF
+#EOF
Comment 1 Matt Lewis 2017-05-16 17:24:37 PDT
I was able to reproduce the timeout on Sierra Release with:

run-webkit-tests webaudio/silent-audio-interrupted-in-background.html -f --iterations=100 --no-retry-failure -g

It is flaky so it did occasionally pass all test.
Comment 2 Alexey Proskuryakov 2017-05-16 19:59:42 PDT
+#PID UNRESPONSIVE - com.apple.WebKit.WebContent.Development (pid 95499)

I believe that this error means that the issue is caused by the preceding test. Perhaps both this and preceding test can cause this?

Definitely seems worth a deeper look.
Comment 3 Alexey Proskuryakov 2017-05-16 20:04:09 PDT
Sorry, I misread. "Timed out waiting for notifyDone to be called" is normally about the current test.
Comment 4 Alexey Proskuryakov 2017-05-16 20:07:12 PDT
This test is normally very fast, but times out fairly frequently.
Comment 5 Chris Dumez 2017-05-16 20:13:08 PDT
I will take a look soon.
Comment 6 Radar WebKit Bug Importer 2017-05-16 20:16:13 PDT
<rdar://problem/32239545>
Comment 7 Chris Dumez 2017-06-01 13:29:53 PDT
(In reply to Matt Lewis from comment #1)
> I was able to reproduce the timeout on Sierra Release with:
> 
> run-webkit-tests webaudio/silent-audio-interrupted-in-background.html -f
> --iterations=100 --no-retry-failure -g
> 
> It is flaky so it did occasionally pass all test.

I have tried this repeatedly:
Tools/Scripts/run-webkit-tests webaudio/silent-audio-interrupted-in-background.html -f --iterations=1000 --no-retry-failure -g

So far, no failures though :/
Comment 8 Chris Dumez 2017-06-01 13:49:04 PDT
(In reply to Chris Dumez from comment #7)
> (In reply to Matt Lewis from comment #1)
> > I was able to reproduce the timeout on Sierra Release with:
> > 
> > run-webkit-tests webaudio/silent-audio-interrupted-in-background.html -f
> > --iterations=100 --no-retry-failure -g
> > 
> > It is flaky so it did occasionally pass all test.
> 
> I have tried this repeatedly:
> Tools/Scripts/run-webkit-tests
> webaudio/silent-audio-interrupted-in-background.html -f --iterations=1000
> --no-retry-failure -g
> 
> So far, no failures though :/

Looks like I have managed to reproduce with:
Tools/Scripts/run-webkit-tests webaudio/silent-audio-interrupted-in-background.html -f --iterations=1000 --no-retry-failure -g --child-processes=50
Comment 9 Chris Dumez 2017-06-01 14:43:24 PDT
This is somehow related to page visibility. The onvisibilitychange event handler on document sometimes does not get called after calling testRunner.setPageVisibility("hidden").
Comment 10 Chris Dumez 2017-06-01 16:33:05 PDT
(In reply to Chris Dumez from comment #9)
> This is somehow related to page visibility. The onvisibilitychange event
> handler on document sometimes does not get called after calling
> testRunner.setPageVisibility("hidden").

When setting the page visibility to hidden, we end up calling:
void TestController::setHidden(bool hidden)
{
    NSWindow *window = [mainWebView()->platformView() window];
    if (!window)
        return;

    if (hidden)
        [window orderOut:nil];
    else
        [window makeKeyAndOrderFront:nil];
}
Comment 11 Ryan Haddad 2017-06-22 11:06:06 PDT
Marked test as flaky in http://trac.webkit.org/projects/webkit/changeset/218706
Comment 12 Alexey Proskuryakov 2019-09-23 10:32:27 PDT
It continues to be a flaky timeout on new OSes.