Bug 230950

Summary:

Web Audio panner node quality deteriorates over time

Product:

WebKit

Reporter:

Ashley Gullen <ashley>

Component:

Web Audio

Assignee:

Chris Dumez <cdumez>

Status:

RESOLVED FIXED

Severity:

Normal

CC:

cdumez, eric.carlson, ews-watchlist, glenn, jer.noble, kevin_neal, philipj, sergio, webkit-bug-importer

Priority:

Keywords:

InRadar

Version:

Other

Hardware:

Unspecified

OS:

Unspecified

See Also:

https://bugs.webkit.org/show_bug.cgi?id=230966
https://bugs.webkit.org/show_bug.cgi?id=227199
https://bugs.webkit.org/show_bug.cgi?id=231314

Attachments:

Description	Flags
Patch	none

Description Ashley Gullen 2021-09-29 06:00:18 PDT

Repro URL: https://downloads.scirra.com/labs/bugs/safaripannerquality/

Steps to reproduce: Load the URL, wait for it to finish loading, then when instructed tap the screen to initiate audio playback.
Keep the screen on (e.g. by occasionally touching the screen) and keep the audio playback going for ~5 minutes (when the displayed time reaches ~300 seconds).

Observed result: after some time, the audio quality noticeably deteriorates. This reproduces for me on an iPad Pro with Safari 15, and becomes obvious around the 300 second mark, with clear buzzing/distortion sounds around that time.

Expected result: audio playback quality to remain good throughout.

The demo uses the Web Audio AP, with panner nodes and a listener set up for positioned sounds. (You can hear that each sound is panned to match the location of the square that appears on screen simultaneously.) It appears the audio deterioration is specific to panner nodes.

This demo is made in Construct 3 and the issue was originally reported to us by a Construct 3 user: https://github.com/Scirra/Construct-3-bugs/issues/5063
They state that it did not reproduce in iOS 12, but does in iOS 14.7.1. This means it appears to be a regression. Perhaps this is related to the new standards-complaint non-prefixed Web Audio API that was new in Safari 14.1.

Comment 1 Chris Dumez 2021-09-29 08:10:25 PDT

I see a large number of AudioBufferSourceNodes getting allocated and seemingly not be deallocated. WebKit may be leaking them.

Comment 2 Chris Dumez 2021-09-29 09:11:19 PDT

I don't see the JS wrappers from this AudioBufferSourceNodes getting garbage collected on JS side. Is the script holding on to these AudioBufferSourceNodes somehow?

These nodes have an internal buffer so they can use a lot of memory.

Comment 3 Ashley Gullen 2021-09-29 09:43:37 PDT

I don't believe the script is leaking anything. We do have wrapper objects that represent a playback, and the AudioBufferSourceNodes is held in the wrapper, and the wrapper objects get reused (from a quick debug it looks like it creates about 11 and then cycles through them). But both a new playback and the onended handlers disconnect the AudioBufferSourceNode and null it out. So old nodes should be disconnected and garbaged.

AFAIK this issue only affects Safari - other browsers seem to handle it OK.

Comment 4 Radar WebKit Bug Importer 2021-09-29 10:00:51 PDT

<rdar://problem/83675934>

Comment 5 Chris Dumez 2021-09-29 17:30:24 PDT

I was wrong. This seems to be related to PannerNode automation. PannerNode processing takes longer and longer because events accumulate in the PannerNode's AudioParam's timelines.

I'll look into this. I bet we need to add logic to AudioParamTimeline to clean up events.

Comment 6 Chris Dumez 2021-09-30 08:03:51 PDT

Created attachment 439738 [details]
Patch

Comment 7 EWS 2021-09-30 12:23:51 PDT

Committed r283322 (242347@main): <https://commits.webkit.org/242347@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 439738 [details].