Bug 230950 - Web Audio panner node quality deteriorates over time
Summary: Web Audio panner node quality deteriorates over time
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Web Audio (show other bugs)
Version: Other
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Chris Dumez
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-09-29 06:00 PDT by Ashley Gullen
Modified: 2021-10-06 11:15 PDT (History)
9 users (show)

See Also:


Attachments
Patch (4.87 KB, patch)
2021-09-30 08:03 PDT, Chris Dumez
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Ashley Gullen 2021-09-29 06:00:18 PDT
Repro URL: https://downloads.scirra.com/labs/bugs/safaripannerquality/

Steps to reproduce: Load the URL, wait for it to finish loading, then when instructed tap the screen to initiate audio playback.
Keep the screen on (e.g. by occasionally touching the screen) and keep the audio playback going for ~5 minutes (when the displayed time reaches ~300 seconds).

Observed result: after some time, the audio quality noticeably deteriorates. This reproduces for me on an iPad Pro with Safari 15, and becomes obvious around the 300 second mark, with clear buzzing/distortion sounds around that time.

Expected result: audio playback quality to remain good throughout.

The demo uses the Web Audio AP, with panner nodes and a listener set up for positioned sounds. (You can hear that each sound is panned to match the location of the square that appears on screen simultaneously.) It appears the audio deterioration is specific to panner nodes.

This demo is made in Construct 3 and the issue was originally reported to us by a Construct 3 user: https://github.com/Scirra/Construct-3-bugs/issues/5063
They state that it did not reproduce in iOS 12, but does in iOS 14.7.1. This means it appears to be a regression. Perhaps this is related to the new standards-complaint non-prefixed Web Audio API that was new in Safari 14.1.
Comment 1 Chris Dumez 2021-09-29 08:10:25 PDT
I see a large number of AudioBufferSourceNodes getting allocated and seemingly not be deallocated. WebKit may be leaking them.
Comment 2 Chris Dumez 2021-09-29 09:11:19 PDT
I don't see the JS wrappers from this AudioBufferSourceNodes getting garbage collected on JS side. Is the script holding on to these AudioBufferSourceNodes somehow?

These nodes have an internal buffer so they can use a lot of memory.
Comment 3 Ashley Gullen 2021-09-29 09:43:37 PDT
I don't believe the script is leaking anything. We do have wrapper objects that represent a playback, and the AudioBufferSourceNodes is held in the wrapper, and the wrapper objects get reused (from a quick debug it looks like it creates about 11 and then cycles through them). But both a new playback and the onended handlers disconnect the AudioBufferSourceNode and null it out. So old nodes should be disconnected and garbaged.

AFAIK this issue only affects Safari - other browsers seem to handle it OK.
Comment 4 Radar WebKit Bug Importer 2021-09-29 10:00:51 PDT
<rdar://problem/83675934>
Comment 5 Chris Dumez 2021-09-29 17:30:24 PDT
I was wrong. This seems to be related to PannerNode automation. PannerNode processing takes longer and longer because events accumulate in the PannerNode's AudioParam's timelines.

I'll look into this. I bet we need to add logic to AudioParamTimeline to clean up events.
Comment 6 Chris Dumez 2021-09-30 08:03:51 PDT
Created attachment 439738 [details]
Patch
Comment 7 EWS 2021-09-30 12:23:51 PDT
Committed r283322 (242347@main): <https://commits.webkit.org/242347@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 439738 [details].