STP140 audio/video issues

• Component: WebRTC
• Status: RESOLVED DUPLICATE

[Gavin Lefebvre]

WebRTC audio (and some video) is unusable in Safari Technology Preview 140:

- Local audio MediaStreamTrack from Intel MacBook Pro Microphone is generally not delivered to remote peer, only occasional, discontinuous chunks of audio make it through (unintelligible)
- Also local microphone audio level is generally NOT detected by WebAudio AnalyserNode in STP
- However using Mock Capture Device setting in STP DOES deliver audio (and is detected by AnalyserNode)
- Under this setting, Mock Capture Video is only able to deliver 5-6 fps, even when just viewed locally (???)
- Remote audio MediaStreamTrack OCCASIONALLY is detected by AnalyserNode in STP, but RARELY creates any output (and when it does, it is essentially just "clicks")

Looks like buffer under-runs in console (verbose WebRTC Logs enabled)

LocalAudio (consistently 7 samples short):

[Error] AudioSampleDataSource::pullSamples(61979E3D82774AFC) need more data, sample 1429952 with offset 1034944, trying to get 128 samples, but not completely in range [1430455 .. 1495991]
[Log] AudioSampleDataSource::pushSamplesInternal(61979E3D82774AFC) needed more data, pushing 121 samples

RemoteAudio (consistently 24 samples short):

[Error] AudioSampleDataSource::pullSamples(14665F5E17826BDE) need more data, sample 1258664 with offset 968744, trying to get 480 samples, but not completely in range [1260216 .. 1292984]
[Log] AudioSampleDataSource::pushSamplesInternal(14665F5E17826BDE) needed more data, pushing 456 samples
[Info] RealtimeIncomingVideoSource::videoSampleAvailable(8DF9FB82FB5AD353) 24 frames sent in 1.0015237659999912 seconds
[Info] RealtimeVideoCaptureSource::videoSampleAvailable(F9D25F3356E64DF1) 26 frames sent in 1.0576630170003227 seconds
[Info] RealtimeMediaSource::videoSampleAvailable(14A82F22ADAE606E) 26 frames sent in 1.0577196350000122 seconds
[Info] RealtimeOutgoingVideoSource::sendFrame(F9D25F3356E64DF1) 26 frames sent in 1.0577254170002561 seconds
[Error] AudioSampleDataSource::pullSamples(14665F5E17826BDE) need more data, sample 1301984 with offset 1002464, trying to get 480 samples, but not completely in range [1303536 .. 1336304]
[Log] AudioSampleDataSource::pushSamplesInternal(14665F5E17826BDE) needed more data, pushing 456 samples

I have not such issues on the same machine / hardware (even using the Mock Capture Devices) on Safari Version 15.3 (17612.4.9.1.8)

We've had reports from a particular customer of similar behavior where one party was connected via Safari (though doubtful STP). Is this a chipset incompatibility (i.e. 15.3 works on Intel but not Apple, and STP140/15.4 works on Apple but not Intel)?

[youenn fablet]

Hi Gabin, do you have a repro case?
https://webrtc.github.io/samples/src/content/peerconnection/webaudio-input/ seems to work for me locally on STP140.

[Gavin Lefebvre]

I was having issues with that site as well- testing it just now:

- the first attempt, I got `A MediaStreamTrack ended due to a capture failure
onIceCandidate — main.js:129` just after the SDP exchange
- the second attempt, the MediaStreamTrack seemed to survive negotiation, but I soon received the following verbose logs:

[Log] IncomingAudioMediaStreamTrackRendererUnit::addSource (EFB295B9B2612C0A)
[Log] AudioMediaStreamTrackRendererUnit::addSource (3BEF086C56522B09)
[Error] AudioSampleDataSource::pullSamples(3BEF086C56522B09) need more data, sample 11907 with offset 882, trying to get 441 samples, but not completely in range [13301 .. 46069]
[Log] AudioSampleDataSource::pushSamplesInternal(3BEF086C56522B09) needed more data, pushing 419 samples
...
[Log] RealtimeIncomingAudioSource::logTimerFired(EFB295B9B2612C0A) chunk 197
[Log] RealtimeIncomingAudioSource::logTimerFired(EFB295B9B2612C0A) new audio buffer list for sampleRate 16000 and 1 channel(s)
...
[Error] AudioSampleDataSource::pullSamples(3BEF086C56522B09) need more data, sample 56102 with offset 34493, trying to get 441 samples, but not completely in range [57291 .. 90059]
[Log] AudioSampleDataSource::pushSamplesInternal(3BEF086C56522B09) needed more data, pushing 419 samples
[Error] AudioSampleDataSource::pullSamples(3BEF086C56522B09) need more data, sample 100532 with offset 67898, trying to get 441 samples, but not completely in range [101501 .. 134269]
[Log] AudioSampleDataSource::pushSamplesInternal(3BEF086C56522B09) needed more data, pushing 419 samples
...


Main difference between here and my initial post is that I changed my system audio config to 44.1kHz from 48kHz.

We will email you a repro link to test with our live app, thanks!

[youenn fablet]

Is it similar to https://bugs.webkit.org/show_bug.cgi?id=236779?
Are you using a BT headset?

[Gavin Lefebvre]

Created attachment 452876 [details]
Audio Midi Config

No BT devices attached, and confirmed that STP140 is grabbing the MacBook Pro Microphone as the MST source in the WebRTC demo.

Best guess is that the presence of Aggregate Devices (even though they're not used) could be causing SRC confusion?

[Radar WebKit Bug Importer]

<rdar://problem/89382486>

[Brent Fulgham]

*** This bug has been marked as a duplicate of bug 239513 ***