Bug 279432 – MediaRecorder generates huge chunks when pausing & resuming camera access on macOS and iOS

• Status: NEW
• Component: Media

[octavn]

1. Using Safari on macOS, go to https://addpipe.com/media-recorder-api-demo/ and click "Allow" when asked for camera & mic access
2. Start recording, you'll see each chunk size and the associated timecode logged on the page
3. Now click on the red camera icon in the browser's URL input bar and click "Pause", you'll see the size drop to 0 while the timecode remains unchanged (as expected)
4. After a few seconds, using the same red camera icon, click Resume. For a few ondataavailable calls, the size is will be way larger than usual. After a while the size value comes back down.

Here's a recent log where the chunk size reaches 6.59MB after Resuming instead of the usual 1-1.3MB
----
Start recording...
Using video/mp4
audio:{"deviceId":"0746899703E64653F52874987F3BD33569CC892A","echoCancellation":true,"groupId":"E54695174C70D97DC34C99D75601F73794ADC3D3","sampleRate":16000,"volume":1}
video:{"aspectRatio":1.7777777777777777,"deviceId":"D188CC318FEB99436764CF524348EDF6F25E8A0F","frameRate":30,"groupId":"7A20AE9C1F2F7B4515E48F13942DDB78BF529458","height":720,"width":1280}
mediaRecorder.onstart, mediaRecorder.state = recording
onstart - Audio track.readyState=live, track.muted=false
onstart - Video track.readyState=live, track.muted=false
mediaRecorder.ondataavailable, e.data.size=650491, e.timecode=0
mediaRecorder.ondataavailable, e.data.size=976849, e.timecode=1.024
mediaRecorder.ondataavailable, e.data.size=973571, e.timecode=2.016
mediaRecorder.ondataavailable, e.data.size=1222025, e.timecode=3.04
mediaRecorder.ondataavailable, e.data.size=1215702, e.timecode=4.032
mediaRecorder.ondataavailable, e.data.size=1355689, e.timecode=5.056
mediaRecorder.ondataavailable, e.data.size=1177430, e.timecode=6.048
mediaRecorder.ondataavailable, e.data.size=1326199, e.timecode=7.04
mediaRecorder.ondataavailable, e.data.size=1228857, e.timecode=8.064
mediaRecorder.ondataavailable, e.data.size=1627209, e.timecode=9.056
mediaRecorder.ondataavailable, e.data.size=1339671, e.timecode=10.08
mediaRecorder.ondataavailable, e.data.size=164405, e.timecode=11.072
mediaRecorder.ondataavailable, e.data.size=0, e.timecode=11.2
mediaRecorder.ondataavailable, e.data.size=0, e.timecode=11.2
mediaRecorder.ondataavailable, e.data.size=0, e.timecode=11.2
mediaRecorder.ondataavailable, e.data.size=0, e.timecode=11.2
mediaRecorder.ondataavailable, e.data.size=0, e.timecode=11.2
mediaRecorder.ondataavailable, e.data.size=0, e.timecode=11.2
mediaRecorder.ondataavailable, e.data.size=355, e.timecode=11.2
mediaRecorder.ondataavailable, e.data.size=3890620, e.timecode=11.264
mediaRecorder.ondataavailable, e.data.size=6592507, e.timecode=12.288
mediaRecorder.ondataavailable, e.data.size=4531837, e.timecode=13.28
mediaRecorder.ondataavailable, e.data.size=2248763, e.timecode=14.304
mediaRecorder.ondataavailable, e.data.size=1537077, e.timecode=15.32
mediaRecorder.ondataavailable, e.data.size=980034, e.timecode=16.344
mediaRecorder.ondataavailable, e.data.size=1109501, e.timecode=17.344
mediaRecorder.ondataavailable, e.data.size=1536439, e.timecode=18.336
mediaRecorder.ondataavailable, e.data.size=1271784, e.timecode=19.36
mediaRecorder.ondataavailable, e.data.size=1079751, e.timecode=20.352
mediaRecorder.ondataavailable, e.data.size=1220153, e.timecode=21.376
mediaRecorder.ondataavailable, e.data.size=962462, e.timecode=0
mediaRecorder.onstop, mediaRecorder.state = inactive
---

The same effect is observed on Safari on macOS by using the grey camera icon that shows up in the browser page tab (with multiple tabs open).

The same effect is observed on Safari on iOS by using the "Stop Using Camera" and "Start Using Camera" options.

Tested with Safari 17.6 on macOS Sonoma 14.6.1 and Safari on iOS 17.6.1 .

[Radar WebKit Bug Importer]

<rdar://problem/136134334>

[octavn]

Here's another log which includes the `onmute` and `onunmute` events. The final .mp4 recording was 53277239 bytes in size which is the sum of all chunks including the larger ones.


```
2024-09-27T11:56:17.137Z Start recording...
2024-09-27T11:56:17.137Z Using video/mp4
2024-09-27T11:56:17.138Z audio:{"deviceId":"4370A5A545A85F31A3C7B8ACD8166B6CD916E00B","echoCancellation":true,"groupId":"08B91934E4B5EC23217E8F7C9B2ED1C2C3413DDF","sampleRate":48000,"volume":1}
2024-09-27T11:56:17.138Z video:{"aspectRatio":1.7777777777777777,"deviceId":"A176E28C74D4A44566239B973AEA27A0382E1EB5","frameRate":30,"groupId":"B87E91C05B2542B28532739EAFA3B2303DE1EC39","height":720,"width":1280}
2024-09-27T11:56:17.141Z mediaRecorder.onstart, mediaRecorder.state = recording
2024-09-27T11:56:17.142Z onstart - Audio track.readyState=live, track.muted=false
2024-09-27T11:56:17.142Z onstart - Video track.readyState=live, track.muted=false
2024-09-27T11:56:18.143Z mediaRecorder.ondataavailable, e.data.size=1185745, e.timecode=0
2024-09-27T11:56:19.149Z mediaRecorder.ondataavailable, e.data.size=1307271, e.timecode=1.0026666666666666
2024-09-27T11:56:20.156Z mediaRecorder.ondataavailable, e.data.size=1340063, e.timecode=2.005333333333333
2024-09-27T11:56:21.160Z mediaRecorder.ondataavailable, e.data.size=1288266, e.timecode=3.008
2024-09-27T11:56:22.168Z mediaRecorder.ondataavailable, e.data.size=1292348, e.timecode=4.010666666666666
2024-09-27T11:56:22.708Z audio track.onmute track.readyState=live, track.muted=true
2024-09-27T11:56:22.708Z video track.onmute track.readyState=live, track.muted=true
2024-09-27T11:56:23.175Z mediaRecorder.ondataavailable, e.data.size=688381, e.timecode=5.013333333333334
2024-09-27T11:56:24.176Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:25.178Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:26.180Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:27.181Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:28.183Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:29.186Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:30.188Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:31.190Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:32.192Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:33.194Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:34.195Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:35.198Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:36.200Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:37.201Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:38.204Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:39.205Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:40.207Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:41.208Z mediaRecorder.ondataavailable, e.data.size=0, e.timecode=5.568
2024-09-27T11:56:44.573Z audio track.onunmute track.readyState=live, track.muted=false
2024-09-27T11:56:44.574Z video track.onunmute track.readyState=live, track.muted=false
2024-09-27T11:56:44.601Z mediaRecorder.ondataavailable, e.data.size=18126, e.timecode=5.568
2024-09-27T11:56:45.616Z mediaRecorder.ondataavailable, e.data.size=4429680, e.timecode=6.810666666666667
2024-09-27T11:56:46.632Z mediaRecorder.ondataavailable, e.data.size=5074715, e.timecode=7.824
2024-09-27T11:56:47.646Z mediaRecorder.ondataavailable, e.data.size=4964964, e.timecode=8.837333333333333
2024-09-27T11:56:48.664Z mediaRecorder.ondataavailable, e.data.size=5486805, e.timecode=9.856
2024-09-27T11:56:49.672Z mediaRecorder.ondataavailable, e.data.size=2706574, e.timecode=10.869333333333334
2024-09-27T11:56:50.684Z mediaRecorder.ondataavailable, e.data.size=2539160, e.timecode=11.882666666666667
2024-09-27T11:56:51.696Z mediaRecorder.ondataavailable, e.data.size=2722593, e.timecode=12.890666666666666
2024-09-27T11:56:52.706Z mediaRecorder.ondataavailable, e.data.size=2515794, e.timecode=13.904
2024-09-27T11:56:53.720Z mediaRecorder.ondataavailable, e.data.size=2601201, e.timecode=14.917333333333334
2024-09-27T11:56:54.727Z mediaRecorder.ondataavailable, e.data.size=2631340, e.timecode=15.925333333333333
2024-09-27T11:56:55.737Z mediaRecorder.ondataavailable, e.data.size=2634556, e.timecode=16.933333333333334
2024-09-27T11:56:56.747Z mediaRecorder.ondataavailable, e.data.size=2213167, e.timecode=17.946666666666665
2024-09-27T11:56:57.763Z mediaRecorder.ondataavailable, e.data.size=1779012, e.timecode=18.954666666666668
2024-09-27T11:56:58.773Z mediaRecorder.ondataavailable, e.data.size=1915310, e.timecode=19.973333333333333
2024-09-27T11:56:59.777Z mediaRecorder.ondataavailable, e.data.size=1305331, e.timecode=20.981333333333332
2024-09-27T11:57:00.303Z mediaRecorder.ondataavailable, e.data.size=636837, e.timecode=0
2024-09-27T11:57:00.303Z mediaRecorder.onstop, mediaRecorder.state = inactive
```

ffprobe might reveal what's in those big chunks.

[octavn]

A quick inspection with `ffprobe -show_frames video.mp4` revealed a continuous stream of audio frames during the *paused* period but no video frames.

[youenn fablet]

Thanks for the investigation.

This is indeed an issue, WebKit::MediaRecorderPrivate::audioSamplesAvailable is handling muting locally and should actually notify GPUProcess about that.

[youenn fablet]

Ditto with MediaRecorderPrivateAVFImpl::audioSamplesAvailable