Bug 216832 – REGRESSION (iOS/Safari 14): MediaRecorder produces invalid video files

• Component: WebRTC
• Status: RESOLVED FIXED

[AndyDeveloper]

Since iOS 14 and Safari 14 on MacOS. Most media players can't play the recorded files. It seems the metadata (duration, frame rate...) are not set properly. If I re-encode the file assuming the frame rate, then I get a valid file.

Steps to reproduce:
1. Record the camera stream using the getUserMedia and MediaRecorder APIs
2. Play back the recorded chunks in a video element within Safari
OBSERVATION: The recorded video plays back properly in Safari
3. Save the recorded chunks in a file
4. Play back the saved file
OBSERVATIONS:
- Safari and Quicktime play the file properly. 
- Other video players cannot play the file (Chrome, VLC)
- The file's duration is incorrect (usually very large)
- ffmpeg cannot determine the file's duration or frame rate
EXPECTED: The file should play properly and have valid metadata. This was the case in iOS 13.

[Radar WebKit Bug Importer]

<rdar://problem/69377550>

[youenn fablet]

Did some tests and repro the issue.
On both Chrome and VLC, audio-only is rendered ok, video-only is rendered ok, except that video duration is not correctly computed.
In case of audio-video, there is probably an issue in trying to sync audio and video, which makes it fail to display the video.

[youenn fablet]

Pre Safari 14, the moov box was probably giving the duration of the media, which is no longer the case now that we are supporting chunk-based outputs

[youenn fablet]

Created attachment 409500 [details]
Patch

[Eric Carlson]

Comment on attachment 409500 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=409500&action=review

> Source/WebCore/platform/mediarecorder/cocoa/MediaRecorderPrivateWriterCocoa.mm:249
> +    [m_writer.get() startSessionAtSourceTime: kCMTimeZero];

Nit: extra space

[youenn fablet]

Created attachment 409534 [details]
Patch for landing

[EWS]

Committed r267521: <https://trac.webkit.org/changeset/267521>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 409534 [details].

[AndyDeveloper]

bug is still present in iOS 14.0.1 
(just noting this for future reference as I don't know which version the fix is targeted for)

[d2vid]

Created attachment 410116 [details]
MediaRecorder recorded video with STP Release 113 (Safari 14.0.1, WebKit 15610.2.3.1)

[d2vid]

Created attachment 410117 [details]
FFMPEG output video - dropping frames/stuttery but playable on Chrome

[d2vid]

(In reply to AndyDeveloper from comment #8)
> bug is still present in iOS 14.0.1 
> (just noting this for future reference as I don't know which version the fix
> is targeted for)

Can you share the ffmpeg command you used to transcode a valid video file? And is it literally only a "valid" file but has transcoding issues, or are you getting a quality video out?

Here's the command I'm using to create a Chrome-playable video from a Safari MediaRecorder recorded video, but it's dropping frames/stuttery:
  $ ffmpeg -r 30 -i in.mp4 -y -threads 8 -analyzeduration 2147483647 -probesize 2147483647 -max_muxing_queue_size 9999 out.mp4

(I've attached for input/output video files for anyone's reference)

[AndyDeveloper]

@d2vid are you using the latest version of ffmpeg?  I remember I did have to upgrade. I just tried the exact command on your invalid input file and got a good output video that doesn't skip. My version is ffmpeg version N-99213-ga265e6604e-tessus.
The command I normally use is a bit different and requires you to know the actual duration
ffmpeg -r 29.97 -t 6000ms -i in.mp4 -r 29.97 -profile:v main  -max_muxing_queue_size 4096 out.mp4

[AndyDeveloper]

@d2vid are you using the latest version of ffmpeg?  I remember I did have to upgrade. I just tried YOUR exact command on your invalid input file and got a good output video that doesn't skip. My version is ffmpeg version N-99213-ga265e6604e-tessus.

ffmpeg -r 30 -i in.mp4 -y -threads 8 -analyzeduration 2147483647 -probesize 2147483647 -max_muxing_queue_size 9999 out.mp4

Note that the command I normally use is a bit different and requires you to know the actual duration, although it often works without it.
ffmpeg -r 29.97 -t 6000ms -i in.mp4 -r 29.97 -profile:v main -pix_fmt yuv420p -max_muxing_queue_size 4096 out.mp4

Maybe a stupid question but is there any way to determine the duration if the metadata in the file is invalid?

[AndyDeveloper]

> is there any way to determine the duration if the metadata in the file is invalid?

Great question!  I just tried this and it seems to work:

ffprobe -v error -count_frames -select_streams v:0 -show_entries stream=nb_read_frames -of default=nokey=1:noprint_wrappers=1 in.mp4 | { read num_frames; echo "$num_frames * 1000 / 29.97"; } | bc

[d2vid]

(In reply to AndyDeveloper from comment #13)
> @d2vid are you using the latest version of ffmpeg?  I remember I did have to
> upgrade. I just tried YOUR exact command on your invalid input file and got
> a good output video that doesn't skip. My version is ffmpeg version
> N-99213-ga265e6604e-tessus.
> 
> ffmpeg -r 30 -i in.mp4 -y -threads 8 -analyzeduration 2147483647 -probesize
> 2147483647 -max_muxing_queue_size 9999 out.mp4
> 
> Note that the command I normally use is a bit different and requires you to
> know the actual duration, although it often works without it.
> ffmpeg -r 29.97 -t 6000ms -i in.mp4 -r 29.97 -profile:v main -pix_fmt
> yuv420p -max_muxing_queue_size 4096 out.mp4

@AndyDeveloper Thank you so much, that did the trick! I'm now on FFMPEG version N-99557-g6bdfea8 and the command `ffmpeg -r 30 -i in.mp4 -y -threads 8 -analyzeduration 2147483647 -probesize 2147483647 -max_muxing_queue_size 9999 out.mp4` is working.

For the few sample videos I have from users/myself I can confirm that the transcoding command works without the `-t` time argument, but I'll report back if I encounter any where that argument is necessary.