MediaStream API: support MediaStreamRecorder implementation

• Component: WebRTC
• Status: NEW ---

[William Lin]

To support MediaStreamRecorder implementation in MediaStream API

[Tommy Widenflycht]

Just FYI this feature is quite far in the future since the specification is way behind.

[Miguel Casas-Sanchez]

Update: This is available in Chrome and Firefox:

http://caniuse.com/#search=mediarecorder

and the Spec is on its way to become Candidate Recommendation.

[Jeremy Noring]

I'd also like to add this is one feature that's important to have to abandon flash support.  Right now to record media in the browser, really the only way besides javascript encoder/decoder implementations (performance isn't there yet) is to use a media server and stream rtmp (yuck!).

[Frank Faubert]

Please add support for this so we can finally get rid of flash...

[waza123]

MediaRecorder
PLEASE CREATE THIS Class faster !

[Jeremy Noring]

I'm at the point where I need this feature so badly that I'm willing to try and do the work myself.  Questions:

1. Would webkit accept a contribution like this, assuming I went through the normal process to make contributions to webkit, and 
2. Is there a webkit developer I could work with to make this happen?

Roundabout story: we've tried to make a MediaRecorder polyfill with webassembly, and it is prohibitively difficult for a number of reasons (tl;dr codec royalties are a thing, javascript garbage collection and video are hard, script processor nodes are finicky and working with a canvas has some idiosyncrasies that are difficult to address).  

Any advice: hugely welcome.

[Eric Carlson]

Apologies for the late reply, I didn't see your questions before now. 

(In reply to Jeremy Noring from comment #6)
> I'm at the point where I need this feature so badly that I'm willing to try
> and do the work myself.  Questions:
> 
> 1. Would webkit accept a contribution like this, assuming I went through the
> normal process to make contributions to webkit, and 

Yes of course!

> 2,. Is there a webkit developer I could work with to make this happen?
> 

If you mean someone to help with questions and patch reviews, I will be happy to help out.

[Eric Carlson]

<rdar://problem/40939511>

[youenn fablet]

> 1. Would webkit accept a contribution like this, assuming I went through the
> normal process to make contributions to webkit, and 

+1 !!

> 2. Is there a webkit developer I could work with to make this happen?

I can also help a bit.

> Roundabout story: we've tried to make a MediaRecorder polyfill with
> webassembly, and it is prohibitively difficult for a number of reasons
> (tl;dr codec royalties are a thing, javascript garbage collection and video
> are hard, script processor nodes are finicky and working with a canvas has
> some idiosyncrasies that are difficult to address).  

+1

[Octavian Naicu]

Having this feature in Safari on iOS would allow us to move away from the limited HTML Media Capture spec (which on Safari/iOS doesn't even work for audio just video) and implement proper video recording directly in iOS devices using our UI, our start/stop/pause controls, our constraints, JavaScript, we could trigger events, etc.

Having this feature in Safari on macOS would allow us to retire the last piece of Flash we're using.

(In reply to Jeremy Noring from comment #6)

> Roundabout story: we've tried to make a MediaRecorder polyfill with
> webassembly, and it is prohibitively difficult for a number of reasons
> (tl;dr codec royalties are a thing, javascript garbage collection and video
> are hard, script processor nodes are finicky and working with a canvas has
> some idiosyncrasies that are difficult to address).  

We've actually considered doing the same thing: taking snapshots from the canvas and encoding them as video using a WebAssembly video encoding library. I'm not sure it's possible though or, if works, if it could be used in production. For example these FFMPEG WebAssembly builds are 12MB in size and 4-5MB gzipped https://github.com/BrianJFeldman/ffmpeg.js.wasm. Another problem is audio would have to be captured separately and thus it will most certainly be out of sync.

As an example, there is a working MediaRecorder polyfill for audio that records uncompressed pcm in .wav https://github.com/ai/audio-recorder-polyfill

We've also tried using WebRTC to establish a p2p connection to a server and record the video server side but the video quality is bad (since WebRTC prioritizes low latency as it should) and the connection process is extremely complex (to account for all possible networks) resulting in failed connections: https://addpipe.com/blog/troubleshooting-webrtc-connection-issues/

Chrome supports mediastream recording since January 2016. It was "The most starred Chrome feature EVER" with 2877 stars https://developers.google.com/web/updates/2016/01/mediarecorder

I am not even that much concerned about codecs as we can easily convert those server side to whatever we need.

[Octavian Naicu]

Just a thought: If this does not make it in Safari/12 and Safari 12/macOS we'll most probably have to wait at least another year until Safari 13.

[Andrey Sitnik]

I wrote a polyfill for MediaRecorder for audio https://github.com/ai/audio-recorder-polyfill

But it will be much better to have native support. Really useful feature.

+1 from me.
I am working on PWAs for clients who would love proper audio and especially video support.

[Remus Negrota]

Having this feature will greatly improve usability on iOS. As of now on Safari on iOS if you want to capture just audio you have to rely on HTML Media Capture and this does not even allow you to record audio directly, it records both videa and audio and after that we must extract the audio.

Not the simplest of tasks as it should be and is on all other modern browsers.

[Jeremy Noring]

(In reply to Octavian Naicu from comment #10)
> Having this feature in Safari on iOS would allow us to move away from the
> limited HTML Media Capture spec (which on Safari/iOS doesn't even work for
> audio just video) and implement proper video recording directly in iOS
> devices using our UI, our start/stop/pause controls, our constraints,
> JavaScript, we could trigger events, etc.
> 
> Having this feature in Safari on macOS would allow us to retire the last
> piece of Flash we're using.
> 
> (In reply to Jeremy Noring from comment #6)
> 
> > Roundabout story: we've tried to make a MediaRecorder polyfill with
> > webassembly, and it is prohibitively difficult for a number of reasons
> > (tl;dr codec royalties are a thing, javascript garbage collection and video
> > are hard, script processor nodes are finicky and working with a canvas has
> > some idiosyncrasies that are difficult to address).  
> 
> We've actually considered doing the same thing: taking snapshots from the
> canvas and encoding them as video using a WebAssembly video encoding
> library. I'm not sure it's possible though or, if works, if it could be used
> in production. For example these FFMPEG WebAssembly builds are 12MB in size
> and 4-5MB gzipped https://github.com/BrianJFeldman/ffmpeg.js.wasm. Another
> problem is audio would have to be captured separately and thus it will most
> certainly be out of sync.
> 
> As an example, there is a working MediaRecorder polyfill for audio that
> records uncompressed pcm in .wav
> https://github.com/ai/audio-recorder-polyfill
> 
> We've also tried using WebRTC to establish a p2p connection to a server and
> record the video server side but the video quality is bad (since WebRTC
> prioritizes low latency as it should) and the connection process is
> extremely complex (to account for all possible networks) resulting in failed
> connections:
> https://addpipe.com/blog/troubleshooting-webrtc-connection-issues/
> 
> Chrome supports mediastream recording since January 2016. It was "The most
> starred Chrome feature EVER" with 2877 stars
> https://developers.google.com/web/updates/2016/01/mediarecorder
> 
> I am not even that much concerned about codecs as we can easily convert
> those server side to whatever we need.

So the issue with codecs is: if you want to playback the video you just recorded in the browser--which is a really common use case--then you either need MediaRecorder producing files supported by that particular browser, or you need yet another polyfill (we've been using ogv.js, for example) to decode whatever random codec you recorded in locally.

This is where the codec royalty issue comes into play: if you ship a polyfill that records in H.264, your company owes royalties to MPEG-LA. The easy way around this is to record in something that doesn't require royalties, like VP8, but then that necessitates yet another webassembly project to playback the video you just recorded.

[Octavian Naicu]

On desktop we're using Media Recorder API + Flash on Safari/IE/legacy and that works great. We're most concerned about mobile where on Safari we're stuck with HTML Media Capture while Chrome on Android has full Media Recorder API support.

[Michael L.]

This is also crucial to us, for the exact same reasons as previous contributors, getting rid of Flash and Media Capture for the sake of user experience.

Happy to help if I can, please update us on the current status !

[jp pincheira]

Crucial to us too, we really don't want to use Flash, so sadly, we're encouraging our users to move away from Desktop Safari at the moment, to Chrome/FF/Opera.

[Robert Gerald (Rob) Porter]

This is something where we've been forced to recommend only Android devices (&Chrome) for doing short video recording for our clients. Would love to see this having proper cross-browser support. Feels like such a large gap item to me, especially considering the push to abandon Flash.

[Shradha]

We are using the api to let our users record short videos(<1 min). We've been forced to direct our users to download chrome/ff. We do want it on Safari as well but right now it just seems like too much effort.Waiting for it to be released for safari.

[Matt]

Even Adobe recommends disabling Flash:

"Adobe said that it will now 'encourage content creators to build with new web standards,' such as HTML5, rather than Flash. It's also beginning to deprecate the Flash name by renaming its animation app to Animate CC, away from Flash Professional CC."

https://www.theverge.com/2015/12/1/9827778/stop-using-flash

[jp pincheira]

I feel that "Status:	UNCONFIRMED" is a bit of a disrespect to developers using and caring about Webkit. It is really offensive that this is the status since 2012.

[Timothy Swieter]

I'd love to see this implemented so that Safari and Chrome/FF can have similar functionality.

6 years passed, the bug still exists...

[Thomas Sigdestad]

Let's get this done Apple!
Apple is slowly turning into a new Microsoft by stalling the development of the web - just like they did with ie6. That did not end well...

I'm sad to say I have many friends and colleagues that have used iOS since 2009, that have now abandoned it for Android due to the limited web support. Under Jobs, Safari/iOS was the best mobile web platform on earth - not anymore :-(

Can someone from the webkit team tell us whether there is a plan to fix/implement this? If it's going to come and we just need to wait, then that's useful information. If it's not going to happen at all, then we know we need to make other plans.
Any info either way would be very useful.

Thankyou

Stuart

[youenn fablet]

As detailed in https://webkit.org/status/#feature-mediastream-recording-api, this feature is under consideration. Corresponding WPT tests have also been imported in WebKit and are run in WebKit CI.

Any help around that feature is welcome.
Contributing media stream recording API tests to https://github.com/web-platform-tests/wpt would be useful. It would require MediaStream recording API knowledge plus general JS/HTML knowledge.

[Zach Rattner]

+1 that this would be great to see ton mobile Safari since Chrome has had it for some time now. Recording streams through WebRTC sacrifices video quality since it's fundamentally a different use case than record + upload. Sort of strange to still be in unconfirmed status after 6 years...

[Octavian Naicu]

We are now working on implementing recording through WebRTC for Safari on iOS and macOS and it is a pain: you need a server to handle signalling and act as the other client, the connection process is complex and might fail especially without another TURN server/service and the quality is just not there because WebRTC is made for low latency live video.

On Chrome/Android we can just use the MediaStream Recording API. Its the same code we use on desktop plus a few extra lines to handle the front/back camera.

[Allan]

We are working on doing a mobile version of our application and the challenge is to record videos using camera of IOS devices such as Iphone or iPADs, we are stuck and so far only android /chrome implementation is possible. If anyone has ideas, it will be grateful to share..

[Allan]

We are working on doing a web app version of our application on mobile devices,  the challenge is to use safari and access the camera and record videos on IOS devices such as Iphone or iPADs, we are stuck and so far only android /chrome implementation is possible. If anyone has ideas, it will be grateful to share..

[Mark Beeby]

Have the Safari team taken note of Edge's latest maneuver? In adopting the Chromium project they're going to be able to take the Microsoft brand forward without holding everyone back, Safari really is going to be the biggest pain in back for developers. This feature is one of too many that are sat in the ignored column by Apple...

[David Mannion]

We really need this feature implemented.  We have an application that works great in Chrome and Firefox on non-Apple devices but we had to write a work around for Webkit that requires the user to jump out of the browser to the device's video recording app to capture their video.  Please get this done ASAP!

[Octavian Naicu]

Great news! 

Safari Technology Preview 73 implements a basic working version of the MediaStream Recorder API.

I've covered what works and what doesn't here: https://addpipe.com/blog/safari-technology-preview-73-adds-limited-mediastream-recorder-api-support/

Hopefully a fully working implementation will be shipped in Safari 12.x .

[Adam Szaloczi]

I am planning to make an app, where the main functionality would be audio recording in browser, but currently, the mediarecorder API is the bottleneck. There is a workaround with webaudio API, but thats very poorly documented, so very hard to implement.

Please add support, working with media stream server is scary and not supported by our current stack.

[Jeff]

iOS 12.2 still does not support MediaRecorder. Please add!

[Octavian Naicu]

Quick update:  the MediaRecorder feature in Safari is an Experimental Feature in both iOS 12.4 and iOS 13 public beta 4. It is turned off by default. You need to manually enable it from the iOS Settings > Safari > Advanced > Experimental Features for it to work.

[Mayowa Daniel]

finally!!!!(In reply to Octavian Naicu from comment #38)
> Quick update:  the MediaRecorder feature in Safari is an Experimental
> Feature in both iOS 12.4 and iOS 13 public beta 4. It is turned off by
> default. You need to manually enable it from the iOS Settings > Safari >
> Advanced > Experimental Features for it to work.

[Nam]

The recorded video frames are rotated 90 degrees and stretched by the MediaRecorder feature in Safari is an Experimental Feature

[Alon Meytal]

Any idea when is this feature going to be out of the experimental phase??

[Octavian Naicu]

(In reply to Alon Meytal from comment #41)
> Any idea when is this feature going to be out of the experimental phase??

Hopefully in iOS 13 - Apple's event is tomorrow - together with an HTML Media Capture implementation for audio-only recordings. That would bring it in line with Chrome on Android in terms of audio & video recording capabilities.

[Octavian Naicu]

Installed Safari 13 on my Mac and iPhone yesterday. 

Media Stream Recorder API is still experimental :(.

On top of that trying to record just audio with HTML Media Capture crashes Safari on iOS and iPad OS, I've reported the issue here: https://bugs.webkit.org/show_bug.cgi?id=202039

[Senthil Ramachandran]

Hi, with experimental features turned on, recorded videos look stretched and flipped -90 degrees.

Here is the my repo I used to reproduce this.
https://github.com/senpost/mediarecorderX

If you don't have node or you don't want to run this locally, I can try to publish this site into netlify so it is accessible from internet.

The experimental implementation still results in stretched/flipped video files. Here is a video of what this looks like in iOS 13.3: https://youtu.be/OVDM-aQhoiM

Repro steps:
1. Turn on MediaRecorder in Safari Experimental Settings (iOS 13)
2. Navigate to https://addpipe.com/media-recorder-api-demo/
3. Record + Save video
4. Open saved video in Chrome (no iOS webm playback)

Also described here: https://bugs.webkit.org/show_bug.cgi?id=198912

[salvo]

Hi there!,

Currently even last Safari 13.1 does not have MediaRecorder activated by default.

Is there a release date for this Api?
Thanks

[Diego Caravana]

Just curious to know why this feature is just half-baked.

Why even start implementing it if there is no interest (from Apple at least) in releasing it?

And it's clear there is no interest cause MediaRecorder is there as an experimental feature since more than one year, and given the fairly good implementations in other browsers, there is clearly no technical reason not to complete and deliver it.

Our clients are demanding more and more that our software works on iOS, too, but we cannot satisfy their needs as currently there is no simple alternative on iOS to do video recording, except using a distorted implementation of WebRTC perhaps, or creating an app, which we don't want to create yet.

In these hard times, enabling communication in every possible way should be the first priority of a company like Apple, a company that has a huge impact on many millions of lives every day.

Thanks for listening.

[salvo]

@Diego. i think the biggest problem is the webm video replay after record.

it is very strange that MediaReacorder api can record in webm format but <video tag cannot reproduce it.

i have impementend a workaorund for this problem: convert the video client side with ffmepg_asm.js with codec: copy. just convert the container from webm to mp4.

So i need only to convice my clients to turn on the Experminetal Features flag.

Please Apple, take us a release date!!!!!

[Octavian Naicu]

(In reply to Diego Caravana from comment #47)

> there is no simple
> alternative on iOS to do video recording, except using a distorted
> implementation of WebRTC perhaps, or creating an app, which we don't want to
> create yet.

We're actually still using a Flash-based client and a Wowza media server to record audio & video from Safari users. I think we'll stop supporting Safari end of 2020 unless Media Stream Recorder API is implemented.

[Kim Scott]

We're currently just not supporting Safari at https://lookit.mit.edu (video-based online developmental research) because of this issue. Would be very excited about MediaStreamRecorder support!

[AndyDeveloper]

@salvo (In reply to salvo from comment #48)

> it is very strange that MediaReacorder api can record in webm format but
> <video tag cannot reproduce it.

MediaRecorder on Safari can record directly in mp4 
 new MediaRecorder(stream, { type: 'video/mp4' })

Cheers

[Diego Caravana]

(In reply to Kim Scott from comment #50)
> We're currently just not supporting Safari at https://lookit.mit.edu
> (video-based online developmental research) because of this issue. Would be
> very excited about MediaStreamRecorder support!

Thank you Kim, I was planning to do something to fill this gap anyway, but your project gave me another strong reason to do it.

So I'm even happier now to share the proof of concept I put together.

Basically it's a MediaRecorder polyfill that merges code from these two projects:

- https://github.com/GoogleChromeLabs/webm-wasm
- https://github.com/kbumsik/opus-media-recorder


The first one provides a way to efficiently create a webm video file through Wasm, while the second is a (good) example of MediaRecorder polyfill (just for audio recording, though).

All the credit goes to these two projects, I just packaged them together.

Both are fairly complex projects, so to make things simpler I worked on this file only:

https://github.com/dcaravana/webm-wasm/blob/ios_test/demo/live_download_mediarecorder.html


I want to reiterate that this is just a prototype, the code is quite rough and far from being usable in production, so feel free to take inspiration from it or even contribute back.

I'm quite proud of this result as it defeats any  reasoning behind holding the MediaRecorder implementation back in Safari (in fact, it took me a bit more that one day of work).

[Diego Caravana]

No feedback in one month.

This bug has been filed in 2012.

After considering what I heard at WWDC keynote about Safari, I'm assuming there is an explicit will not to finish this implementation.

I just wonder why?

An official word from Apple (and I see Apple employees in the CC list here) would at least make it all more polite and put everyone's hopes finally to rest.

Maybe something like this?

https://www.zdnet.com/article/apple-declined-to-implement-16-web-apis-in-safari-due-to-privacy-concerns/


Thanks.

[youenn fablet]

Implementation of MediaRecorder is making some progress with the implementation of progressive results now implemented.
Testing is of course most welcome when these changes will hit STP.

[Zach Rattner]

The latest Safari Technology Preview (release 110) adds support for MediaRecorder.onstart and other APIs crucial for recording: 

https://developer.apple.com/safari/technology-preview/release-notes/

The Apple Developer site is only clear on how to get Safari Technology Preview on macOS. I couldn't find iOS instructions, but I'll try getting the latest iOS 14 beta on my iPhone and see if that does the trick.

[Jeremy Noring]

This is now available in Chrome, Firefox, and Edge.  The only missing browser is safari.  Is there _any_ hope of this making into into safari without being hidden behind a switch?  I don't even want to go into the sort of horrific crap we've had to do to continue supporting safari for recording.

[Octavian Naicu]

Safari 14 is one or two months out, is this going to be in Safari 14 (without the flag)? We need to know to prepare for either case:

1. not supported at launch or behind a flag: we'll not support Safari 14 and need to announce clients
2. supported at launch: we need to make sure our current MediaStream Recorder implementation works as expected

[Jeremy Noring]

Ditto.  I'd like to know if we can stop blocking safari.

[Diego Caravana]

For the recordi, Safari 14, out today for desktop and mobile, doesn't support this API yet.

Expected given the issues that are still open here.

[AndyDeveloper]

I'm observing that in iOS 14 Safari, the video no longer appears stretched (it is still rotated -90 degrees). Any existing workarounds will have to become conditional, based on the iOS version.

[AndyDeveloper]

Also, I am experiencing a new issue with the recorded files since iOS 14 and Safari 14 on MacOS. Most media players can't play the recorded files. It seems the metadata (duration, frame rate...) are not set properly. If I re-encode the file assuming the frame rate, then I get a valid file.

[youenn fablet]

(In reply to AndyDeveloper from comment #61)
> Also, I am experiencing a new issue with the recorded files since iOS 14 and
> Safari 14 on MacOS. Most media players can't play the recorded files. It
> seems the metadata (duration, frame rate...) are not set properly. If I
> re-encode the file assuming the frame rate, then I get a valid file.

Can you file a new bug report and provide some repro steps?

[AndyDeveloper]

Filed this bug: 
MediaRecorder produces invalid video files since iOS/Safari 14
https://bugs.webkit.org/show_bug.cgi?id=216832

[Marcos G.]

Using safari 14, I am able to get video recording working using this example from https://webrtc.github.io/samples/src/content/getusermedia/record/.

In order to get the example working, I ignored the isTypeSupported check, specified video/mp4 vs video/webm, updated the constraints {video: true, audio:true}, and I am able to record video but not audio.

https://codepen.io/marcosdg3/pen/rNeoyzz

I used ffprobe and found the audio stream is missing. Am I missing something, is there any documentation you can share on how this is being used/tested in-house?

Any help would be much appreciated as we would like to support safari.

Video recorded in chrome:
$ ffprobe tested-in-chrome.mp4 
ffprobe version 4.2.2 Copyright (c) 2007-2019 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'tested-in-chrome.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf57.56.101
  Duration: 00:00:03.31, start: 0.000000, bitrate: 167 kb/s
    Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 480x360 [SAR 1:1 DAR 4:3], 126 kb/s, 16 fps, 16 tbr, 16384 tbn, 32 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 24000 Hz, mono, fltp, 32 kb/s (default)
    Metadata:
      handler_name    : SoundHandler


Video recorded in safari:
$ ffprobe tested-in-safari.mp4
ffprobe version 4.2.2 Copyright (c) 2007-2019 the FFmpeg developers
  built with Apple clang version 11.0.0 (clang-1100.0.33.17)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/4.2.2_2 --enable-shared --enable-pthreads --enable-version3 --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libbluray --enable-libmp3lame --enable-libopus --enable-librubberband --enable-libsnappy --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-librtmp --enable-libspeex --enable-libsoxr --enable-videotoolbox --disable-libjack --disable-indev=jack
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'tested-in-safari.mp4':
  Metadata:
    major_brand     : iso5
    minor_version   : 1
    compatible_brands: isomiso5hlsf
    creation_time   : 2020-09-22T18:20:34.000000Z
  Duration: 18:47:50.35, start: 67669.391667, bitrate: 0 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 0 kb/s, 30.39 fps, 30 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      creation_time   : 2020-09-22T18:20:34.000000Z
      handler_name    : Core Media Video

[Marcos G.]

I had 'Modern WebAudio API' enabled in experimental features which prevented sound from being recorded. After disabling and restarting the browser, audio is recorded. The video is corrupted though.

Also updating my audio constraints to: 
{sampleRate: 24000, noiseSuppression: true, echoCancellation: true} 
prevented audio recording. 

Noting that the video constraints are ignored as well. 

Output from ffprobe for video recording:

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fbd52008200] Could not find codec parameters for stream 0 (Video: h264 (avc1 / 0x31637661), none, 640x480, 1 kb/s): unspecified pixel format
Consider increasing the value for the 'analyzeduration' and 'probesize' options
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'safari.mp4':
  Metadata:
    major_brand     : iso5
    minor_version   : 1
    compatible_brands: isomiso5hlsf
    creation_time   : 2020-09-22T20:16:42.000000Z
  Duration: 20:44:27.94, start: 2.249333, bitrate: 1 kb/s
    Stream #0:0(und): Video: h264 (avc1 / 0x31637661), none, 640x480, 1 kb/s, SAR 1:1 DAR 4:3, 30.05 fps, 600 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      creation_time   : 2020-09-22T20:16:42.000000Z
      handler_name    : Core Media Video
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 174 kb/s (default)
    Metadata:
      creation_time   : 2020-09-22T20:16:42.000000Z
      handler_name    : Core Media Audio

[youenn fablet]

(In reply to Marcos G. from comment #65)

This bug entry is to keep track of all MediaRecorder bugs.
Would you be able to file a specific bug entry for your issue and cc me?
Repro steps are always helpful.

[Marcos G.]

Yes of course, I'll review and create tickets for issues that aren't already listed.

[alovell]

I have noticed a lot of work being done by apple for MediaRecorder. Do we know if this is planed to be released with macOS Big Sur? https://developer.apple.com/safari/technology-preview/release-notes/

(In reply to AndyDeveloper from comment #61)
> Also, I am experiencing a new issue with the recorded files since iOS 14 and
> Safari 14 on MacOS. Most media players can't play the recorded files. It
> seems the metadata (duration, frame rate...) are not set properly. If I
> re-encode the file assuming the frame rate, then I get a valid file.

We are experiencing the same issue. Can you please provide the values that you re-encode the file with as a temporary workaround? If you have the ffmpeg command that would be even better. Thanks.

[d2vid]

(In reply to Marcos G. from comment #65)
> unspecified pixel format
> Consider increasing the value for the 'analyzeduration' and 'probesize'

On my end, I'm able to use ffprobe on a Safari MediaRecorder created video if I increase the "-analyzeduration" and "-probesize" options. My (very incomplete) understanding is that this is caused by the moov atom header info being placed at the end of the file, so these option flags are require for ffprobe to scan enough of the file to find that info.

That being said, I'm still unable to play these videos outside of Safari/Quicktime, hoping that https://bugs.webkit.org/show_bug.cgi?id=216832 might resolve the issue.

Here's my command and output:

$ ffprobe -analyzeduration 2147483647 -probesize 2147483647 in.mp4 
ffprobe version N-93789-g06ba4783a0 Copyright (c) 2007-2019 the FFmpeg developers
  built with Apple LLVM version 10.0.1 (clang-1001.0.46.3)
  configuration: --prefix=/usr/local --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --disable-stripping --enable-avresample --enable-avisynth --enable-libbs2b --enable-libcaca --enable-libfontconfig --enable-libfribidi --enable-libgme --enable-libgsm --enable-libopenjpeg --enable-libpulse --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtwolame --enable-libwavpack --enable-libwebp --enable-libxml2 --enable-libxvid --enable-libzmq --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-chromaprint --enable-frei0r --enable-shared --enable-nonfree
  libavutil      56. 26.101 / 56. 26.101
  libavcodec     58. 52.101 / 58. 52.101
  libavformat    58. 27.103 / 58. 27.103
  libavdevice    58.  7.100 / 58.  7.100
  libavfilter     7. 50.100 /  7. 50.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'in.mp4':
  Metadata:
    major_brand     : iso5
    minor_version   : 1
    compatible_brands: isomiso5hlsf
    creation_time   : 2020-09-29T14:46:52.000000Z
  Duration: 49:36:09.46, start: 3.668000, bitrate: 0 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 0 kb/s, 30.02 fps, 600 tbr, 600 tbn, 1200 tbc (default)
    Metadata:
      creation_time   : 2020-09-29T14:46:52.000000Z
      handler_name    : Core Media Video
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, mono, fltp, 132 kb/s (default)
    Metadata:
      creation_time   : 2020-09-29T14:46:52.000000Z
      handler_name    : Core Media Audio

[d2vid]

Created attachment 410114 [details]
MediaRecorder recorded video with STP Release 113 (Safari 14.0.1, WebKit 15610.2.3.1)

[d2vid]

Created attachment 410115 [details]
FFMPEG output video - dropping frames/stuttery but playable on Chrome

[youenn fablet]

Would it be possible to continue discussing this specific issue in https://bugs.webkit.org/show_bug.cgi?id=216832?

[Marcos G.]

Is this related to https://bugs.webkit.org/show_bug.cgi?id=169870 ?

I attempted to specify the mediastream constraints using https://codepen.io/marcosdg3/pen/jOqogQd

Ex:

navigator.mediaDevices.getUserMedia({
                    audio: {
                        sampleRate: 24000,
                        echoCancellation: true
                    },
                    video: {
                        width: {
                            min: 480,
                            ideal: 640,
                            max: 800
                        },
                        height: {
                            min: 360,
                            ideal: 480,
                            max: 600
                        },
                        frameRate: 16,
                        aspectRatio: (4 / 3)
                    }
                })


No sound is recorded (I think this may have been addressed in another ticket). Also the frameRate config is ignored.

The recorded output from ffprobe (30.36 fps vs the 16 fps configured in the recorder):
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 0 kb/s, 30.36 fps, 29.92 tbr, 600 tbn, 1200 tbc (default)

[youenn fablet]

(In reply to Marcos G. from comment #74)
> Is this related to https://bugs.webkit.org/show_bug.cgi?id=169870 ?
> 
> I attempted to specify the mediastream constraints using
> https://codepen.io/marcosdg3/pen/jOqogQd

Please file a new bug entry.
For frameRate, we are doing integer decimation.
Can you try 14 or 15 instead of 16, given the native frame rate might be 30fps.

For audio not playing, maybe try with audio:true only.

[Marcos G.]

Thank you youenn, 15 worked for me.

Created https://bugs.webkit.org/show_bug.cgi?id=217147 to track.

[Zach Rattner]

Does anyone here have any info on how to test this on an iPhone or iPad?

MediaRecorder is enabled by default on iOS in WebKit revision 267400: https://trac.webkit.org/changeset/267400/webkit

Safari Tech Preview 115 (released Oct 22) covered revisions 267325-267719: https://webkit.org/blog/11333/release-notes-for-safari-technology-preview-115/ 

It appears Apple bundles Safari with iOS and doesn’t make specific WebKit version numbers available to mere mortals. So I’m not clear on how to get STP 115 on a mobile device.

[AndyDeveloper]

(In reply to Zach Rattner from comment #77)
> Does anyone here have any info on how to test this on an iPhone or iPad?

I usually get the latest Safari version by installing the latest iOS developer beta version. I just upgraded now to iOS 14.2 RC by installing the iOS developer "beta profile". This upgraded Safari to 14.0.1.  However, I don't think that version corresponds to Release 115, as the issues I tested are still present, and the version we need is probably Safari 14.1 (same as Mac OS).

fyi - I've tested the encoding issue on MacOS Safari Technology Preview Release 115 (Safari 14.1) and it seems to be fixed.

[AndyDeveloper]

UPDATE:  I just updated to iOS 14.3 developer beta. Safari appears to have the bug fixes from this thread (orientation, encoding). I can't tell whether MediaRecorder has become enabled by default, and I can't tell what webkit version is used. The user agent string hasn't changed much from iOS 14.2 to 14.3.
As a result of the bug fixes, any workarounds need to be made conditional on the version of iOS.

[Zach Rattner]

Thanks AndyDeveloper for the sleuthing. On iOS 14.3 beta, I still see MediaRecorder.isTypeSupported is not defined, so that means https://bugs.webkit.org/show_bug.cgi?id=216856 is not present. The patch for this bug is Subversion revision 267821 (dated Oct 1), so whatever revision is present on iOS 14.3 beta is earlier than this.

The patch to enable MediaRecorder by default on iOS ( https://bugs.webkit.org/show_bug.cgi?id=216664 ) is Subversion revision 267225 (dated Sep 21). When I take a vanilla factory reset iPad on 14.2 and install the 14.3 beta, I see the MediaRecorder flag changed from off in 14.2 to on in 14.3. 

So, for anyone trying to triangulate when MediaRecorder support will hit on iOS/iPadOS, this leads me to believe the Webkit version used in Safari on 14.3 beta has Subversion revision after 267225 (Sep 21) but before 267821 (Oct 1).

[Marcel]

We are trying to get the MediaRecoder working with only one video tag. 

So for playing the GUM stream we do:
const stream = await navigator.mediaDevices.getUserMedia(constraints);
const video = document.querySelector('video#gum');
video.srcObject = stream;


After we have created a recoding with the MediaRecoder we want to play the video we do the following: 
const superBuffer = new Blob(recordedBlobs, {type: 'video/webm'});
video.srcObject = null;
video.src = window.URL.createObjectURL(superBuffer);
video.play();

We see a short flickering in the video playout but the stream form the GUM keeps playing 


We have found the link explaining we should us the Blob directly on the src of the video tag in case of Safari https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/srcObject
const superBuffer = new Blob(recordedBlobs, {type: 'video/webm'});
video.srcObject = superBuffer;
video.play();

We also see a short flickering in the video playout but the stream form the GUM keeps playing 


With two video tags where one is for recoding (the GUM stream) and the other for playing the recoding (blob). Then it works fine.

[youenn fablet]

MediaRecorder is not supporting webm format at the moment.
Ditto for the media element.
Can you retry by using video/mp4?


(In reply to Marcel from comment #81)
> We are trying to get the MediaRecoder working with only one video tag. 
> 
> So for playing the GUM stream we do:
> const stream = await navigator.mediaDevices.getUserMedia(constraints);
> const video = document.querySelector('video#gum');
> video.srcObject = stream;
> 
> 
> After we have created a recoding with the MediaRecoder we want to play the
> video we do the following: 
> const superBuffer = new Blob(recordedBlobs, {type: 'video/webm'});
> video.srcObject = null;
> video.src = window.URL.createObjectURL(superBuffer);
> video.play();
> 
> We see a short flickering in the video playout but the stream form the GUM
> keeps playing 
> 
> 
> We have found the link explaining we should us the Blob directly on the src
> of the video tag in case of Safari
> https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/srcObject
> const superBuffer = new Blob(recordedBlobs, {type: 'video/webm'});
> video.srcObject = superBuffer;
> video.play();
> 
> We also see a short flickering in the video playout but the stream form the
> GUM keeps playing 
> 
> 
> With two video tags where one is for recoding (the GUM stream) and the other
> for playing the recoding (blob). Then it works fine.

[Marcel]

Thank you for your quick response Youenn

My bad. I posted the wrong line of code for the creation of the Blob. It should have been: 
const blob = new Blob(recordedBlobs, {type: mediaRecorder.mimeType});

mediaRecorder.mimeType returns "video/mp4"

And yes after this change we still have the same issues.


(In reply to youenn fablet from comment #82)
> MediaRecorder is not supporting webm format at the moment.
> Ditto for the media element.
> Can you retry by using video/mp4?

[Marcel]

Here is a example of the issue:

https://jsfiddle.net/vyapfckd/1/


(In reply to Marcel from comment #83)
> Thank you for your quick response Youenn
> 
> My bad. I posted the wrong line of code for the creation of the Blob. It
> should have been: 
> const blob = new Blob(recordedBlobs, {type: mediaRecorder.mimeType});
> 
> mediaRecorder.mimeType returns "video/mp4"
> 
> And yes after this change we still have the same issues.
> 
> 
> (In reply to youenn fablet from comment #82)
> > MediaRecorder is not supporting webm format at the moment.
> > Ditto for the media element.
> > Can you retry by using video/mp4?

[youenn fablet]

(In reply to Marcel from comment #84)
> Here is a example of the issue:
> 
> https://jsfiddle.net/vyapfckd/1/
> 
> 
> (In reply to Marcel from comment #83)
> > Thank you for your quick response Youenn
> > 
> > My bad. I posted the wrong line of code for the creation of the Blob. It
> > should have been: 
> > const blob = new Blob(recordedBlobs, {type: mediaRecorder.mimeType});
> > 
> > mediaRecorder.mimeType returns "video/mp4"
> > 
> > And yes after this change we still have the same issues.

Oh, I see, this is probably https://bugs.webkit.org/show_bug.cgi?id=219258.
Can you try in latest STP 117?

[Marcel]

YES, this fixes it :D


(In reply to youenn fablet from comment #85)
> (In reply to Marcel from comment #84)
> > Here is a example of the issue:
> > 
> > https://jsfiddle.net/vyapfckd/1/
> > 
> > 
> > (In reply to Marcel from comment #83)
> > > Thank you for your quick response Youenn
> > > 
> > > My bad. I posted the wrong line of code for the creation of the Blob. It
> > > should have been: 
> > > const blob = new Blob(recordedBlobs, {type: mediaRecorder.mimeType});
> > > 
> > > mediaRecorder.mimeType returns "video/mp4"
> > > 
> > > And yes after this change we still have the same issues.
> 
> Oh, I see, this is probably https://bugs.webkit.org/show_bug.cgi?id=219258.
> Can you try in latest STP 117?

[Octavian Naicu]

Hi,

I've spent the last few days playing with the implementation in Safari 14.0.2 and Safari TP 119. 

As a result I have 2 comments regarding the information presented in https://webkit.org/blog/11353/mediarecorder-api/

1) It mentions Safari Technology Preview 105 as the one that enabled support for the MediaRecorder API by default but in fact Safari TP 114 is the one which lists "Enabled MediaRecorder by default on macOS" in it's release notes:
https://developer.apple.com/safari/technology-preview/release-notes/

2) The code in that post shows the use of 1 second time slices:

    // Let's receive 1 second blobs
    mediaRecorder.start(1000); 

In Safari 14.0.2 this functionality is broken. Using the above code will produce an .mp4 file that's corrupt as it won't play in the browser or locally in VLC.

mediaRecorder.start() does work in Safari 14.0.2 on desktop.

[Octavian Naicu]

(In reply to Octavian Naicu from comment #87)

> 2) The code in that post shows the use of 1 second time slices:
> 
>     // Let's receive 1 second blobs
>     mediaRecorder.start(1000); 
> 
> In Safari 14.0.2 this functionality is broken. Using the above code will
> produce an .mp4 file that's corrupt as it won't play in the browser or
> locally in VLC.

I've tested some more and the `timeslice` argument is supported and it works as expected. I was testing cached code when I was unable to get it to work.

[Pantxisto]

Hello,

I have Safari 14.4 on Ipad (7º gen). The issue is complex, related to orientation and I will try to explain:
There are 2 scenarios:
1-If I get a MediaStream in the Ipad and I record with MediaRecorder (originally on portrait mode), when saving on the serverside the file is on portrait mode; very good. But if I try to pipe that file with stream (nodejs) and pipe the response, if the headers are not setted correctly (meaning, accept-range... and 206 status code) the video is streammed in landscape mode. Well, we can survive this setting all the headers and the status code. I dont know if this behaviour is natural.
2-If I get the mediastream (with the WebRTC API I get from the other peer), I use chrome (in my case ubuntu), and incomming mediastream comes from safari, if I save the stream in a video file on the server side, in this case the video is saved in landscape mode (originally been on portrait mode on the safari browser). Headers doesnt work; if I pipe the file to the response it is seen on landscape mode.

My posibble reasons for this bugs are:
1-When sending the mediastream with WebRTC API (ontrackadded)some metadata is changed or added and the mediastream is corrupted.
2-When receiving a mediastream from Safari (only mp4 supported) peer on Chrome peer (only webm supported) the origin mediastream (with mp4 support, this is a hipotesis) is wrong recorded because different types support.

Thank you very much!!

[youenn fablet]

(In reply to Pantxisto from comment #89)
> Hello,
> 
> I have Safari 14.4 on Ipad (7º gen). The issue is complex, related to
> orientation and I will try to explain:
> There are 2 scenarios:
> 1-If I get a MediaStream in the Ipad and I record with MediaRecorder
> (originally on portrait mode), when saving on the serverside the file is on
> portrait mode; very good. But if I try to pipe that file with stream
> (nodejs) and pipe the response, if the headers are not setted correctly
> (meaning, accept-range... and 206 status code) the video is streammed in
> landscape mode. Well, we can survive this setting all the headers and the
> status code. I dont know if this behaviour is natural.

Could you file a dedicated bug, ideally with some repro steps (jsfiddle would be great).

> 2-If I get the mediastream (with the WebRTC API I get from the other peer),
> I use chrome (in my case ubuntu), and incomming mediastream comes from
> safari, if I save the stream in a video file on the server side, in this
> case the video is saved in landscape mode (originally been on portrait mode
> on the safari browser). Headers doesnt work; if I pipe the file to the
> response it is seen on landscape mode.

I am not exactly clear what the bug is here.
Usually, with WebRTC, CVO information is provided as RTP header and should allow the server to control correctly the orientation.
You might want to try disabling this extension, so that Safari will do the rotation before sending the data.

[Pantxisto]

Hello,
It would be great to file a dedicated bug with some repro steps but unfortunatelly I don’t have time to do that.

I make more tests and I realised the same stream saved as a file on sender (IPad-safari) is saved on portrait mode and on receiver (Ubuntu-Chrome) is saved on ladscape mode. Some weird orientation change is happening.

So it has to be something related with:
1-Metadata added or substracted on sended stream.
2-Codec incompability related to differences between Chrome and safari browsers type supporting on mediarecorder API.

Thanks.

[Michael L]

Hello,

I noticed that on our recordings, it doesn't seem that the webkit implementation follows the bitrate guidelines.

When initiating the MediaRecorder object we pass this options config object:

{
    bitsPerSecond: 2000000
}

In theory that should limit the recording to a maximum of 1953 kb/s.

We see on multiple occurences (eg on Safari 14.1.1 on iOS 14.6) that the output file has a much bigger bitrate (eg 9700 kb/s just for the video) which produces files that are hard to upload given the size. Is this a known issue ? Is there a workaround ? (like using videoBitsPerSecond instead of bitsPerSecond ?)

Thanks

[Michael L]

Hi Youenn,

Any news on this ? Should I open a separate ticket ?

Thanks

(In reply to Michael L from comment #92)
> Hello,
> 
> I noticed that on our recordings, it doesn't seem that the webkit
> implementation follows the bitrate guidelines.
> 
> When initiating the MediaRecorder object we pass this options config object:
> 
> {
>     bitsPerSecond: 2000000
> }
> 
> In theory that should limit the recording to a maximum of 1953 kb/s.
> 
> We see on multiple occurences (eg on Safari 14.1.1 on iOS 14.6) that the
> output file has a much bigger bitrate (eg 9700 kb/s just for the video)
> which produces files that are hard to upload given the size. Is this a known
> issue ? Is there a workaround ? (like using videoBitsPerSecond instead of
> bitsPerSecond ?)
> 
> Thanks

[youenn fablet]

(In reply to Michael L from comment #93)
> Hi Youenn,
> 
> Any news on this ? Should I open a separate ticket ?
> 
> Thanks

Yes a new ticket would be good.
WebKit currently supports audioBitsPerSecond and videoBitsPerSecond but not yet bitsPerSecond.

[Remus Negrota]

MediaRecorder.start() timeslice parameter is not consistent in the way it works on different macOS versions but the SAME Safari version.

Case and point, both scenarios were tested on Safari 14.1.2, with a timeslice parameter of 200.

1. On macOS 11.5.2 the recording works as expected.

2. On macOS 10.15.7 the recording is corrupted(it seems moov atom related). No matter how much you record, the recording stops playing back after 1-2 seconds.
The only way I could get it to work was to reduce the timeslice parameter to 10.

[Tim]

(In reply to youenn fablet from comment #94)
> (In reply to Michael L from comment #93)
> > Hi Youenn,
> > 
> > Any news on this ? Should I open a separate ticket ?
> > 
> > Thanks
> 
> Yes a new ticket would be good.
> WebKit currently supports audioBitsPerSecond and videoBitsPerSecond but not
> yet bitsPerSecond.

Has a ticket been created and where can I track it? I am interested to follow any progress on this. I am running into the same issue and even videoBitsPerSecond is being ignored for me.

[maxcodefaster]

https://bugs.webkit.org/show_bug.cgi?id=234784