Bug 173141 – No VP8/VP9 Video Codec Support in WebRTC

• Component: WebRTC
• Status: RESOLVED CONFIGURATION CHANGED

[Stefan Fochler]

Overview:
I seems like Safari TP 33 has no support for the VP8 video codec, which would be very much desired for interoperability with Google Chrome and native applications that use Chromium's WebRTC library in order to not occupy the HW h264 encoders.
Also, VP8 should be supported by the WebRTC library used: https://webrtc.org/faq/#what-is-the-vp8-video-codec

Steps to Reproduce:
1. Open https://webrtc.github.io/samples/src/content/peerconnection/munge-sdp/
2. Click "Get media", "Create peer connection" & "Create offer"
3. See the generated SDP offer

Actual Result:
The offer includes support for h264, but no VP8/VP9

Expected Result:
Safari supports the VP8/VP9 video codec.

Build Date & Hardware:
Safari TP 33 on macOS 10.12.5

[Jeremy Noring]

I'm assuming https://bugs.webkit.org/show_bug.cgi?id=167257 is where VP8/9 were disabled. It would really be a shame for Webkit/Safari to come this close to having a full WebRTC implementation, only to neuter it at the last moment.  Only having H.264 is going to make serious WebRTC products *much* more difficult to implement.  Honestly I would love to move entirely over to H.264, but the reality is I still have a number of clients that don't support that payload and need to fallback to VP8.  

Also, for complicated products with SFUs, recording, plugin support for old browsers, and existing mobile support....it means interop with Safari just got pretty hard.

Lastly, VP8 is MTI to implement per the WebRTC spec: https://tools.ietf.org/html/rfc7742

[Tsahi Levent-Levi]

This is sad for so many reasons.

Without VP8 support, a lot of group party calling as well as recording services are going to break - and they are going to force the industry to move back to H.264 and later HEVC, instead of taking the route of royalty free codecs.

Not to mention the fact that this isn't following the WebRTC specification itself which mandates the use of both H.264 and VP8.

WebKit adding WebRTC without VP8 is a step backwards - not forwards. Doing it intentionally - that's just plain mean.

[youenn fablet]

(In reply to Stefan Fochler from comment #0)
> Overview:
> I seems like Safari TP 33 has no support for the VP8 video codec, which
> would be very much desired for interoperability with Google Chrome and
> native applications that use Chromium's WebRTC library in order to not
> occupy the HW h264 encoders.

Why would it be a goal to not use HW H264 encoders/decoders?

[Tsahi Levent-Levi]

(In reply to youenn fablet from comment #3)
> (In reply to Stefan Fochler from comment #0)
> > Overview:
> > I seems like Safari TP 33 has no support for the VP8 video codec, which
> > would be very much desired for interoperability with Google Chrome and
> > native applications that use Chromium's WebRTC library in order to not
> > occupy the HW h264 encoders.
> 
> Why would it be a goal to not use HW H264 encoders/decoders?

I don't think it is a goal not to use HW for encoders/decoders. It is a goal to have both VP8 and H.264 - with or without HW encoders/decoders.

[Stefan Fochler]

(In reply to Tsahi Levent-Levi from comment #4)
> (In reply to youenn fablet from comment #3)
> > (In reply to Stefan Fochler from comment #0)
> > > Overview:
> > > I seems like Safari TP 33 has no support for the VP8 video codec, which
> > > would be very much desired for interoperability with Google Chrome and
> > > native applications that use Chromium's WebRTC library in order to not
> > > occupy the HW h264 encoders.
> > 
> > Why would it be a goal to not use HW H264 encoders/decoders?
> 
> I don't think it is a goal not to use HW for encoders/decoders. It is a goal
> to have both VP8 and H.264 - with or without HW encoders/decoders.

Well, in general you all the other people in this thread are right: VP8 would be very important to have because many devices/software implementations support VP8.

But there is in fact one reason why we would prefer having VP8 support in Safari, and that is that the opposing (non-browser) client that is participating in WebRTC calls already uses the HW H264 encoder to encode a live video stream. This, in turn, means that the WebRTC H264 stream has to be encoded in software which we found to provide worse results compared to VP8 in both CPU load and quality.

[youenn fablet]

> But there is in fact one reason why we would prefer having VP8 support in
> Safari, and that is that the opposing (non-browser) client that is
> participating in WebRTC calls already uses the HW H264 encoder to encode a
> live video stream. This, in turn, means that the WebRTC H264 stream has to
> be encoded in software which we found to provide worse results compared to
> VP8 in both CPU load and quality.

This particular issue is related to Mac software H.264 encoder, right?

I guess this case is not an edge case, but is hopefully not very frequent either.
We hope WebKit implementation to have decent performances in the H.264 software case too.

It would be great if you could test WebKit implementation in your setup that is triggering the H.264 software encoder. It would also be great if you could share your use cases or information on how much you see the software H.264 encoder kicking in.

[PhistucK]

> But there is in fact one reason why we would prefer having VP8 support in
> Safari, and that is that the opposing (non-browser) client that is
> participating in WebRTC calls already uses the HW H264 encoder to encode a
> live video stream. This, in turn, means that the WebRTC H264 stream has to
> be encoded in software which we found to provide worse results compared to
> VP8 in both CPU load and quality.

Sorry, either you mistyped H.264 instead of VP8 somewhere here, or you comment does not make a lot of sense to me. Perhaps you meant "HW H264" to be "HW VP8"?

[PhistucK]

(In reply to PhistucK from comment #7)
> > But there is in fact one reason why we would prefer having VP8 support in
> > Safari, and that is that the opposing (non-browser) client that is
> > participating in WebRTC calls already uses the HW H264 encoder to encode a
> > live video stream. This, in turn, means that the WebRTC H264 stream has to
> > be encoded in software which we found to provide worse results compared to
> > VP8 in both CPU load and quality.
> 
> Sorry, either you mistyped H.264 instead of VP8 somewhere here, or you
> comment does not make a lot of sense to me. Perhaps you meant "HW H264" to
> be "HW VP8"?

Oh, or you meant that the browser will encode H.264 through software while the non-browser will encode it through hardware?

[Stefan Fochler]

I think I need to clarify what our exact problem is here, so let me try to explain myself:

1. Our native macOS application is used for live video production and it is therefore very important to deliver a stable, high-quality H264 video stream using the HW H264 encoder. This video can be streamed to Youtube, Facebook, recorded to disk etc.

2. WebRTC allows guests to call-in into shows produced using said software. Currently, we support Google Chrome based clients which use the VP8 codec to exchange video data. If we now want to support Safari (which we do want to), this would mean that the guest (using the Safari browser) enforces the use of H264 instead of VP8, because the later is not supported (hence this bug report).
Now, the guest can of course use its HW H264 encoder if its device provides one. But the host (running our native application) is already using its HW H264 encoder, so it needs to encode the WebRTC video stream in software.

3. When comparing both H264 and VP8 software encoders, we found the VP8 encoder to impose less load on the CPU and still provide better results, although I don't have any empirical measurements of this.

Because of our experiences and the particular requirement of simultaneously encoding multiple video streams, we would prefer having VP8 support in Safari.

[PhistucK]

I think you should file a radar issue for this as well, since WebKit is not the problem, but Safari specifically disabled VP8 support.

[Jeremy Noring]

Just to make it clear why we need this:

1. We have old WebRTC clients that do not support H.264 decode.
2. We have SFU software that does not have H.264 support (we record server-side)
3. We have mobile apps we created ourselves that currently do not support H.264

...what this means is the barrier to entry to provide Safari support is pretty high.  Our only "decent" option would be to transcode incoming streams on the MCU and/or release all new client apps that do have H.264 support.  

The former option sucks (the whole point of an SFU is to avoid re-encoding, which affects latency/quality/cpu load server-side).  The latter option simply isn't possible for us without many months of work.  Which basically means: we'll force a plug-in install in safari until VP8 is available. :(

It sucks this codec issue has to be some technological football between giant companies.  The losers are consumers and developers, and to what end, I don't know.

[youenn fablet]

> The former option sucks (the whole point of an SFU is to avoid re-encoding,
> which affects latency/quality/cpu load server-side).  The latter option
> simply isn't possible for us without many months of work.  Which basically
> means: we'll force a plug-in install in safari until VP8 is available. :(

If you happen to have all clients of a session with hardware H.264 support, VP8 is not the right choice obviously. It would be great for your users if your service could also handle that case, which might not be uncommon.

Also, while I can understand pushing for VP8 software decoding, I haven't yet heard about any clear benefit of VP8 encoding if hardware H264 encoding is available.

[Tsahi Levent-Levi]

If you happen to need to process media in your backend, or use WebRTC in a device that has no H.264 HW encoder (or one that does but where codec royalties were not paid for or paid other purposes), you end up needing VP8. Otherwise, costs can be considerably high for just the simple need of encoding.

That usually means that small companies and companies with freemium based business models will need to lean strongly towards VP8 or just decide to go open a startup in a totally different domain.

HW encoding is great for video codecs when you have it, but we can do without it if we must.

Being able to CHOOSE the codec I want, especially when one is royalty bearing and the other is royalty free has its own huge advantage.

It would be great to get some response in if and when VP8 is planned to be added (especially since it is an MTI codec in WebRTC) than to get questions here about why we would want VP8 when the iPhone supports HW encoding for H.264.

[Brady Eidson]

(In reply to Tsahi Levent-Levi from comment #13)
> 
> It would be great to get some response in if and when VP8 is planned to be
> added (especially since it is an MTI codec in WebRTC) than to get questions
> here about why we would want VP8 when the iPhone supports HW encoding for
> H.264.

bugs.webkit.org is not the appropriate venue to ask Apple when they plan on doing something, as Apple generally does not comment on future plans or products.

It is a great venue for discussing bugs, architecture, etc.

[Stefan Fochler]

Thanks to all the commenters who have provided their own explanation of why it is important for Safari to support VP8 in WebRTC.

I have also filed this bug in the Apple Bugreporter as I do understand that this is not an issue with WebKit itself, but with decisions made in the Safari team.

<radar://problem/32708637>

[youenn fablet]

> I have also filed this bug in the Apple Bugreporter as I do understand that
> this is not an issue with WebKit itself, but with decisions made in the
> Safari team.
> 
> <radar://problem/32708637>

Thanks!

[Art Matsak]

Even if we leave the SFU/recording issues aside, no VP8 support in Safari is still a big interop problem. As things stand right now, Google's WebRTC implementation on Android only supports hardware H.264 encoding and decoding, and with select chipsets only to boot. So if a particular Android device doesn't have a supported H.264 chip, it will only be capable of VP8/VP9 => call establishment with Safari will fail.

[Philipp Hancke]

To make interop issues worse, Safari has inherited this webrtc.org bug:
    https://bugs.chromium.org/p/webrtc/issues/detail?id=4957
i.e. a setRemoteDescription call with type=offer and only VP8 but no H264 will cause an exception. This has been an interop hump with Edge already and it is somewhat sad to see the lack of priority upstream. Workarounds exist at least.

[Jon Lee]

rdar://problem/32708637

[Jeremy Noring]

(In reply to youenn fablet from comment #12)
> 
> If you happen to have all clients of a session with hardware H.264 support,
> VP8 is not the right choice obviously. It would be great for your users if
> your service could also handle that case, which might not be uncommon.
> 
> Also, while I can understand pushing for VP8 software decoding, I haven't
> yet heard about any clear benefit of VP8 encoding if hardware H264 encoding
> is available.

I do not have all clients in a session with hardware H.264 support.  And correct me if I'm wrong, but the "right choice" is dictated by the *mandatory to implement* codecs in the WebRTC standard per https://tools.ietf.org/html/rfc7742, and not by concerns over hardware support.  In other words, no--webkit should not be deciding "the right choice" here because the right choice is either of the MTI codecs in WebRTC.

As it is, if webkit lacks VP8 support, then the features page should be updated to indicate that webkit has partial support of WebRTC.  There is no nice way to say it: Webkit currently is in blatant violation of rfc7742.

Lastly, the benefit is clear to me: if all clients implement these two codecs, then there are no significant interop issues that developers have to work around.

[Jeremy Noring]

(In reply to PhistucK from comment #10)
> I think you should file a radar issue for this as well, since WebKit is not
> the problem, but Safari specifically disabled VP8 support.

I don't believe this is incorrect.  When webkit (i.e. Safari Technical Preview, unless I'm looking at the wrong browser) creates an SDP offer, it advertises:

a=rtpmap:96 red/90000
a=rtpmap:98 ulpfec/90000
a=rtpmap:99 H264/90000

Webkit itself does not appear to support VP8, unless I'm misunderstanding something.  It isn't just a case of "apple disabled it," it's a case of "webkit never enabled it in the first place"

[Jeremy Noring]

*I don't believe this is correct

[Jeremy Noring]

...and not to beat a dead horse, but https://webkit.org/blog/8165/release-notes-for-safari-technology-preview-52/ effectively guarantees that many WebRTC implementations using plug-ins will be broken due to a lack of VP8 support.  Our _only_ option will be to drop support for webkit (and thus, safari) as a browser.

[Oscar Divorra]

Long story short: The lack of VP8 in Safari is a trouble and a headache for delivering good interoperable services and best experience to the user.

[Ben]

VP8 was added:
https://bugs.webkit.org/show_bug.cgi?id=189976

[Justin Uberti]

Regarding VP9 support, Safari Tech Preview 110 indicates WebRTC support for VP9
[https://developer.apple.com/safari/technology-preview/release-notes/], but
when calling createOffer, I don't see VP9 listed as a supported codec [https://webrtc.github.io/samples/src/content/peerconnection/create-offer/]. 

Haven't yet debugged to see what's happening here, is this a known issue?

[youenn fablet]

WebRTC VP9 is off by default currently.
It can be enabled through Develop menu -> Experimental Features -> WebRTC VP9 Codec.

[Justin Uberti]

Confirmed, when enabling that flag in minibrowser and restarting I do see VP9 in the generated SDP.

[Ben]

VP8/VP9 support include hardware acceleration?

[Justin Uberti]

Did VP9 support get dropped from iOS14 GA? Not seeing it on iPhone or iPad when using MediaSource.isTypeSupported, HTMLVideoElement.canPlayType or PeerConnection.createOffer.

[youenn fablet]

(In reply to Justin Uberti from comment #30)
> Did VP9 support get dropped from iOS14 GA? Not seeing it on iPhone or iPad
> when using MediaSource.isTypeSupported, HTMLVideoElement.canPlayType or
> PeerConnection.createOffer.

This is still an off-by-default experimental feature in both MacOS and iOS.

[Ben]

Is VP9 enabled in iOS 15?

[Ben]

I see VP9 in the SDP on iPadOS 15. Is it supported on all iOS 15 devices? Should this issue be closed?

[Ahmad Saleem]

Can this be closed now that we support VP8 and VP9 by mentioning any specific limitations? Thanks!