Bug 173141 - No VP8/VP9 Video Codec Support in WebRTC
Summary: No VP8/VP9 Video Codec Support in WebRTC
Status: NEW
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebCore Misc. (show other bugs)
Version: Safari Technology Preview
Hardware: Macintosh macOS 10.12
: P2 Normal
Assignee: Nobody
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2017-06-09 01:26 PDT by Stefan Fochler
Modified: 2017-08-14 06:11 PDT (History)
25 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Stefan Fochler 2017-06-09 01:26:24 PDT
Overview:
I seems like Safari TP 33 has no support for the VP8 video codec, which would be very much desired for interoperability with Google Chrome and native applications that use Chromium's WebRTC library in order to not occupy the HW h264 encoders.
Also, VP8 should be supported by the WebRTC library used: https://webrtc.org/faq/#what-is-the-vp8-video-codec

Steps to Reproduce:
1. Open https://webrtc.github.io/samples/src/content/peerconnection/munge-sdp/
2. Click "Get media", "Create peer connection" & "Create offer"
3. See the generated SDP offer

Actual Result:
The offer includes support for h264, but no VP8/VP9

Expected Result:
Safari supports the VP8/VP9 video codec.

Build Date & Hardware:
Safari TP 33 on macOS 10.12.5
Comment 1 Jeremy Noring 2017-06-09 14:11:10 PDT
I'm assuming https://bugs.webkit.org/show_bug.cgi?id=167257 is where VP8/9 were disabled. It would really be a shame for Webkit/Safari to come this close to having a full WebRTC implementation, only to neuter it at the last moment.  Only having H.264 is going to make serious WebRTC products *much* more difficult to implement.  Honestly I would love to move entirely over to H.264, but the reality is I still have a number of clients that don't support that payload and need to fallback to VP8.  

Also, for complicated products with SFUs, recording, plugin support for old browsers, and existing mobile support....it means interop with Safari just got pretty hard.

Lastly, VP8 is MTI to implement per the WebRTC spec: https://tools.ietf.org/html/rfc7742
Comment 2 Tsahi Levent-Levi 2017-06-10 22:30:55 PDT
This is sad for so many reasons.

Without VP8 support, a lot of group party calling as well as recording services are going to break - and they are going to force the industry to move back to H.264 and later HEVC, instead of taking the route of royalty free codecs.

Not to mention the fact that this isn't following the WebRTC specification itself which mandates the use of both H.264 and VP8.

WebKit adding WebRTC without VP8 is a step backwards - not forwards. Doing it intentionally - that's just plain mean.
Comment 3 youenn fablet 2017-06-11 07:55:29 PDT
(In reply to Stefan Fochler from comment #0)
> Overview:
> I seems like Safari TP 33 has no support for the VP8 video codec, which
> would be very much desired for interoperability with Google Chrome and
> native applications that use Chromium's WebRTC library in order to not
> occupy the HW h264 encoders.

Why would it be a goal to not use HW H264 encoders/decoders?
Comment 4 Tsahi Levent-Levi 2017-06-11 07:57:30 PDT
(In reply to youenn fablet from comment #3)
> (In reply to Stefan Fochler from comment #0)
> > Overview:
> > I seems like Safari TP 33 has no support for the VP8 video codec, which
> > would be very much desired for interoperability with Google Chrome and
> > native applications that use Chromium's WebRTC library in order to not
> > occupy the HW h264 encoders.
> 
> Why would it be a goal to not use HW H264 encoders/decoders?

I don't think it is a goal not to use HW for encoders/decoders. It is a goal to have both VP8 and H.264 - with or without HW encoders/decoders.
Comment 5 Stefan Fochler 2017-06-11 14:58:02 PDT
(In reply to Tsahi Levent-Levi from comment #4)
> (In reply to youenn fablet from comment #3)
> > (In reply to Stefan Fochler from comment #0)
> > > Overview:
> > > I seems like Safari TP 33 has no support for the VP8 video codec, which
> > > would be very much desired for interoperability with Google Chrome and
> > > native applications that use Chromium's WebRTC library in order to not
> > > occupy the HW h264 encoders.
> > 
> > Why would it be a goal to not use HW H264 encoders/decoders?
> 
> I don't think it is a goal not to use HW for encoders/decoders. It is a goal
> to have both VP8 and H.264 - with or without HW encoders/decoders.

Well, in general you all the other people in this thread are right: VP8 would be very important to have because many devices/software implementations support VP8.

But there is in fact one reason why we would prefer having VP8 support in Safari, and that is that the opposing (non-browser) client that is participating in WebRTC calls already uses the HW H264 encoder to encode a live video stream. This, in turn, means that the WebRTC H264 stream has to be encoded in software which we found to provide worse results compared to VP8 in both CPU load and quality.
Comment 6 youenn fablet 2017-06-11 15:10:35 PDT
> But there is in fact one reason why we would prefer having VP8 support in
> Safari, and that is that the opposing (non-browser) client that is
> participating in WebRTC calls already uses the HW H264 encoder to encode a
> live video stream. This, in turn, means that the WebRTC H264 stream has to
> be encoded in software which we found to provide worse results compared to
> VP8 in both CPU load and quality.

This particular issue is related to Mac software H.264 encoder, right?

I guess this case is not an edge case, but is hopefully not very frequent either.
We hope WebKit implementation to have decent performances in the H.264 software case too.

It would be great if you could test WebKit implementation in your setup that is triggering the H.264 software encoder. It would also be great if you could share your use cases or information on how much you see the software H.264 encoder kicking in.
Comment 7 PhistucK 2017-06-11 22:45:00 PDT
> But there is in fact one reason why we would prefer having VP8 support in
> Safari, and that is that the opposing (non-browser) client that is
> participating in WebRTC calls already uses the HW H264 encoder to encode a
> live video stream. This, in turn, means that the WebRTC H264 stream has to
> be encoded in software which we found to provide worse results compared to
> VP8 in both CPU load and quality.

Sorry, either you mistyped H.264 instead of VP8 somewhere here, or you comment does not make a lot of sense to me. Perhaps you meant "HW H264" to be "HW VP8"?
Comment 8 PhistucK 2017-06-11 22:47:55 PDT
(In reply to PhistucK from comment #7)
> > But there is in fact one reason why we would prefer having VP8 support in
> > Safari, and that is that the opposing (non-browser) client that is
> > participating in WebRTC calls already uses the HW H264 encoder to encode a
> > live video stream. This, in turn, means that the WebRTC H264 stream has to
> > be encoded in software which we found to provide worse results compared to
> > VP8 in both CPU load and quality.
> 
> Sorry, either you mistyped H.264 instead of VP8 somewhere here, or you
> comment does not make a lot of sense to me. Perhaps you meant "HW H264" to
> be "HW VP8"?

Oh, or you meant that the browser will encode H.264 through software while the non-browser will encode it through hardware?
Comment 9 Stefan Fochler 2017-06-12 01:49:28 PDT
I think I need to clarify what our exact problem is here, so let me try to explain myself:

1. Our native macOS application is used for live video production and it is therefore very important to deliver a stable, high-quality H264 video stream using the HW H264 encoder. This video can be streamed to Youtube, Facebook, recorded to disk etc.

2. WebRTC allows guests to call-in into shows produced using said software. Currently, we support Google Chrome based clients which use the VP8 codec to exchange video data. If we now want to support Safari (which we do want to), this would mean that the guest (using the Safari browser) enforces the use of H264 instead of VP8, because the later is not supported (hence this bug report).
Now, the guest can of course use its HW H264 encoder if its device provides one. But the host (running our native application) is already using its HW H264 encoder, so it needs to encode the WebRTC video stream in software.

3. When comparing both H264 and VP8 software encoders, we found the VP8 encoder to impose less load on the CPU and still provide better results, although I don't have any empirical measurements of this.

Because of our experiences and the particular requirement of simultaneously encoding multiple video streams, we would prefer having VP8 support in Safari.
Comment 10 PhistucK 2017-06-12 08:45:57 PDT
I think you should file a radar issue for this as well, since WebKit is not the problem, but Safari specifically disabled VP8 support.
Comment 11 Jeremy Noring 2017-06-12 09:54:26 PDT
Just to make it clear why we need this:

1. We have old WebRTC clients that do not support H.264 decode.
2. We have SFU software that does not have H.264 support (we record server-side)
3. We have mobile apps we created ourselves that currently do not support H.264

...what this means is the barrier to entry to provide Safari support is pretty high.  Our only "decent" option would be to transcode incoming streams on the MCU and/or release all new client apps that do have H.264 support.  

The former option sucks (the whole point of an SFU is to avoid re-encoding, which affects latency/quality/cpu load server-side).  The latter option simply isn't possible for us without many months of work.  Which basically means: we'll force a plug-in install in safari until VP8 is available. :(

It sucks this codec issue has to be some technological football between giant companies.  The losers are consumers and developers, and to what end, I don't know.
Comment 12 youenn fablet 2017-06-12 14:58:53 PDT
> The former option sucks (the whole point of an SFU is to avoid re-encoding,
> which affects latency/quality/cpu load server-side).  The latter option
> simply isn't possible for us without many months of work.  Which basically
> means: we'll force a plug-in install in safari until VP8 is available. :(

If you happen to have all clients of a session with hardware H.264 support, VP8 is not the right choice obviously. It would be great for your users if your service could also handle that case, which might not be uncommon.

Also, while I can understand pushing for VP8 software decoding, I haven't yet heard about any clear benefit of VP8 encoding if hardware H264 encoding is available.
Comment 13 Tsahi Levent-Levi 2017-06-12 21:13:47 PDT
If you happen to need to process media in your backend, or use WebRTC in a device that has no H.264 HW encoder (or one that does but where codec royalties were not paid for or paid other purposes), you end up needing VP8. Otherwise, costs can be considerably high for just the simple need of encoding.

That usually means that small companies and companies with freemium based business models will need to lean strongly towards VP8 or just decide to go open a startup in a totally different domain.

HW encoding is great for video codecs when you have it, but we can do without it if we must.

Being able to CHOOSE the codec I want, especially when one is royalty bearing and the other is royalty free has its own huge advantage.

It would be great to get some response in if and when VP8 is planned to be added (especially since it is an MTI codec in WebRTC) than to get questions here about why we would want VP8 when the iPhone supports HW encoding for H.264.
Comment 14 Brady Eidson 2017-06-13 00:02:51 PDT
(In reply to Tsahi Levent-Levi from comment #13)
> 
> It would be great to get some response in if and when VP8 is planned to be
> added (especially since it is an MTI codec in WebRTC) than to get questions
> here about why we would want VP8 when the iPhone supports HW encoding for
> H.264.

bugs.webkit.org is not the appropriate venue to ask Apple when they plan on doing something, as Apple generally does not comment on future plans or products.

It is a great venue for discussing bugs, architecture, etc.
Comment 15 Stefan Fochler 2017-06-13 01:09:33 PDT
Thanks to all the commenters who have provided their own explanation of why it is important for Safari to support VP8 in WebRTC.

I have also filed this bug in the Apple Bugreporter as I do understand that this is not an issue with WebKit itself, but with decisions made in the Safari team.

<radar://problem/32708637>
Comment 16 youenn fablet 2017-06-13 01:24:16 PDT
> I have also filed this bug in the Apple Bugreporter as I do understand that
> this is not an issue with WebKit itself, but with decisions made in the
> Safari team.
> 
> <radar://problem/32708637>

Thanks!
Comment 17 Art Matsak 2017-06-13 02:01:15 PDT
Even if we leave the SFU/recording issues aside, no VP8 support in Safari is still a big interop problem. As things stand right now, Google's WebRTC implementation on Android only supports hardware H.264 encoding and decoding, and with select chipsets only to boot. So if a particular Android device doesn't have a supported H.264 chip, it will only be capable of VP8/VP9 => call establishment with Safari will fail.
Comment 18 Philipp Hancke 2017-06-19 23:37:16 PDT
To make interop issues worse, Safari has inherited this webrtc.org bug:
    https://bugs.chromium.org/p/webrtc/issues/detail?id=4957
i.e. a setRemoteDescription call with type=offer and only VP8 but no H264 will cause an exception. This has been an interop hump with Edge already and it is somewhat sad to see the lack of priority upstream. Workarounds exist at least.