Bug 56459 – Implement the HTML5 Media Streaming API

• Component: DOM
• Status: RESOLVED CONFIGURATION CHANGED

[Leandro Graciá Gil]

Implement the videoconferencing spec proposed in http://www.whatwg.org/specs/web-apps/current-work/multipage/dnd.html#video-conferencing-and-peer-to-peer-communication .

[Eric Carlson]

That spec is very much a moving target still.

[Leandro Graciá Gil]

Sure, I understand it's a moving target as we have closely followed the relevant standards mailing lists. At the same time, Chromium is very interested in implementing and maintaining this spec and we're fully committed to keeping the WebKit code as close to the standard as possible.

I was wondering if you had any advice about this: do you feel it's premature to add this API to WebKit? Or would you be ok with us adding it already now and committing to keeping it up to date?

(In reply to comment #1)
> That spec is very much a moving target still.

[Eric Carlson]

(In reply to comment #2)
> Sure, I understand it's a moving target as we have closely followed the relevant standards mailing 
> lists. At the same time, Chromium is very interested in implementing and maintaining this spec 
> and we're fully committed to keeping the WebKit code as close to the standard as possible.
> 
> I was wondering if you had any advice about this: do you feel it's premature to add this API to WebKit? 
> Or would you be ok with us adding it already now and committing to keeping it up to date?
> 
  While I agree that this is an important feature, and I definitely think that there is nothing like real implementation experience to make sure an API does everything it should, I am concerned that shipping a feature before an API settles can have a bad effect on both the web and on WebKit's ability to implement the final spec. 

  We implemented and shipped <video> very early, but the element had been in the spec for more than six months and there had very extensive debate on the mailing list. We shipped it knowing that we *would* break existing content if the spec changed, and it indeed it has changed significantly several times since we shipped our first implementation - and we updated and broke existing content when it did. 

  Are you willing to commit to breaking early adopters when the spec changes? It will change.

  Because this is a JavaScript-only API at this point, wouldn't it be possible to implement this in an application and inject the JavaScript API into WebKit at runtime?

[Andrei Popescu]

Hi Eric!

Many thanks for the comments!

(In reply to comment #3)
> (In reply to comment #2)
> > Sure, I understand it's a moving target as we have closely followed the relevant standards mailing 
> > lists. At the same time, Chromium is very interested in implementing and maintaining this spec 
> > and we're fully committed to keeping the WebKit code as close to the standard as possible.
> > 
> > I was wondering if you had any advice about this: do you feel it's premature to add this API to WebKit? 
> > Or would you be ok with us adding it already now and committing to keeping it up to date?
> > 
>   While I agree that this is an important feature, and I definitely think that there is nothing like real implementation experience to make sure an API does everything it should, I am concerned that shipping a feature before an API settles can have a bad effect on both the web and on WebKit's ability to implement the final spec. 
>

Totally agreed. It is not in our intention to ship this API before it is ready but rather get the implementation experience you mentioned and provide feedback to the standardisation effort. We were thinking of keeping this API behind a 'webkit' prefix (and in Chromium, it would also be behind a runtime flag).
 
>   We implemented and shipped <video> very early, but the element had been in the spec for more than six months and there had very extensive debate on the mailing list. We shipped it knowing that we *would* break existing content if the spec changed, and it indeed it has changed significantly several times since we shipped our first implementation - and we updated and broke existing content when it did. 
> 
>   Are you willing to commit to breaking early adopters when the spec changes? It will change.
> 

Yep, it will definitely change. However, there have been precedents of APIs that WebKit implemented despite the standard being a moving target (e.g. IdexedDB). In these cases, using the 'webkit' prefix for the API method names made it clear to the early adopters that the API may change quite dramatically.

>   Because this is a JavaScript-only API at this point, wouldn't it be possible to implement this in an application and inject the JavaScript API into WebKit at runtime?

The API is meant to interact with existing elements such as <video> or APIs, while it's methods are meant to return types such as Blob. I am therefore not sure how feasible it would be to implement it as you suggest. Rather, I feel it would be more beneficial to try and implement it properly in WebKit, provide feedback to the standard and iterate.

What do you think?

All the best,
Andrei

[Eric Carlson]

(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > I was wondering if you had any advice about this: do you feel it's premature to add this API to WebKit? 
> > > Or would you be ok with us adding it already now and committing to keeping it up to date?
> > > 
> >   While I agree that this is an important feature, and I definitely think that there is nothing like real implementation experience to make sure an API does 
> > everything it should, I am concerned that shipping a feature before an API settles can have a bad effect on both the web and on WebKit's ability to 
> > implement the final spec. 
> >
> 
> Totally agreed. It is not in our intention to ship this API before it is ready but rather get the implementation experience you mentioned and provide feedback 
> to the standardisation effort. We were thinking of keeping this API behind a 'webkit' prefix (and in Chromium, it would also be behind a runtime flag).
> 
I think keeping it behind a 'webkit' prefix is definitely a requirement until the spec is much more mature.

> > Because this is a JavaScript-only API at this point, wouldn't it be possible to implement this in an application and inject the JavaScript API into WebKit 
> > at runtime?
> 
> The API is meant to interact with existing elements such as <video> or APIs, while it's methods are meant to return types such as Blob. I am therefore not 
> sure how feasible it would be to implement it as you suggest. Rather, I feel it would be more beneficial to try and implement it properly in WebKit, 
> provide feedback to the standard and iterate.
> 
Good point.

> What do you think?
> 
As I said, there is nothing like real implementation experience to help shape a new API, so I think this is a useful effort. As I also noted the spec is still very young and it will likely change after we have an implementation, but as long as we use a 'webkit' prefix and acknowledge that it is very likely we may have to break early adopters I don't have a problem with proceeding.

[Andrei Popescu]

(In reply to comment #5)
> (In reply to comment #4)
> > (In reply to comment #3)
> > > (In reply to comment #2)
> > > > I was wondering if you had any advice about this: do you feel it's premature to add this API to WebKit? 
> > > > Or would you be ok with us adding it already now and committing to keeping it up to date?
> > > > 
> > >   While I agree that this is an important feature, and I definitely think that there is nothing like real implementation experience to make sure an API does 
> > > everything it should, I am concerned that shipping a feature before an API settles can have a bad effect on both the web and on WebKit's ability to 
> > > implement the final spec. 
> > >
> > 
> > Totally agreed. It is not in our intention to ship this API before it is ready but rather get the implementation experience you mentioned and provide feedback 
> > to the standardisation effort. We were thinking of keeping this API behind a 'webkit' prefix (and in Chromium, it would also be behind a runtime flag).
> > 
> I think keeping it behind a 'webkit' prefix is definitely a requirement until the spec is much more mature.
> 
> > > Because this is a JavaScript-only API at this point, wouldn't it be possible to implement this in an application and inject the JavaScript API into WebKit 
> > > at runtime?
> > 
> > The API is meant to interact with existing elements such as <video> or APIs, while it's methods are meant to return types such as Blob. I am therefore not 
> > sure how feasible it would be to implement it as you suggest. Rather, I feel it would be more beneficial to try and implement it properly in WebKit, 
> > provide feedback to the standard and iterate.
> > 
> Good point.
> 
> > What do you think?
> > 
> As I said, there is nothing like real implementation experience to help shape a new API, so I think this is a useful effort. As I also noted the spec is still very young and it will likely change after we have an implementation, but as long as we use a 'webkit' prefix and acknowledge that it is very likely we may have to break early adopters I don't have a problem with proceeding.


Great, thank you! Any help reviewing this stuff will be greatly appreciated :)

All the best,
Andrei

[Leandro Graciá Gil]

Updating the bug platform settings (had the default ones).

[Leandro Graciá Gil]

Updating the bug platform settings (had the default ones). -- Fixed.

[Arun Patole]

Hi,

We are very much interested in this feature. We have started implementing it and would like to contribute for Gtk port.

Please let me know if the overall architecture/design documents are shared at any place? if the design is not published then I think it may be posted as a webkit blog and/or on a webkit-dev list. That way other people can also share thoughts. I had read old dev-list thread on <device> element implementation but I think design is now changed with the new video conferencing and p2p spec.

Thanks,
Arun

[Adam Bergkvist]

Hi Arun

We have developed a GStreamer backend for MediaStream and PeerConnection for the GTK port. However, it doesn’t fit in to well with the WebKit client based design currently used in WebKit since we are using WebCore platform abstractions for everything except the getUserMedia requests. The good thing is that we’re currently talking to Tommy Widenflycht (Google’s main contributor in this area) about this and he also believes that we should move over to a WebCore platform approach since it suits most platforms best. When this is in place, we plan to contribute our implementation for GTK.

[Arun Patole]

(In reply to comment #10)
> Hi Arun
> 
> We have developed a GStreamer backend for MediaStream and PeerConnection for the GTK port. However, it doesn’t fit in to well with the WebKit client based design currently used in WebKit since we are using WebCore platform abstractions for everything except the getUserMedia requests. The good thing is that we’re currently talking to Tommy Widenflycht (Google’s main contributor in this area) about this and he also believes that we should move over to a WebCore platform approach since it suits most platforms best. When this is in place, we plan to contribute our implementation for GTK.

Ok, Thanks Adam! Could you guys move your discussions to webkit-dev list please so that other people can also contribute to it.

[Adam Bergkvist]

(In reply to comment #11)
> Ok, Thanks Adam! Could you guys move your discussions to webkit-dev list please so that other people can also contribute to it.

I've created a bug to discuss this: http://webkit.org/b/67946

[Matt Diamond]

Any estimate on when this feature will be available to developers in experimental webkit/chrome builds?

[Nick Dolezal]

It's been four years, unfortunately there is still no getUserMedia() support in Safari.  This seems the most direct bug relating to that(?).  Is this something that is being worked on?  It seems to have been assigned to a Google engineer long ago, perhaps this should be marked unassigned instead, as AFAIK Chrome/Blink supports the Media Streaming API now.

[Adam Bergkvist]

Hi

There is a new effort to bring WebRTC to WebKit. I've created a fresh beta bug to track the progress (bug 143211).

[Anne van Kesteren]

Closing as this is no longer relevant.