Bug 123907 - HTMLMediaElement support for timed metadata
Summary: HTMLMediaElement support for timed metadata
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Enhancement
Assignee: Nobody
URL:
Keywords:
: 124034 (view as bug list)
Depends on:
Blocks:
 
Reported: 2013-11-06 11:35 PST by will
Modified: 2014-10-27 14:08 PDT (History)
8 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description will 2013-11-06 11:35:24 PST
Shoutcast and Icecast servers will push metadata as part of the streams containing titles and other information. Ideally, Webkit would handle this metadata and on receipt fire a JS event. This is useful for live streams where we want to react to song changes and such.
Comment 1 Eric Carlson 2013-11-08 16:00:07 PST
Some relevant information from bug 124034:

Apple iOS applications support the playback of MPEG-2 transport streams containing ID3 PES Streams for synchronized timed-metadata [1]. On iOS, it is thus possible to write native applications that use this timed metadata, such as in [2]. The HTML 5 specification provides a way to expose timed metadata to Web applications in the form of an HTML 5 TextTrack of kind 'metadata' [3]. It would be good to expose this ID3 metadata using the TexTrack to be able to reproduce the behavior of the native application in Web apps. 

Note that some work towards standardizing how to expose metadata tracks is being carried in a W3C community group [4].

[1] https://developer.apple.com/library/ios/documentation/AudioVideo/Conceptual/HTTP_Live_Streaming_Metadata_Spec/HTTP_Live_Streaming_Metadata_Spec.pdf
[2] http://jmacmullin.wordpress.com/2010/11/03/adding-meta-data-to-video-in-ios/
[3] http://www.w3.org/html/wg/drafts/html/master/embedded-content-0.html#timed-text-tracks
[4] http://www.w3.org/community/inbandtracks/
Comment 2 Eric Carlson 2013-11-08 16:00:29 PST
*** Bug 124034 has been marked as a duplicate of this bug. ***
Comment 3 Jeroen Wijering 2014-02-27 08:56:55 PST
+1

In addition to Shoutcast/Icecast, Wowza/USP and other streaming servers inject ID3 metadata. 

Song title/artist/info is a key use case, video recording timecode is another one. JW Player detects & exposes ID3 metadata on desktop browsers, but must rely on Safari on iOS.
Comment 4 Eric Carlson 2014-04-07 08:19:45 PDT
If anyone following this bug has specifics about how timed metadata is or will be used on the open web, you might want to weigh in on Chromium bug 224833002 [1], where a patch to implement the w3c DataCue interface is getting serious pushback with comments like "I'm not aware of a use case for DataCue for normal desktop or mobile browsers".

[1] https://codereview.chromium.org/224833002/
Comment 5 will 2014-04-07 09:17:52 PDT
I don't know how the code review process works, and I am loath to spam systems when not desired. So I'll attach a comment here describing a use case:

I work for a large public radio station, and we have quite a few live streams across our services. These streams contain in stream metadata which changes when the piece or show being played changes. With the fixes in these tickets, we can refresh the player display on change, rather than polling every 15 seconds or so across thousands of users.

A nice bonus with this is that we get high levels of accuracy for when a song/show changes and so can start to make the player UI handle much more granular changes, down to a few seconds.

On the other end, in the case of a classical symphony for example, you could have a piece which plays for an hour and then is followed by a series of 15 second promos. Right now, every player has to poll every 10 seconds for an hour to have any degree of accuracy for the following promos. Assuming 0.5k per poll and 300 concurrent listeners, we have about 50MB in wasted requests and the overhead of 108,000 unneeded http requests.

For this reason, we default our player to flash over html5. We want to prefer html5 and use Flash as a fallback, but this is a blocker. We are looking at alternatives, but they are inconsistent and brittle.
Comment 6 Eric Carlson 2014-10-27 14:08:06 PDT
This was enabled by the changes for bug 131799.