SoundCloud <audio> test failures

• Component: Media
• Status: ASSIGNED ---

[Eric Carlson]

SoundCloud has published an HTML 5 <audio> interop test.  Mac WebKit fails 8 tests:

1) Seeking to unbuffered position with seamless playback, http://areweplayingyet.org/support-seeking-unbuffered-position
2) Supports OGG format, http://areweplayingyet.org/support-ogg-format
3) Consistent timeupdate interval (15ms - 250ms) - http://areweplayingyet.org/support-consistent-timeupdate-interval
4) roperty "seeking" - http://areweplayingyet.org/prop-seeking
5) Property "seek able" - http://areweplayingyet.org/prop-seekable
6) Property "currentTime" - http://areweplayingyet.org/prop-currentTime
7) Property "buffered" - http://areweplayingyet.org/prop-buffered
8) Event "progress" - http://areweplayingyet.org/event-progress

None of the tests say why they fail, so we will have to investigate each to see what the problems are.

[Eric Carlson]

#3 fails on my machine because the interval between 'timeupdate' events is sometimes 251ms. This isn't surprising because the timer we use to post these events fires every 250ms, so we don't have any room for error. We should decrease the timer interval.

[Eric Carlson]

#7 fails because the test is incorrect. It fails if audio.buffered.length is 0 when 'havemetadata' fires, but that event fires when [1]: 

    The user agent has just determined the duration and dimensions 
    of the media resource and the text tracks are ready.

which does *not* necessarily mean that any media data has loaded

[1] http://dev.w3.org/html5/spec/the-iframe-element.html#event-media-loadedmetadata

[Eric Carlson]

#2 checks for OGG support, which we don't have with a default QuickTime install.

[Yves Van Goethem]

Hey guys, looks like you found our little test suite :)
It's not officially live yet, we will redesign it, polish the tests and make sure everything is solid and well documented, the announcement will be done on Wednesday.
In the meantime we appreciate your feedback, we are looking into fixing tests you mentioned and linking to webkit bugs.
If you like to help us you can create pull requests or open issues on our github repo:
https://github.com/soundcloud/areweplayingyet
https://github.com/soundcloud/areweplayingyet/issues

Thanks!

[Eric Carlson]

rdar://10425842

[Eric Carlson]

"Seeking to unbuffered position with seamless playback"[1] fails because the test is incorrect. 

The script sets preload = 'metadata, sets 'src', and seeks when the 'loadedmetadata' event fires:

    audio.addEventListener('loadedmetadata', function() {
      audio.currentTime = seekedTime = AWPY.sound.long.duration * 0.8;
    }, false);

    audio.preload = 'metadata';
    audio.src = AWPY.sound.long.stream_url();

This means that if the UA implements preload='metadata' correctly, *only* metadata will have been loaded when 'currentTime' is set. 

When the AVFoundation media engine is used, preload = 'metadata' is implemented by only allocating the AVAsset and literally only loading a file's metadata. This means that when the 'loadedmetadata' event fires, 'seekable()' returns a Range object with length 0. Step 7 of the seeking algorithm [2] says:

    If the (possibly now changed) new playback position is not 
    in one of the ranges given in the seekable attribute, then 
    let it be the position in one of the ranges given in the 
    seekable attribute that is the nearest to the new playback 
    position.

In other words, the seek time is clamped to the the seekable range. In this case, the causes the seek time to be clamped to 0 and no seek happens.

I think the test would be more reasonable if 'preload' was not set to 'metadata' and if the seek was requested when 'canplay' fires.

    audio.addEventListener('canplay', function() {
      audio.currentTime = seekedTime = AWPY.sound.long.duration * 0.8;
    }, false);

    audio.src = AWPY.sound.long.stream_url();



[1] http://areweplayingyet.org/support-seeking-unbuffered-position
[2] http://dev.w3.org/html5/spec/the-iframe-element.html#seeking

[Eric Carlson]

Test #3, "Consistent timeupdate interval (15ms - 250ms)", http://areweplayingyet.org/event-progress, is also incorrect. 

This test "fails" if two 'timeupate' event fire less than 15ms or more than 250ms apart:

    audio.addEventListener('timeupdate', function() {
      if (lastTime) {
        var now = new Date();
        if ((now - lastTime) < 15 || (now - lastTime) > 250) {
          finish(false);
        } else if (++count === 30) {
          finish(true);
        }
      }
      lastTime = new Date();
 
This is wrong because the spec says that the interval between events should be 150ms to 550ms [1]:

    While the load is not suspended (see below), every 350ms 
    (±200ms) or for every byte received, whichever is least frequent, 
    queue a task to fire a simple event named progress at the element.


[1] http://dev.w3.org/html5/spec/the-iframe-element.html#concept-media-load-resource

[Tomás Senart]

(In reply to comment #6)
> "Seeking to unbuffered position with seamless playback"[1] fails because the test is incorrect. 
> 
> The script sets preload = 'metadata, sets 'src', and seeks when the 'loadedmetadata' event fires:
> 
>     audio.addEventListener('loadedmetadata', function() {
>       audio.currentTime = seekedTime = AWPY.sound.long.duration * 0.8;
>     }, false);
> 
>     audio.preload = 'metadata';
>     audio.src = AWPY.sound.long.stream_url();
> 
> This means that if the UA implements preload='metadata' correctly, *only* metadata will have been loaded when 'currentTime' is set. 
> 
> When the AVFoundation media engine is used, preload = 'metadata' is implemented by only allocating the AVAsset and literally only loading a file's metadata. This means that when the 'loadedmetadata' event fires, 'seekable()' returns a Range object with length 0. Step 7 of the seeking algorithm [2] says:
> 
>     If the (possibly now changed) new playback position is not 
>     in one of the ranges given in the seekable attribute, then 
>     let it be the position in one of the ranges given in the 
>     seekable attribute that is the nearest to the new playback 
>     position.
> 
> In other words, the seek time is clamped to the the seekable range. In this case, the causes the seek time to be clamped to 0 and no seek happens.
> 
> I think the test would be more reasonable if 'preload' was not set to 'metadata' and if the seek was requested when 'canplay' fires.
> 
>     audio.addEventListener('canplay', function() {
>       audio.currentTime = seekedTime = AWPY.sound.long.duration * 0.8;
>     }, false);
> 
>     audio.src = AWPY.sound.long.stream_url();
> 
> 
> 
> [1] http://areweplayingyet.org/support-seeking-unbuffered-position
> [2] http://dev.w3.org/html5/spec/the-iframe-element.html#seeking

Hello! I tried to alter the test to what you suggest and it still fails. Why shouldn't the audio object be capable of seeking after loadedmetadata was triggered?

[Tomás Senart]

(In reply to comment #7)
> Test #3, "Consistent timeupdate interval (15ms - 250ms)", http://areweplayingyet.org/event-progress, is also incorrect. 
> 
> This test "fails" if two 'timeupate' event fire less than 15ms or more than 250ms apart:
> 
>     audio.addEventListener('timeupdate', function() {
>       if (lastTime) {
>         var now = new Date();
>         if ((now - lastTime) < 15 || (now - lastTime) > 250) {
>           finish(false);
>         } else if (++count === 30) {
>           finish(true);
>         }
>       }
>       lastTime = new Date();
> 
> This is wrong because the spec says that the interval between events should be 150ms to 550ms [1]:
> 
>     While the load is not suspended (see below), every 350ms 
>     (±200ms) or for every byte received, whichever is least frequent, 
>     queue a task to fire a simple event named progress at the element.
> 
> 
> [1] http://dev.w3.org/html5/spec/the-iframe-element.html#concept-media-load-resource

I think you confused http://areweplayingyet.org/support-consistent-timeupdate-interval with http://areweplayingyet.org/event-progress.

[Tomás Senart]

(In reply to comment #8)
> (In reply to comment #6)
> > "Seeking to unbuffered position with seamless playback"[1] fails because the test is incorrect. 
> > 
> > The script sets preload = 'metadata, sets 'src', and seeks when the 'loadedmetadata' event fires:
> > 
> >     audio.addEventListener('loadedmetadata', function() {
> >       audio.currentTime = seekedTime = AWPY.sound.long.duration * 0.8;
> >     }, false);
> > 
> >     audio.preload = 'metadata';
> >     audio.src = AWPY.sound.long.stream_url();
> > 
> > This means that if the UA implements preload='metadata' correctly, *only* metadata will have been loaded when 'currentTime' is set. 
> > 
> > When the AVFoundation media engine is used, preload = 'metadata' is implemented by only allocating the AVAsset and literally only loading a file's metadata. This means that when the 'loadedmetadata' event fires, 'seekable()' returns a Range object with length 0. Step 7 of the seeking algorithm [2] says:
> > 
> >     If the (possibly now changed) new playback position is not 
> >     in one of the ranges given in the seekable attribute, then 
> >     let it be the position in one of the ranges given in the 
> >     seekable attribute that is the nearest to the new playback 
> >     position.
> > 
> > In other words, the seek time is clamped to the the seekable range. In this case, the causes the seek time to be clamped to 0 and no seek happens.
> > 
> > I think the test would be more reasonable if 'preload' was not set to 'metadata' and if the seek was requested when 'canplay' fires.
> > 
> >     audio.addEventListener('canplay', function() {
> >       audio.currentTime = seekedTime = AWPY.sound.long.duration * 0.8;
> >     }, false);
> > 
> >     audio.src = AWPY.sound.long.stream_url();
> > 
> > 
> > 
> > [1] http://areweplayingyet.org/support-seeking-unbuffered-position
> > [2] http://dev.w3.org/html5/spec/the-iframe-element.html#seeking
> 
> Hello! I tried to alter the test to what you suggest and it still fails. Why shouldn't the audio object be capable of seeking after loadedmetadata was triggered?

So the test code was updated to reflect this. Thanks for the input. Webkit passes now.
I had to add audio.removeEventListener('canplay', arguments.callee, false); inside the canplay event handler because canplay is triggered again after the seek.

[Eric Carlson]

(In reply to comment #9)
> 
> I think you confused http://areweplayingyet.org/support-consistent-timeupdate-interval with http://areweplayingyet.org/event-progress.

Right you are - silly me!

[Tomás Senart]

(In reply to comment #11)
> (In reply to comment #9)
> > 
> > I think you confused http://areweplayingyet.org/support-consistent-timeupdate-interval with http://areweplayingyet.org/event-progress.
> 
> Right you are - silly me!

No problem! Referring to http://areweplayingyet.org/support-seeking-unbuffered-position:
Mozilla people are saying this on their bug tracker:
"Requiring the canplay event before seeking corresponds to HAVE_FUTURE_DATA.  It should be possible to seek as soon as you reach HAVE_METADATA (and receive the loadedmetadata event)."

It makes sense to me. What do you think?

[Tomás Senart]

Please continue discussion here: https://github.com/soundcloud/areweplayingyet/issues/28

[Yves Van Goethem]

Hey guys,
We made minors update to AWPY:
 - Introduced check for Opus support http://areweplayingyet.org/support-format-opus :)
 - Split the test 'support-seeking-unbuffered-position' into 3 tests to take into account different codecs and removed the former one. (you can read more about the motivations: https://github.com/soundcloud/areweplayingyet/pull/41)
  * http://areweplayingyet.org/support-seeking-unbuffered-position-ogg
  * http://areweplayingyet.org/support-seeking-unbuffered-position-mp3
  * http://areweplayingyet.org/support-seeking-unbuffered-position-aac

We will also introduce an equivalent test for Opus.
Or if you want to contribute you can propose a patch :)