Bug 162366 – Allow multiple playing videos on a page with 'autoplay' and 'playsinline' attributes

• Component: Media
• Status: RESOLVED FIXED

In a recent blogpost on the Webkit blog (https://webkit.org/blog/6784/new-video-policies-for-ios/) the 'playsinline' attribute, in combination with the 'autoplay' attribute is named as a way to replace the animated GIF format. Animated GIFs are not the best format for moving images because they tend to be a lot bigger than comparable MP4 files and have a maximum of 256 colors.

Unfortunately, the current implementation of playsinline/autoplay on iOS Safari makes it very hard to use the <video> tag as a GIF replacement.

Characteristics of GIFs are that they're muted, looping, autoplaying and inline. Crucially for this bug report, there can be many GIFs on the same page playing page. Unfortunately, iOS Safari doesn't seem able to play more than one <video> element at the same time (see https://twitter.com/jernoble/status/778619308385509377). Of course, this wasn't very noticeable given that inline video was impossible on the iPhone anyway. However, when using the new playsinline/autoplay combination on more than one video on a page this becomes noticeable immediately given that only one video will start playing. The other ones *can't even play* because the play button is disabled on 'playsinline' videos.

As a web developer, for anything else than a really basic usecase ,this is very frustrating. The only way i can imagine to get GIF-like behaviour with the current implementation is to have some Javascript running to check whether the GIF is in the viewport and then toggle the current playing video, which is pretty complex and error-prone compared to just using regular GIF images.

A testcase with two videos is available here: http://codepen.io/hay/pen/xERaVB

[Radar WebKit Bug Importer]

<rdar://problem/28639600>

[Jer Noble]

This comes down to the following line in MediaSessionManagerIOS.mm:
156:    addRestriction(PlatformMediaSession::Video, ConcurrentPlaybackNotPermitted);

We add a "no concurrent playback" restriction for all <video> elements. We'll need to come up with something more dynamic now that inline autoplaying of silent <video> elements is allowed.

[Ricky Blaha]

+1

We're trying to use the playsinline attribute and the new muted scripted autoplay at AOL Platforms for ads, but for iOS may not be able to since video can't reliably autoplay when either the publisher page or another ad also has an autoplay video.

Android Chrome 53 allows inline autoplay of multiple videos concurrently.

[Evol Greaves]

+2.

JW Player's publishers have been clamoring for this feature.

[João Machado]

(In reply to comment #3)
> +1
> 
> We're trying to use the playsinline attribute and the new muted scripted
> autoplay at AOL Platforms for ads, but for iOS may not be able to since
> video can't reliably autoplay when either the publisher page or another ad
> also has an autoplay video.
> 
> Android Chrome 53 allows inline autoplay of multiple videos concurrently.

+1

[Jer Noble]

Created attachment 293166 [details]
Patch

[Eric Carlson]

Comment on attachment 293166 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=293166&action=review

> Source/WebCore/platform/audio/ios/MediaSessionManagerIOS.mm:156
> -    addRestriction(PlatformMediaSession::Video, ConcurrentPlaybackNotPermitted);
> +    removeRestriction(PlatformMediaSession::Video, ConcurrentPlaybackPermitted);

There is no ConcurrentPlaybackPermitted flag, only ConcurrentPlaybackNOTPermitted. Did you mean to add a new flag or is this backwards?

> Source/WebCore/platform/audio/ios/MediaSessionManagerIOS.mm:159
> +    addRestriction(PlatformMediaSession::AudioVideo, ConcurrentPlaybackPermitted);

Ditto.

[Jer Noble]

Created attachment 293171 [details]
Patch

[Jer Noble]

(In reply to comment #7)
> Comment on attachment 293166 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=293166&action=review
> 
> > Source/WebCore/platform/audio/ios/MediaSessionManagerIOS.mm:156
> > -    addRestriction(PlatformMediaSession::Video, ConcurrentPlaybackNotPermitted);
> > +    removeRestriction(PlatformMediaSession::Video, ConcurrentPlaybackPermitted);
> 
> There is no ConcurrentPlaybackPermitted flag, only
> ConcurrentPlaybackNOTPermitted. Did you mean to add a new flag or is this
> backwards?

Nope, just fat-fingered. I fixed this locally, but the change didn't make it into the patch. New version uploaded.

> > Source/WebCore/platform/audio/ios/MediaSessionManagerIOS.mm:159
> > +    addRestriction(PlatformMediaSession::AudioVideo, ConcurrentPlaybackPermitted);
> 
> Ditto.

[Build Bot]

Comment on attachment 293171 [details]
Patch

Attachment 293171 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/2393829

New failing tests:
media/video-background-playback.html

[Build Bot]

Created attachment 293179 [details]
Archive of layout-test-results from ews104 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5

[Build Bot]

Comment on attachment 293171 [details]
Patch

Attachment 293171 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/2394030

New failing tests:
media/video-background-tab-playback.html
media/video-background-playback.html

[Build Bot]

Created attachment 293180 [details]
Archive of layout-test-results from ews103 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Jer Noble]

(In reply to comment #13)
> Created attachment 293180 [details]
> Archive of layout-test-results from ews103 for mac-yosemite
> 
> The attached test failures were seen while running run-webkit-tests on the
> mac-ews.
> Bot: ews103  Port: mac-yosemite  Platform: Mac OS X 10.10.5

Will fix these tests before landing.

[Build Bot]

Comment on attachment 293171 [details]
Patch

Attachment 293171 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/2394411

New failing tests:
media/video-background-playback.html

[Build Bot]

Created attachment 293196 [details]
Archive of layout-test-results from ews113 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-yosemite  Platform: Mac OS X 10.10.5

[Jer Noble]

Created attachment 293261 [details]
Patch for landing

[WebKit Commit Bot]

Comment on attachment 293261 [details]
Patch for landing

Clearing flags on attachment: 293261

Committed r208149: <http://trac.webkit.org/changeset/208149>

[Jason Ormand]

Hey, Jason Ormand from Vox Media here. Thanks for all the work on this issue thus far!

It appears there is traction and possibly a patch in the queue. But I wanted to +1 this issue and ensure it's getting plenty of attention.

This feature/bug is preventing us from shipping GIFs as videos to all of our sites. We can obviously save users torrents of bytes over the wire by using videos instead of GIFs. But we really need this to work. Using something like intersection observer/shim might be possible but we will definitely have more than one video in the viewport at a time so it would be impossible to fully emulate a GIF-like experience.

I look forward to throwing the ON switch for this new feature and saving the internet and iPhone users many terabytes of data, much battery and giving smoother playback.

Happy to assist if I can : )

[w0ts0n]

Wanted to weigh in and say that I'm running into this bug. I want to convert my site from gifs to html5 video and currently the first video pauses. 

Hopefully you can land that patch soon.

[Jer Noble]

(In reply to comment #20)
> Wanted to weigh in and say that I'm running into this bug. I want to convert
> my site from gifs to html5 video and currently the first video pauses. 
> 
> Hopefully you can land that patch soon.

The patch is landed, and if you have a developer.apple.com account, you can follow the instructions at <https://webkit.org/blog/3457/building-webkit-for-ios-simulator/> to build WebKit for the iOS Simulator and test the new behavior against your site.  It would be great to verify that this patch fixes the behavior you're seeing before the fix comes to an iOS update.

[w0ts0n]

I ran the instructions to build WebKit for the iOS Simulator and I'm still running into the same issue on the simulator.

Release-iphonesimulator.
com.apple.mobilesafari: 31123
webkit version reported: 602.3.12
Tested on: http://codepen.io/hay/pen/xERaVB

Let me know if I missed something.

[Jer Noble]

(In reply to comment #22)
> I ran the instructions to build WebKit for the iOS Simulator and I'm still
> running into the same issue on the simulator.
> 
> Release-iphonesimulator.
> com.apple.mobilesafari: 31123
> webkit version reported: 602.3.12
> Tested on: http://codepen.io/hay/pen/xERaVB
> 
> Let me know if I missed something.

I suspect you're still running against the shipping WebKit framework rather than your built framework, as 602.3.12 is a pretty old tag. Tip-of-tree should be 604.1.3 or so. If you jump on #webkit, we may be able to help you get your configuration sorted to help verify this.

[w0ts0n]

Got help in #webkit and confirmed the patch works as expected. 
R+

[Blaze Burg]

Moving to FIXED, patch has landed.

[Kilian Pauwels]

Is there an estimate on when this will hit ios update?

[Jon Lee]

In the latest iOS update.

Can confirm my testcase is working on the current 10.3.1 iOS version. Thanks a lot for fixing this guys!

[Ali G]

Is there a limit on the maximum number of concurrent playing in-line videos? If so, is the max static or hardware specific?

[Jer Noble]

(In reply to Ali G from comment #29)
> Is there a limit on the maximum number of concurrent playing in-line videos?
> If so, is the max static or hardware specific?

Hardware specific.

[Ali G]

Thanks, I am noticing videos that played but got paused still count toward this max. Is that be design? Is there an alternative to `pause` that would remove it from the queue? (other than destructing it which means user will lose their position in the video if they go back to it)

[Jer Noble]

(In reply to Ali G from comment #31)
> Thanks, I am noticing videos that played but got paused still count toward
> this max. Is that be design? Is there an alternative to `pause` that would
> remove it from the queue? (other than destructing it which means user will
> lose their position in the video if they go back to it)

If the video is merely paused, it's still using up hardware decode resources. The only way to free all these up is to set the src= to the empty string and call .load().  It will have the side effect you mentioned.

[Ali G]

Thanks for clarifying Jer.

[Nikhil Carmatec]

I'd like to use mp4s interspersed with text on my blog, like gifs. iOS 10 now allows the playsinline attribute on video tags, which works great.

However, when another video comes into the viewport and starts playing it pauses any other video. On the iPad this is confusing because there are often more than one video elements visible.

I've added a click handler to play a video when it is tapped. In this case too, playing one video pauses the others. Is there any way to play more than one video at a time in iOS Safari / Webkit?

web developer @ https://www.railscarma.com

(In reply to Nikhil Carmatec from comment #39)
> 
> I'd like to use mp4s interspersed with text on my blog, like gifs. iOS 10
> now allows the playsinline attribute on video tags, which works great.
> 
> However, when another video comes into the viewport and starts playing it
> pauses any other video. On the iPad this is confusing because there are
> often more than one video elements visible.
> 
> I've added a click handler to play a video when it is tapped. In this case
> too, playing one video pauses the others. Is there any way to play more than
> one video at a time in iOS Safari / Webkit?
> 
> web developer @ https://www.railscarma.com

Are you sure you're using iOS 10.3.1 or higher? If so, i think a test in codepen.io or something could be useful for reproduction.

I guess he didn't have the 'muted' attribute set.

I created this to show the behaviour:
http://curious-electric.com/_temp/video-bug/

Playing one video stops the other one.
Once you mute them, both can be played.

I still consider this a bug, but it seems this is the desired behaviour, right?

[Jer Noble]

(In reply to dirkk0 from comment #41)
> I guess he didn't have the 'muted' attribute set.
> 
> I created this to show the behaviour:
> http://curious-electric.com/_temp/video-bug/
> 
> Playing one video stops the other one.
> Once you mute them, both can be played.
> 
> I still consider this a bug, but it seems this is the desired behaviour,
> right?

Yes, we consider this the intended behavior.  If you disagree, please file a new bug detailing your use case, and we'll look into it.

Thanks for clarifying this Jer, honestly.
I won't file a separate bug unless you really want me to.

I think your clarification is important because
a) every web dev can send this comment to their clients saying 'I know it works on most other systems, but here you go.'
b) also I created a feeble workaround:
http://curious-electric.com/_temp/video-bug/workaround.html

Cheers,
  Dirk

[Jer Noble]

(In reply to dirkk0 from comment #43)
> Thanks for clarifying this Jer, honestly.
> I won't file a separate bug unless you really want me to.
> 
> I think your clarification is important because
> a) every web dev can send this comment to their clients saying 'I know it
> works on most other systems, but here you go.'
> b) also I created a feeble workaround:
> http://curious-electric.com/_temp/video-bug/workaround.html
> 
> Cheers,
>   Dirk

Aha, I see from your workaround that you're playing two muted <video> elements simultaneously with two video-less <audio> elements, all of which are allowed to play simultaneously, by design.  What you're losing through this workaround is the frame-level-synchronization between the <audio> and <video> elements. There used to be a Web API called "media group" <https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/mediaGroup>, which allowed pages to "group" together multiple media elements to play, synchronized with each other. This API was removed from the HTML specification when Chrome removed their implementation.

Conceptually speaking, I think we'd be happy to consider allowing a "group" of un-muted <video> elements to play, simultaneous to one another and on the same timeline. But I'd really like to hear some use cases for such. So I'd like to reiterate that we really would like you to file a new bug with your use cases; we can then try to sketch out a solution and perhaps take those use cases to the standards groups.

Done:
https://bugs.webkit.org/show_bug.cgi?id=186605

Thanks, Jer