Bug 162366 - Allow multiple playing videos on a page with 'autoplay' and 'playsinline' attributes
Summary: Allow multiple playing videos on a page with 'autoplay' and 'playsinline' att...
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: WebKit Nightly Build
Hardware: iPhone / iPad iOS 10
: P2 Normal
Assignee: Jer Noble
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2016-09-21 16:44 PDT by huskyr
Modified: 2021-01-19 10:04 PST (History)
45 users (show)

See Also:


Attachments
Patch (180.44 KB, patch)
2016-10-28 09:42 PDT, Jer Noble
no flags Details | Formatted Diff | Diff
Patch (180.44 KB, patch)
2016-10-28 10:12 PDT, Jer Noble
eric.carlson: review+
buildbot: commit-queue-
Details | Formatted Diff | Diff
Archive of layout-test-results from ews104 for mac-yosemite-wk2 (1.05 MB, application/zip)
2016-10-28 11:17 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews103 for mac-yosemite (1.04 MB, application/zip)
2016-10-28 11:51 PDT, Build Bot
no flags Details
Archive of layout-test-results from ews113 for mac-yosemite (1.72 MB, application/zip)
2016-10-28 13:24 PDT, Build Bot
no flags Details
Patch for landing (186.65 KB, patch)
2016-10-28 17:52 PDT, Jer Noble
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description huskyr 2016-09-21 16:44:27 PDT
In a recent blogpost on the Webkit blog (https://webkit.org/blog/6784/new-video-policies-for-ios/) the 'playsinline' attribute, in combination with the 'autoplay' attribute is named as a way to replace the animated GIF format. Animated GIFs are not the best format for moving images because they tend to be a lot bigger than comparable MP4 files and have a maximum of 256 colors.

Unfortunately, the current implementation of playsinline/autoplay on iOS Safari makes it very hard to use the <video> tag as a GIF replacement.

Characteristics of GIFs are that they're muted, looping, autoplaying and inline. Crucially for this bug report, there can be many GIFs on the same page playing page. Unfortunately, iOS Safari doesn't seem able to play more than one <video> element at the same time (see https://twitter.com/jernoble/status/778619308385509377). Of course, this wasn't very noticeable given that inline video was impossible on the iPhone anyway. However, when using the new playsinline/autoplay combination on more than one video on a page this becomes noticeable immediately given that only one video will start playing. The other ones *can't even play* because the play button is disabled on 'playsinline' videos.

As a web developer, for anything else than a really basic usecase ,this is very frustrating. The only way i can imagine to get GIF-like behaviour with the current implementation is to have some Javascript running to check whether the GIF is in the viewport and then toggle the current playing video, which is pretty complex and error-prone compared to just using regular GIF images.

A testcase with two videos is available here: http://codepen.io/hay/pen/xERaVB
Comment 1 Radar WebKit Bug Importer 2016-10-05 14:27:28 PDT
<rdar://problem/28639600>
Comment 2 Jer Noble 2016-10-06 09:17:05 PDT
This comes down to the following line in MediaSessionManagerIOS.mm:
156:    addRestriction(PlatformMediaSession::Video, ConcurrentPlaybackNotPermitted);

We add a "no concurrent playback" restriction for all <video> elements. We'll need to come up with something more dynamic now that inline autoplaying of silent <video> elements is allowed.
Comment 3 Ricky Blaha 2016-10-12 11:35:09 PDT
+1

We're trying to use the playsinline attribute and the new muted scripted autoplay at AOL Platforms for ads, but for iOS may not be able to since video can't reliably autoplay when either the publisher page or another ad also has an autoplay video.

Android Chrome 53 allows inline autoplay of multiple videos concurrently.
Comment 4 Evol Greaves 2016-10-17 15:46:03 PDT
+2.

JW Player's publishers have been clamoring for this feature.
Comment 5 João Machado 2016-10-20 09:08:14 PDT
(In reply to comment #3)
> +1
> 
> We're trying to use the playsinline attribute and the new muted scripted
> autoplay at AOL Platforms for ads, but for iOS may not be able to since
> video can't reliably autoplay when either the publisher page or another ad
> also has an autoplay video.
> 
> Android Chrome 53 allows inline autoplay of multiple videos concurrently.

+1
Comment 6 Jer Noble 2016-10-28 09:42:31 PDT
Created attachment 293166 [details]
Patch
Comment 7 Eric Carlson 2016-10-28 10:02:32 PDT
Comment on attachment 293166 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=293166&action=review

> Source/WebCore/platform/audio/ios/MediaSessionManagerIOS.mm:156
> -    addRestriction(PlatformMediaSession::Video, ConcurrentPlaybackNotPermitted);
> +    removeRestriction(PlatformMediaSession::Video, ConcurrentPlaybackPermitted);

There is no ConcurrentPlaybackPermitted flag, only ConcurrentPlaybackNOTPermitted. Did you mean to add a new flag or is this backwards?

> Source/WebCore/platform/audio/ios/MediaSessionManagerIOS.mm:159
> +    addRestriction(PlatformMediaSession::AudioVideo, ConcurrentPlaybackPermitted);

Ditto.
Comment 8 Jer Noble 2016-10-28 10:12:36 PDT
Created attachment 293171 [details]
Patch
Comment 9 Jer Noble 2016-10-28 10:13:12 PDT
(In reply to comment #7)
> Comment on attachment 293166 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=293166&action=review
> 
> > Source/WebCore/platform/audio/ios/MediaSessionManagerIOS.mm:156
> > -    addRestriction(PlatformMediaSession::Video, ConcurrentPlaybackNotPermitted);
> > +    removeRestriction(PlatformMediaSession::Video, ConcurrentPlaybackPermitted);
> 
> There is no ConcurrentPlaybackPermitted flag, only
> ConcurrentPlaybackNOTPermitted. Did you mean to add a new flag or is this
> backwards?

Nope, just fat-fingered. I fixed this locally, but the change didn't make it into the patch. New version uploaded.

> > Source/WebCore/platform/audio/ios/MediaSessionManagerIOS.mm:159
> > +    addRestriction(PlatformMediaSession::AudioVideo, ConcurrentPlaybackPermitted);
> 
> Ditto.
Comment 10 Build Bot 2016-10-28 11:17:50 PDT
Comment on attachment 293171 [details]
Patch

Attachment 293171 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/2393829

New failing tests:
media/video-background-playback.html
Comment 11 Build Bot 2016-10-28 11:17:53 PDT
Created attachment 293179 [details]
Archive of layout-test-results from ews104 for mac-yosemite-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-yosemite-wk2  Platform: Mac OS X 10.10.5
Comment 12 Build Bot 2016-10-28 11:51:55 PDT
Comment on attachment 293171 [details]
Patch

Attachment 293171 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/2394030

New failing tests:
media/video-background-tab-playback.html
media/video-background-playback.html
Comment 13 Build Bot 2016-10-28 11:51:59 PDT
Created attachment 293180 [details]
Archive of layout-test-results from ews103 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 14 Jer Noble 2016-10-28 12:21:02 PDT
(In reply to comment #13)
> Created attachment 293180 [details]
> Archive of layout-test-results from ews103 for mac-yosemite
> 
> The attached test failures were seen while running run-webkit-tests on the
> mac-ews.
> Bot: ews103  Port: mac-yosemite  Platform: Mac OS X 10.10.5

Will fix these tests before landing.
Comment 15 Build Bot 2016-10-28 13:24:40 PDT
Comment on attachment 293171 [details]
Patch

Attachment 293171 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/2394411

New failing tests:
media/video-background-playback.html
Comment 16 Build Bot 2016-10-28 13:24:44 PDT
Created attachment 293196 [details]
Archive of layout-test-results from ews113 for mac-yosemite

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-yosemite  Platform: Mac OS X 10.10.5
Comment 17 Jer Noble 2016-10-28 17:52:43 PDT
Created attachment 293261 [details]
Patch for landing
Comment 18 WebKit Commit Bot 2016-10-31 09:38:56 PDT
Comment on attachment 293261 [details]
Patch for landing

Clearing flags on attachment: 293261

Committed r208149: <http://trac.webkit.org/changeset/208149>
Comment 19 Jason Ormand 2016-11-21 22:04:33 PST
Hey, Jason Ormand from Vox Media here. Thanks for all the work on this issue thus far!

It appears there is traction and possibly a patch in the queue. But I wanted to +1 this issue and ensure it's getting plenty of attention.

This feature/bug is preventing us from shipping GIFs as videos to all of our sites. We can obviously save users torrents of bytes over the wire by using videos instead of GIFs. But we really need this to work. Using something like intersection observer/shim might be possible but we will definitely have more than one video in the viewport at a time so it would be impossible to fully emulate a GIF-like experience.

I look forward to throwing the ON switch for this new feature and saving the internet and iPhone users many terabytes of data, much battery and giving smoother playback.

Happy to assist if I can : )
Comment 20 w0ts0n 2017-01-08 22:26:10 PST
Wanted to weigh in and say that I'm running into this bug. I want to convert my site from gifs to html5 video and currently the first video pauses. 

Hopefully you can land that patch soon.
Comment 21 Jer Noble 2017-01-09 11:00:12 PST
(In reply to comment #20)
> Wanted to weigh in and say that I'm running into this bug. I want to convert
> my site from gifs to html5 video and currently the first video pauses. 
> 
> Hopefully you can land that patch soon.

The patch is landed, and if you have a developer.apple.com account, you can follow the instructions at <https://webkit.org/blog/3457/building-webkit-for-ios-simulator/> to build WebKit for the iOS Simulator and test the new behavior against your site.  It would be great to verify that this patch fixes the behavior you're seeing before the fix comes to an iOS update.
Comment 22 w0ts0n 2017-01-09 14:51:12 PST
I ran the instructions to build WebKit for the iOS Simulator and I'm still running into the same issue on the simulator.

Release-iphonesimulator.
com.apple.mobilesafari: 31123
webkit version reported: 602.3.12
Tested on: http://codepen.io/hay/pen/xERaVB

Let me know if I missed something.
Comment 23 Jer Noble 2017-01-10 00:31:22 PST
(In reply to comment #22)
> I ran the instructions to build WebKit for the iOS Simulator and I'm still
> running into the same issue on the simulator.
> 
> Release-iphonesimulator.
> com.apple.mobilesafari: 31123
> webkit version reported: 602.3.12
> Tested on: http://codepen.io/hay/pen/xERaVB
> 
> Let me know if I missed something.

I suspect you're still running against the shipping WebKit framework rather than your built framework, as 602.3.12 is a pretty old tag. Tip-of-tree should be 604.1.3 or so. If you jump on #webkit, we may be able to help you get your configuration sorted to help verify this.
Comment 24 w0ts0n 2017-01-18 18:24:32 PST
Got help in #webkit and confirmed the patch works as expected. 
R+
Comment 25 Blaze Burg 2017-02-21 10:00:03 PST
Moving to FIXED, patch has landed.
Comment 26 Kilian Pauwels 2017-03-29 02:28:23 PDT
Is there an estimate on when this will hit ios update?
Comment 27 Jon Lee 2017-03-30 15:42:55 PDT
In the latest iOS update.
Comment 28 huskyr 2017-04-08 04:01:00 PDT
Can confirm my testcase is working on the current 10.3.1 iOS version. Thanks a lot for fixing this guys!
Comment 29 Ali G 2017-08-11 08:30:39 PDT
Is there a limit on the maximum number of concurrent playing in-line videos? If so, is the max static or hardware specific?
Comment 30 Jer Noble 2017-08-11 10:21:37 PDT
(In reply to Ali G from comment #29)
> Is there a limit on the maximum number of concurrent playing in-line videos?
> If so, is the max static or hardware specific?

Hardware specific.
Comment 31 Ali G 2017-08-11 13:24:32 PDT
Thanks, I am noticing videos that played but got paused still count toward this max. Is that be design? Is there an alternative to `pause` that would remove it from the queue? (other than destructing it which means user will lose their position in the video if they go back to it)
Comment 32 Jer Noble 2017-08-11 13:37:12 PDT
(In reply to Ali G from comment #31)
> Thanks, I am noticing videos that played but got paused still count toward
> this max. Is that be design? Is there an alternative to `pause` that would
> remove it from the queue? (other than destructing it which means user will
> lose their position in the video if they go back to it)

If the video is merely paused, it's still using up hardware decode resources. The only way to free all these up is to set the src= to the empty string and call .load().  It will have the side effect you mentioned.
Comment 33 Ali G 2017-08-11 13:43:17 PDT
Thanks for clarifying Jer.
Comment 39 Nikhil Carmatec 2018-06-05 23:24:53 PDT

I'd like to use mp4s interspersed with text on my blog, like gifs. iOS 10 now allows the playsinline attribute on video tags, which works great.

However, when another video comes into the viewport and starts playing it pauses any other video. On the iPad this is confusing because there are often more than one video elements visible.

I've added a click handler to play a video when it is tapped. In this case too, playing one video pauses the others. Is there any way to play more than one video at a time in iOS Safari / Webkit?

web developer @ https://www.railscarma.com
Comment 40 huskyr 2018-06-06 04:23:19 PDT
(In reply to Nikhil Carmatec from comment #39)
> 
> I'd like to use mp4s interspersed with text on my blog, like gifs. iOS 10
> now allows the playsinline attribute on video tags, which works great.
> 
> However, when another video comes into the viewport and starts playing it
> pauses any other video. On the iPad this is confusing because there are
> often more than one video elements visible.
> 
> I've added a click handler to play a video when it is tapped. In this case
> too, playing one video pauses the others. Is there any way to play more than
> one video at a time in iOS Safari / Webkit?
> 
> web developer @ https://www.railscarma.com

Are you sure you're using iOS 10.3.1 or higher? If so, i think a test in codepen.io or something could be useful for reproduction.
Comment 41 dirkk0 2018-06-10 09:15:45 PDT
I guess he didn't have the 'muted' attribute set.

I created this to show the behaviour:
http://curious-electric.com/_temp/video-bug/

Playing one video stops the other one.
Once you mute them, both can be played.

I still consider this a bug, but it seems this is the desired behaviour, right?
Comment 42 Jer Noble 2018-06-11 11:09:26 PDT
(In reply to dirkk0 from comment #41)
> I guess he didn't have the 'muted' attribute set.
> 
> I created this to show the behaviour:
> http://curious-electric.com/_temp/video-bug/
> 
> Playing one video stops the other one.
> Once you mute them, both can be played.
> 
> I still consider this a bug, but it seems this is the desired behaviour,
> right?

Yes, we consider this the intended behavior.  If you disagree, please file a new bug detailing your use case, and we'll look into it.
Comment 43 dirkk0 2018-06-11 12:29:04 PDT
Thanks for clarifying this Jer, honestly.
I won't file a separate bug unless you really want me to.

I think your clarification is important because
a) every web dev can send this comment to their clients saying 'I know it works on most other systems, but here you go.'
b) also I created a feeble workaround:
http://curious-electric.com/_temp/video-bug/workaround.html

Cheers,
  Dirk
Comment 44 Jer Noble 2018-06-11 15:02:49 PDT
(In reply to dirkk0 from comment #43)
> Thanks for clarifying this Jer, honestly.
> I won't file a separate bug unless you really want me to.
> 
> I think your clarification is important because
> a) every web dev can send this comment to their clients saying 'I know it
> works on most other systems, but here you go.'
> b) also I created a feeble workaround:
> http://curious-electric.com/_temp/video-bug/workaround.html
> 
> Cheers,
>   Dirk

Aha, I see from your workaround that you're playing two muted <video> elements simultaneously with two video-less <audio> elements, all of which are allowed to play simultaneously, by design.  What you're losing through this workaround is the frame-level-synchronization between the <audio> and <video> elements. There used to be a Web API called "media group" <https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/mediaGroup>, which allowed pages to "group" together multiple media elements to play, synchronized with each other. This API was removed from the HTML specification when Chrome removed their implementation.

Conceptually speaking, I think we'd be happy to consider allowing a "group" of un-muted <video> elements to play, simultaneous to one another and on the same timeline. But I'd really like to hear some use cases for such. So I'd like to reiterate that we really would like you to file a new bug with your use cases; we can then try to sketch out a solution and perhaps take those use cases to the standards groups.
Comment 45 dirkk0 2018-06-13 12:55:21 PDT
Done:
https://bugs.webkit.org/show_bug.cgi?id=186605

Thanks, Jer