Bug 242059 – HTML5 Video tag, HLS VOD: duration increases as the chunks are loading

• Component: Media
• Status: NEW

Steps to reproduce:

1. Limit internet connection speed. I tested using 1 mbps down.
2. Create a stream using <video> tag (I just used the sandbox at https://www.w3schools.com/html/tryit.asp?filename=tryhtml5_video_all): 
```
<video width="640" height="400" controls="controls" src="https://multiplatform-f.akamaihd.net/i/multi/april11/sintel/sintel-hd_,512x288_450_b,640x360_700_b,768x432_1000_b,1024x576_1400_m,.mp4.csmil/master.m3u8"></video>
</video>
```
3. Observe the displayed video duration in video controls. Initially the duration is correct, but as the .ts start loading, the duration drops to a few seconds and slowly increases as more chunks get downloaded.

The expected result is that the duration is always equal to the duration of the entire VOD.
 
The issue could be reproduced on Safari on MacOS (12.4) as well as on iOS

[Radar WebKit Bug Importer]

<rdar://problem/96093024>

[Eknoor]

(In reply to vaerlux from comment #0)
> Steps to reproduce:
> 
> 1. Limit internet connection speed. I tested using 1 mbps down.
> 2. Create a stream using <video> tag (I just used the sandbox at
> https://www.w3schools.com/html/tryit.asp?filename=tryhtml5_video_all): 
> ```
> <video width="640" height="400" controls="controls"
> src="https://multiplatform-f.akamaihd.net/i/multi/april11/sintel/sintel-hd_,
> 512x288_450_b,640x360_700_b,768x432_1000_b,1024x576_1400_m,.mp4.csmil/master.
> m3u8"></video>
> </video>
> ```
> 3. Observe the displayed video duration in video controls. Initially the
> duration is correct, but as the .ts start loading, the duration drops to a
> few seconds and slowly increases as more chunks get downloaded.
> 
> The expected result is that the duration is always equal to the duration of
> the entire VOD.
>  
> The issue could be reproduced on Safari on MacOS (12.4) as well as on iOS

Thank you for the report. Can we get a new link for the playlist that reproduces the issue since the current one no longer appears to be available? 
Is the issue still reproducible on macOS 12.5?

[Radosław Włodkowski]

Created attachment 462972 [details]
bipbop HLS stream

[Radosław Włodkowski]

We stumbled upon the same issue and was able to reproduce it with the official bipbop HLS stream:
https://devstreaming-cdn.apple.com/videos/streaming/examples/bipbop_16x9/bipbop_16x9_variant.m3u8

feed directly to the <video> element (test html attached).

We analysed this thoroughly and was able to determine that:
 
The affected platforms (we were able to reproduce this problem):
- macOS Monterey 12.6 (21G115) on MacBook Pro (16-inch, 2021) with Apple M1 Max:
-- Safari 16.0 (17614.1.25.9.10, 17614)
-- Safari Technology Preview Release 155 (Safari 16.4, WebKit 17615.1.7.1)

- iOS 16.0.2 (20A380) on iPhone 13 Pro:
-- Mozilla/5.0 (iPhone; CPU iPhone OS 16_0_2 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.0 Mobile/15E148 Safari/604.1

- iPadOS 15.7 (19H12) on iPad Air 2:
-- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.6,2 Safari/605.1.15

The unaffected platforms (we were able to test and couldn't reproduce this issue): 
- macOS Big Sur 11.7 (20G817) on MacBook Pro (Retina, 15-inch, Mid 2014) with 2,5 GHz Quad-Core Intel Core i7 
-- Safari 16.0 (16614.1.25.9.10, 16614)
-- Safari Technology Preview Release 146 (Safari 15.4, WebKit 16614.1.14.10.6)

Based on the above we assume that could be more about the underlying AVFoundation implementation on ARM platform than Safari/WebKit or OS. But it's just a supposition. We don't have Monterey on Intel platform at our disposal at the moment. 

Findings

The issue is pronounced when we synthetically throttle the bandwidth to really low speed like 512kbps, so the ABR algorithm is forced to use lower bitrate variants. This means in our case the audio only variant. As we observed the difference between affected and unaffected platforms is that on the affected ones Safari after reload of the same video is taking the last observed bandwidth into account and on the unaffected ones it is not doing that. So it could take few reloads to reproduce this issue. 

We've made screencasts to show exactly what’s going on: 
- bipbop-duration-issue-screencast-macOS-Monterey-M1-Safari-16.mov - https://drive.google.com/file/d/1vmzKpe0_2Kp017hZMLn9f0S63floor0o/view?usp=sharing 
- bipbop-duration-noissue-screencast-macOS-BigSur-Intel-Safari-16.mov - https://drive.google.com/file/d/1EB2WtRvyTgwz76yWOsE5L8cSCv6sqc1O/view?usp=sharing

So as you can see the order of playlist and segments loading on ARM platform is different than on Intel. On Intel the algorithm is always loading some TS segments no matter what. On ARM if the bandwidth stays low, the ABR algorithm decides to load only aac - that's when the duration issue is triggered. 

Sometimes the duration at the beginning of playback is 30 min, but then immediately shortens and then grows. Sometimes there is no “blink” and the duration is short at the start and growing.

The Safari algorithm is also using the order of variants in the loaded playlist as a prioritisation mechanism, so if there is a higher bandwidth variant before the lowest one it will take that which appeared first no matter what. This is giving us an opportunity to fix this issue by just moving the audio only variant with lowest bandwidth before the audio/video variants. 

Another finding is about the difference in duration between audio only playlist and audio/video playlist. The audio only playlist has duration of ~29:59 minutes and it contains 180 segments. The audio/video playlists have duration of 30:00 minutes and they contain 181 segments. So there is discrepancy in duration and segments count. We found that if we increase the duration of the audio-only playlist by manipulating #EXTINF values so the duration is equal or greater than the duration of audio/video playlists, the issue is also no longer present. It seams that Safari is overzealous in this matter.

We believe that this behaviour is erroneous.

If we could provide any further details please let us know.

[Radosław Włodkowski]

Quick update:
- Safari 16.1 on macOS Ventura (M1) is still affected. 

It would be much appreciated if someone could take care of it. Thanks.

[Eric Carlson]

(In reply to Radosław Włodkowski from comment #5)
> Quick update:
> - Safari 16.1 on macOS Ventura (M1) is still affected. 
> 
> It would be much appreciated if someone could take care of it. Thanks.

This is a bug in one of the frameworks WebKit uses for media playback. It is being investigated.