250548 2023-01-13 00:19:29 -0800 Excessive 'timeupdate' events triggered by video element with HLS streams carrying in-band EMSG metadata 2023-01-20 09:40:23 -0800 1 1 1 Unclassified WebKit Media Safari 16 Mac (Apple Silicon) macOS 13 RESOLVED DUPLICATE 250741 P2 Normal --- 1 jameson.steiner webkit-unassigned ahmad.saleem792 eric.carlson jameson.steiner jer.noble oldest_to_newest 1925261 0 464479 jameson.steiner 2023-01-13 00:19:29 -0800 Created attachment 464479 HTML page loading an HLS stream carrying EMSG metadata into a video element and enabling the metadata track. Logging out 'timeupdate' events during playback to observe the issue. When playing HLS streams carrying EMSG in-band metadata, excessive/duplicate 'timeupdate' events would be fired by the video element if the respective metadata track is enabled. Observations: 1. If the metadata track's 'mode' property is set to 'disabled' (default) the browser does not process the track at all as per the API spec and 'timeupdate' events from the video element are observed in normal 250ms intervals. 2. Once the track is enabled by setting its mode to 'hidden', excessive 'timeupdate' events are triggered by the video element. Looking at the timing of the events it looks like some of them are triggered at the exact same time, i.e. are duplicates. This causes event handlers on the 'timeupdate' event to be called more often and thereby also causes more load on the device. In the attached reproduction case, around twice as many events are fired if the metadata track is enabled. Furthermore, with a stream of one of our customers that also carries a lot of EMSG metadata (but we cannot share publically), we are seeing around 750 'timeupdate' events within a 10-second period of playback if the metadata track is enabled vs. 40 events if the track is disabled. Additionally, with that stream, we are seeing CPU utilization and memory consumption steadily rising during playback up to a point (after around 1 hr) that Safari shows a banner saying "This web page is using significant memory. Closing it may improve the responsiveness of your Mac." and the page becomes very unresponsive. This is also only observed if the metadata track is enabled. Reproduction case: Attached is a simple HTML page loading an HLS stream that carries EMSG metadata into a video element and enabling the metadata track. Start playback and observe logged 'timeupdate' events in the browser console during playback. After 15 seconds playback is finished and the total count of received 'timeupdate' events is logged out. Different behavior can be observed depending on whether the metadata track is enabled or not. 1926156 1 ahmad.saleem792 2023-01-17 17:34:06 -0800 Marking this as duplicate to bug 250741, where I will fix it after discussing this with Eric. *** This bug has been marked as a duplicate of bug 250741 *** 1926934 2 ahmad.saleem792 2023-01-20 09:40:23 -0800 Just to confirm, I built WebKit from Trunk (today - so the patch has already landed) and run the testcase in mini browser and observed via Console and there is no duplicate event and there were 60 total time update event in Console. https://bug-250548-attachments.webkit.org/attachment.cgi?id=464479:22:17: CONSOLE WARN 2023-01-20T17:37:57.947Z - ended fired, got 60 timeupdate events in total So I think the duplicate issue seems to be fixed now after bug bug 250741. Thanks! 464479 2023-01-13 00:19:29 -0800 2023-01-13 00:19:29 -0800 HTML page loading an HLS stream carrying EMSG metadata into a video element and enabling the metadata track. Logging out 'timeupdate' events during playback to observe the issue. hls-video-element.html text/html 1139 jameson.steiner PCFET0NUWVBFIGh0bWw+CjxodG1sIGxhbmc9ImVuIj4KPGhlYWQ+CiAgICA8bWV0YSBjaGFyc2V0 PSJVVEYtOCI+CiAgICA8bWV0YSBodHRwLWVxdWl2PSJYLVVBLUNvbXBhdGlibGUiIGNvbnRlbnQ9 IklFPWVkZ2UiPgogICAgPG1ldGEgbmFtZT0idmlld3BvcnQiIGNvbnRlbnQ9IndpZHRoPWRldmlj ZS13aWR0aCwgaW5pdGlhbC1zY2FsZT0xLjAiPgogICAgPHRpdGxlPkRvY3VtZW50PC90aXRsZT4K PC9oZWFkPgo8Ym9keT4KICAgIDx2aWRlbyBoZWlnaHQ9NzIwIGNvbnRyb2xzIG11dGVkPjwvdmlk ZW8+CjwvYm9keT4KPHNjcmlwdD4KICBsZXQgdGltZVVwZGF0ZUNvdW50ID0gMDsKICBjb25zdCB2 aWRlbyA9IGRvY3VtZW50LnF1ZXJ5U2VsZWN0b3IoJ3ZpZGVvJyk7CgogIHZpZGVvLmFkZEV2ZW50 TGlzdGVuZXIoJ3RpbWV1cGRhdGUnLCAoZSkgPT4gewogICAgdGltZVVwZGF0ZUNvdW50Kys7CiAg ICBjb25zb2xlLndhcm4oYCR7bmV3IERhdGUoKS50b0lTT1N0cmluZygpfSAtIHRpbWV1cGRhdGUg ZmlyZWRgKTsKICB9KTsKCiAgdmlkZW8uYWRkRXZlbnRMaXN0ZW5lcignZW5kZWQnLCAoZSkgPT4g ewogICAgY29uc29sZS53YXJuKGAke25ldyBEYXRlKCkudG9JU09TdHJpbmcoKX0gLSBlbmRlZCBm aXJlZCwgZ290ICR7dGltZVVwZGF0ZUNvdW50fSB0aW1ldXBkYXRlIGV2ZW50cyBpbiB0b3RhbGAp OwogIH0pOwoKICB2aWRlby50ZXh0VHJhY2tzLmFkZEV2ZW50TGlzdGVuZXIoJ2FkZHRyYWNrJywg KGUpID0+IHsKICAgIGNvbnN0IHRyYWNrID0gZS50cmFjazsKCiAgICBpZiAodHJhY2sua2luZCA9 PT0gJ21ldGFkYXRhJykgewogICAgICAvLyBFbmFibGUgcHJvY2Vzc2luZyBvZiBtZXRhZGF0YSB0 cmFjawogICAgICB0cmFjay5tb2RlID0gJ2hpZGRlbic7CiAgICB9CiAgfSk7CgogIGNvbnN0IHNv dXJjZSA9IGRvY3VtZW50LmNyZWF0ZUVsZW1lbnQoJ3NvdXJjZScpOwogIHNvdXJjZS5zcmMgPSAn aHR0cHM6Ly9zdG9yYWdlLmdvb2dsZWFwaXMuY29tL2RsZXZpbi90bXAvZm1wNC1lbXNnLWlkMy8x NTY3MTkxNDA5L21hc3Rlci5tM3U4JzsKICBzb3VyY2UudHlwZSA9ICdhcHBsaWNhdGlvbi94LW1w ZWdVUkwnOwoKICB2aWRlby5hcHBlbmRDaGlsZChzb3VyY2UpOwo8L3NjcmlwdD4KPC9odG1sPgo=