NEW 284039
Unable to achieve frame perfect synchronization with Video Frames 
https://bugs.webkit.org/show_bug.cgi?id=284039
Summary Unable to achieve frame perfect synchronization with Video Frames
Artem Verkhovsky
Reported 2024-12-04 06:03:11 PST
Created attachment 473457 [details] An external hardware recording demonstrating the mismatch of frame count during playback of a 60 FPS MP4 video using requestVideoFrameCallback() Dear WebKit Developers, We are encountering a synchronization issue when trying to align our WebGL-based augmentations on a canvas element with DRM-encrypted HLS streams played in web browsers. Specifically, the requestVideoFrameCallback() method is not reliably firing for each new frame in Safari or other browsers. As we are working with DRM-encrypted sources, we cannot use the WebCodecs API to access frame data directly. The problem is not limited to DRM-encrypted videos but also occurs with regular videos. We tried using requestAnimationFrame() combined with video.currentTime to calculate the frame which is supposed to be on the screen. However, this method is not precise enough for frame-perfect synchronization. Some HLS streams we receive are encoded at 59.94 FPS due to legacy TV standards. Safari and other browsers occasionally duplicate frames to match screen refresh rates (e.g., 60Hz), introducing additional desynchronization challenges. Is there any method or event in the HTMLVideoElement API that can signal when a frame duplication occurs to align video and screen frame rates? We observed that frame duplication for 59.94 FPS streams happens in a rather ordered way in Safari, but we couldn’t allign our calculations to this duplication. Are there any alternative techniques you would recommend for achieving frame-perfect synchronization between a DRM-encrypted stream and canvas-based augmentations, considering the limitations of current APIs? Attached to this message is a recording captured with an external hardware demonstrating the mismatch of frame count during playback of a 60 FPS MP4 video using requestVideoFrameCallback(). Thank you for your time and any guidance you can provide. Best regards, Artem Verkhovskiy BrandBrigade
Attachments
An external hardware recording demonstrating the mismatch of frame count during playback of a 60 FPS MP4 video using requestVideoFrameCallback() (6.40 MB, video/x-matroska)
2024-12-04 06:03 PST, Artem Verkhovsky 		no flags
Details
View All Add attachment proposed patch, testcase, etc.
Radar WebKit Bug Importer
Comment 1 2024-12-11 06:04:14 PST
<rdar://problem/141280229>
Note You need to log in before you can comment on or make changes to this bug.