Bug 194986 – Incorrect initial video frame drawn to canvas with canvas.drawImage(video, ...)

• Component: Media
• Status: NEW

Created attachment 362846 [details]
Self contained test.

I'm trying to get Safari to seek and draw a single frame of video on a canvas element.

It seems impossible to reliably get anything other than the first frame of video.

The sample code below just tries to seek and draw a single frame but exposes at least 2 bugs:

1. Video element claims it has seeked to the target time but when drawImage is called it draws the first frame.

2. onseeked event may not even be called.


Although not in the code below, I've also tried various hacks like seeking multiple times to wildly different points in the video to try and force different behaviour. These work most of the time but there's still cases where the onseeked event is not called and everything falls apart.


Added a complete, self-contained repro as an attachment.

Typical log looks like:
[Log] onloadedmetadata (tmp.html, line 42)
[Log] onloadeddata: time:2 (tmp.html, line 58)
[Log] ontimeupdate: time:2 (tmp.html, line 66)
[Log] onseeked: time:2 (tmp.html, line 48)
[Log] updateCanvas: time:2 (tmp.html, line 71)
But updateCanvas renders the wrong frame.


When onseeked is not called, the log reads:
[Log] onloadedmetadata (tmp.html, line 42)
[Log] onloadeddata: time:0 (tmp.html, line 58)
and the browser does nothing else.




var canvas = null
var video = null

// BUG1: If we litter currentTime all over the place to try and get the right
// thing to happen, we find that onseeked is not called half the time.
// When set to false, only a single seek in onloadedmetadata is used. Even then
// onseeked is not called sporadically (<5% failure rate).
// 
let redundantSetCurrentTime = false

function startLoad() {
  if(redundantSetCurrentTime) {
    video.currentTime = 2
  }

  video.src = data

  if(redundantSetCurrentTime) {
    video.currentTime = 2
  }
}

function onloadedmetadata() {
  console.log("onloadedmetadata")

  video.currentTime = 2
}

// BUG: onseeked may not be called
function onseeked() {
  console.log(`onseeked: time:${video.currentTime}`)

  // BUG2: video is always rendered at time=0, not time=2
  updateCanvas()

  // BUG3(?): timer will not fire occasionally (comment out updateCanvas above)
  //setTimeout(updateCanvas, 500)
}

function onloadeddata() {
  console.log(`onloadeddata: time:${video.currentTime}`)
}

function onplaying() {
  console.log(`onplaying: time:${video.currentTime}`)
}

function ontimeupdate() {
  console.log(`ontimeupdate: time:${video.currentTime}`)
}


function updateCanvas() {
  console.log(`updateCanvas: time:${video.currentTime}`)
  canvas.width = video.videoWidth
  canvas.height = video.videoHeight
  let ctx = canvas.getContext('2d')
  // Inconsistent bug: browser goes away for 10 seconds or so in the drawImage
  ctx.drawImage(video, 0, 0)
}

function onload() {
  canvas = document.getElementById("canvas")
  video = document.createElement("video")

  video.onloadedmetadata = onloadedmetadata
  video.onloadeddata = onloadeddata
  video.onseeked = onseeked
  video.onplaying = onplaying
  video.ontimeupdate = ontimeupdate
  video.crossOrigin = 'anonymous'

  document.body.appendChild(video)

  startLoad()
}

Created attachment 363182 [details]
Second test using MSE, video not copied to canvas at all.

Added a second test using MediaSource where the video is not copied to the canvas at all. You should click the video element to copy the data to the canvas, on Chrome/Firefox this happens but not on Safari or Safari Tech preview.

[Radar WebKit Bug Importer]

<rdar://problem/48460630>