VideoEncoder with `latencyMode: "realtime"` but no `bitrate` or `framerate` specified does not output any frames, nor error messages. Related: https://bugs.webkit.org/show_bug.cgi?id=264893 & https://bugs.webkit.org/show_bug.cgi?id=258669 Steps to reproduce: 1. Visit https://codesandbox.io/s/cocky-ioana-pjyz5v?file=/src/index.mjs 2. Open the console tab and observe the output is "{keyFrames: 0, deltaFrames: 0}" -- i.e. no frames were output whatsoever. No error messages are present. 3. Uncomment `framerate: fps` and `bitrate: 100000` and observe that the output changes to "{keyFrames: 1, deltaFrames: 19}". Expected behavior: If encoding without these parameters is not supported, then the browser should not claim to support the configuration (via isConfigSupported and by not calling the error callback). Full code below: ``` import "./styles.css"; async function main() { const width = 200; const height = 100; const img = new ImageData(width, height); for (let r=0;r<height;r++) { for (let c=0;c<width;c++) { img.data[(r*width+c)*4+0] = Math.floor(r/height*255); img.data[(r*width+c)*4+1] = Math.floor(c/width*255); img.data[(r*width+c)*4+2] = 127; img.data[(r*width+c)*4+3] = 255; } } const bitmap = await createImageBitmap(img); let keyFrames=0; let deltaFrames=0; const encoder = new VideoEncoder({ output: (chunk, metadata) => { if (chunk.type==="key") { ++keyFrames; } else if (chunk.type === "delta") { ++deltaFrames; } console.log('output chunk',chunk,metadata); }, error: (err) => { console.error('encode error', err) } }) encoder.addEventListener("dequeue", (event) => { console.log('dequeued') }) const fps = 30; encoder.configure({ codec: "avc1.42001f", // Baseline profile (42 00) with level 3.1 (1f) width, height, latencyMode: "realtime", // framerate: fps, avc: { format: "annexb" }, // hardwareAcceleration:"prefer-software" // bitrate:100000, // bitrate: 10000000, }); for (let i=0; i < 20; i++) { const frame = new VideoFrame(bitmap, { timestamp:i * 1/fps * 1e6, duration:1/fps * 1e6, }); encoder.encode(frame, {keyFrame: i === 0}); frame.close(); } bitmap.close(); console.log("sent frames"); await encoder.flush(); console.log("flushed", {keyFrames,deltaFrames}) encoder.close(); console.log("closed", {keyFrames,deltaFrames}) } main().catch((e) => console.error(e,e.message)); ```
Tested in Safari Technology Preview 182 (Safari 17.4, WebKit 19618.1.4.1). Note that due to https://bugs.webkit.org/show_bug.cgi?id=258669, step 3 does not work in Safari 17.0.
<rdar://problem/118725549>
We might need to set a default bitrate allocation in this case, or bail out. I wonder what other browsers are doing here.
Pull request: https://github.com/WebKit/WebKit/pull/20842
Committed 271087@main (ef16bd3f9b1c): <https://commits.webkit.org/271087@main> Reviewed commits have been landed. Closing PR #20842 and removing active labels.
Thanks for your quick response on this ticket. Would there be any way of ensuring that other similar configuration problems in the future produce an isConfigSupported=false response?