Bug 263901 – [Cocoa] WebCodecs H264 decoder should reorder frames according presentation time

• Status: RESOLVED
• Resolution: FIXED
• Component: Media

[youenn fablet]

[Cocoa] WebCodecs H264 decoder should reorder frames according presentation time

[Radar WebKit Bug Importer]

<rdar://problem/117692801>

[youenn fablet]

Pull request: https://github.com/WebKit/WebKit/pull/19725

[Marcello Bastea-Forte]

I can reproduce this by modifying the w3c demo project to log timestamps and check for out-of-order:
https://github.com/w3c/webcodecs/pull/734/files

I get out-of-order output callbacks for the test h264 file already in that repo (and a bunch of my own test files).

Repro'ed on macOS 14.0 (23A344):
- Safari Version 17.0 (19616.1.27.211.1)
- Safari Technology Preview Release 181 (Safari 17.4, WebKit 19618.1.3.1)

[Marcello Bastea-Forte]

Full repro steps:
- pull the code:
  - either from https://github.com/marcello3d/webcodecs/tree/marcello/frame-out-of-order-logging
  - apply the patch yourself to https://github.com/w3c/webcodecs https://patch-diff.githubusercontent.com/raw/w3c/webcodecs/pull/734.patch
- run the server with `node server` in the samples folder
- open http://localhost:8888/ in safari
- open developer console to see javascript logs
- leave settings at 2D & h264
- click start

Expected:
- no error logs, everything decodes in ascending order

Actual:
- alternating decode and decode out of order logs:

[Log] Decoded frame 66666 µs (worker.js, line 78)
[Log] Decoded frame 100000 µs (worker.js, line 78)
[Log] Decoded frame 133333 µs (worker.js, line 78)
[Log] Decoded frame 200000 µs (worker.js, line 78)
[Error] Decoded frame 166666 µs out of order, last frame was 200000 µs
	output (worker.js:76)
[Log] Decoded frame 233333 µs (worker.js, line 78)
[Log] Decoded frame 366666 µs (worker.js, line 78)
[Error] Decoded frame 300000 µs out of order, last frame was 366666 µs
	output (worker.js:76)
[Error] Decoded frame 266666 µs out of order, last frame was 300000 µs
	output (worker.js:76)
[Log] Decoded frame 333333 µs (worker.js, line 78)
[Log] Decoded frame 500000 µs (worker.js, line 78)
[Error] Decoded frame 433333 µs out of order, last frame was 500000 µs
	output (worker.js:76)

...etc

[Marcello Bastea-Forte]

I don't know if this should be a separate ticket, but the same issue happens with HEVC/h265 on Safari Technology Preview Release 181 (Safari 17.4, WebKit 19618.1.3.1)

Repro is the same except choose h265 before clicking start. VP8, VP9, and AV1 all run successfully with no out-of-order errors

[EWS]

Committed 271004@main (cf345a7967b3): <https://commits.webkit.org/271004@main>

Reviewed commits have been landed. Closing PR #19725 and removing active labels.

[Marcello Bastea-Forte]

How do I run the webkit nightly build? 

I just tried Safari Technical Preview Release 184 (Safari 17.4, WebKit 19618.1.7.1.1) and still see the bug, but not sure if this fix is in there?

[Marcello Bastea-Forte]

Also still broken on Release 185 (Safari 17.4, WebKit 19618.1.9.8)

[Marcello Bastea-Forte]

Also still broken on Safari Technical Preview Release 186 (Safari 17.4, WebKit 19618.1.10.1.1) on macOS Sonoma 14.1.2 (23B92)

[Marcello Bastea-Forte]

Still broken on Safari Technical Preview Release 187 (Safari 17.4, WebKit 19619.0.1.2) on Sonoma 14.1.2 (23B92)

[youenn fablet]

I filed https://bugs.webkit.org/show_bug.cgi?id=268987 to keep track of this, it seems additional work is needed.