Bug 222781 – HLS fMP4 transformation matrix ignored

• Status: NEW
• Component: Media

Created attachment 422338 [details]
Sample video with 90 degree clockwise transformation matrix

The MovieHeaderBox and TrackHeaderBox within MPEG-4 both support transformation matrices. These transformation matrices enable movies or tracks to be rotated, amongst other affine transformations. While these transformation matrices are respected by Quicktime Player, Chrome, and other products, Safari and iOS do not appear to support them. For instance, the attached sample video specifies a 90 degree clockwise rotation via the TrackHeaderBox transformation matrix. While it plays with the rotation in Quicktime Player and Chrome, it does not in Safari or on iOS.

[Jer Noble]

Could you provide a screenshot of the problem? When I play the attached file in Safari, it's displayed rotated by 90º clockwise, which looks correct according to the matrix specified in the track header.

Created attachment 422420 [details]
HLS example using fragmented MP4 segments and 90 degree clockwise track transformation matrix

The issue occurs when playing an HLS playlist with fragmented MP4 segments. Added hls.tar.gz attachment that demos the issue.

Created attachment 422421 [details]
Screenshot of issue (90 degree rotation transformation is not applied)

Adding screenshot demoing issue (90 degree rotation transformation is not applied -- doll in video appears right-side up, should appear rotated 90 degrees right).

[Jer Noble]

Okay, the HLS example does appear to reproduce the issue; the content appears untransformed. However, this same sample stream is also untransformed in QuickTime Player, and contra your first comment, fails to play in Chrome or Firefox due to a lack of HLS support there.

When the files inside the playlist are concatenated together, the resulting video is correctly displayed rotated, both in QTP and Safari. This sounds like a platform bug, where playback of content through HLS will ignore track and movie matrices.

[Jer Noble]

Chris, this isn't an issue which can be fixed by WebKit; you may consider filing a bug against macOS and iOS at bugreport.apple.com with your sample stream & use case details.

(In reply to Jer Noble from comment #5)
> Chris, this isn't an issue which can be fixed by WebKit; you may consider
> filing a bug against macOS and iOS at bugreport.apple.com with your sample
> stream & use case details.

Jer, thanks, have filed the issue with the AVFoundation team (best guess at owner).

In Chrome and Firefox, I use hls.js, which translates HLS into the Media Source Extensions API -- plays with transformation applied.

[Radar WebKit Bug Importer]

<rdar://problem/75350189>

[Hanno Gödecke]

Hey Chris, can you give a follow-up reference to the bug report on AVFoundation? Thanks a lot

(In reply to Hanno Gödecke from comment #8)
> Hey Chris, can you give a follow-up reference to the bug report on
> AVFoundation? Thanks a lot

Current status in Feedback Assistant shows:

  Recent Similar Reports: None
  Resolution: Open

So likely hasn't gotten any traction, unless the radr ticket above is related. Issue number is FB9031689.