Bug 198583 – MediaSource.isTypeSupported claims FLAC-in-MP4 support on iOS and macOS, but plays silence

• Component: Media
• Status: RESOLVED FIXED

[Brooke Vibber]

MediaSource.isTypeSupported('audio/mp4; codecs="flac"') returns true in Safari 12.1.1 and Tech Preview on macOS 10.14, and iOS 12.4 & iPadOS 13 beta, however this doesn't appear to be supported in my testing, with FLAC-in-MP4 files produced by ffmpeg or manually synthesized.

Example page with short h.264-in-fMP4 and FLAC-in-fMP4 tracks that plays no audio in Safari:
https://brionv.com/misc/msetest4/flac.html

Expected behavior:
Either an exception should be thrown when creating the buffer due to unsupported codec, or it should work.

Actual behavior:
Acts as though audio is valid and present but only silence is heard.

Behavior is the same in both macOS and iOS: no audio, no error.


Note that a _bare_ FLAC stream with 'audio/flac' type works with MSE on macOS and correctly returns false for isMediaType / throws an exception on buffer creation on iOS 13 beta, where I guess no FLAC support is present in the system. Variant example showing correct expected behavior for the different container: https://brionv.com/misc/msetest4/flac2.html

[Brooke Vibber]

Sorry, that should be just 'iPadOS 13 beta', obvs MediaSource is not in iOS 12. :)

[Radar WebKit Bug Importer]

<rdar://problem/51487853>

[Jer Noble]

Created attachment 374619 [details]
Patch

Uploading patch on behalf of Eric Carlson.

[Jer Noble]

Created attachment 374620 [details]
Patch

[Jer Noble]

Created attachment 374621 [details]
Patch

[Jer Noble]

Created attachment 374641 [details]
Patch

[Sam Weinig]

Comment on attachment 374641 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=374641&action=review

> Source/WebCore/platform/graphics/avfoundation/objc/AVStreamDataParserMIMETypeCache.mm:86
> +    // FIXME(rdar://50502771) AVStreamDataParser does not have an -isPlayableExtendedMIMEType: method on this system,

Do you mean canParseExtendedMIMEType:?

[Jer Noble]

(In reply to Sam Weinig from comment #7)
> Comment on attachment 374641 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=374641&action=review
> 
> > Source/WebCore/platform/graphics/avfoundation/objc/AVStreamDataParserMIMETypeCache.mm:86
> > +    // FIXME(rdar://50502771) AVStreamDataParser does not have an -isPlayableExtendedMIMEType: method on this system,
> 
> Do you mean canParseExtendedMIMEType:?

Looks so.

[Maciej Stachowiak]

Comment on attachment 374641 [details]
Patch

This broke the build on iOS, so r- for that (but otherwise looks fine as far as I know).

/Volumes/Data/worker/iOS-12-Build-EWS/build/Source/WebCore/platform/graphics/avfoundation/objc/AVStreamDataParserMIMETypeCache.mm:43:12: error: cannot define category for undefined class 'AVStreamDataParser'
@interface AVStreamDataParser (AVStreamDataParserExtendedMIMETypePrivate)
           ^
In file included from /Volumes/Data/worker/iOS-12-Build-EWS/build/Source/WebCore/platform/graphics/avfoundation/objc/AVStreamDataParserMIMETypeCache.mm:34:
PAL/pal/cocoa/AVFoundationSoftLink.h:70:33: note: forward declaration of class here
SOFT_LINK_CLASS_FOR_HEADER(PAL, AVStreamDataParser)
                                ^
/Volumes/Data/worker/iOS-12-Build-EWS/build/Source/WebCore/platform/graphics/avfoundation/objc/AVStreamDataParserMIMETypeCache.mm:83:64: error: undeclared selector 'canParseExtendedMIMEType:' [-Werror,-Wundeclared-selector]
    if ([PAL::getAVStreamDataParserClass() respondsToSelector:@selector(canParseExtendedMIMEType:)])
                                                               ^
/Volumes/Data/worker/iOS-12-Build-EWS/build/Source/WebCore/platform/graphics/avfoundation/objc/AVStreamDataParserMIMETypeCache.mm:84:51: error: class method '+canParseExtendedMIMEType:' not found (return type defaults to 'id') [-Werror,-Wobjc-method-access]
        return [PAL::getAVStreamDataParserClass() canParseExtendedMIMEType:mimeType];
                                                  ^~~~~~~~~~~~~~~~~~~~~~~~
3 errors generated.

[Eric Carlson]

Created attachment 375089 [details]
Patch

[Eric Carlson]

Created attachment 375094 [details]
Patch

[EWS Watchlist]

Attachment 375094 [details] did not pass style-queue:


ERROR: Source/WebCore/platform/graphics/avfoundation/objc/AVStreamDataParserMIMETypeCache.mm:35:  Alphabetical sorting problem.  [build/include_order] [4]
Total errors found: 1 in 3 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[Eric Carlson]

Created attachment 375099 [details]
Patch

[EWS Watchlist]

Attachment 375099 [details] did not pass style-queue:


ERROR: Source/WebCore/platform/graphics/avfoundation/objc/AVStreamDataParserMIMETypeCache.mm:35:  Alphabetical sorting problem.  [build/include_order] [4]
Total errors found: 1 in 3 files


If any of these errors are false positives, please file a bug against check-webkit-style.

[WebKit Commit Bot]

Comment on attachment 375099 [details]
Patch

Clearing flags on attachment: 375099

Committed r247934: <https://trac.webkit.org/changeset/247934>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Brooke Vibber]

This is listed as fixed in the release notes for Safari Technology Preview 89, however I see no difference in behavior after updating (on macOS 10.14.6).

MediaSource.isTypeSupported('audio/mp4; codecs="flac"') still returns true, and playback still does not work on https://brionv.com/misc/msetest4/flac.html

[Jer Noble]

(In reply to Brion Vibber from comment #17)
> This is listed as fixed in the release notes for Safari Technology Preview
> 89, however I see no difference in behavior after updating (on macOS
> 10.14.6).
> 
> MediaSource.isTypeSupported('audio/mp4; codecs="flac"') still returns true,
> and playback still does not work on
> https://brionv.com/misc/msetest4/flac.html

This change in WebKit has an associated platform OS change needed as well. We'll ping this bug when that OS change is available for testing.

[Brooke Vibber]

Great, thanks!