Bug 185839 – Avoid loading AVFoundation to check supported MIME types if possible

• Status: ASSIGNED
• Component: Media

[Eric Carlson]

Loading AVFoundation to call +[AVURLAssetClass audiovisualMIMETypes] triggers an allocation of more than 2MB. Move the call to the UI process and pass the list of types to each new web process so we can do it once per application instead of once per web process.

[Eric Carlson]

<rdar://problem/40182010>

[Eric Carlson]

Created attachment 340895 [details]
Patch

[Simon Fraser (smfr)]

Comment on attachment 340895 [details]
Patch

How much time does this add to Safari launch on a slow iOS device? Before, we would do this lazily in the web process when we found an extension we didn't know how to handle. Now, you're paying the cost every time. I'm not sure this is an improvement.

[Eric Carlson]

Comment on attachment 340895 [details]
Patch

Removing r?, this needs to be revised to work in WK1.

[Jer Noble]

Comment on attachment 340895 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=340895&action=review

So it appears that, as it stands, this patch will break WebKitLegacy, since there's no code path for it to initialize the AVFoundationMIMETypeCache. One idea to circumvent this problem would be to leave the AVFoundationMIMETypeCache implementation as-is, but add a "non-initializing singleton()" accessor that WK2 could use to fill the cache with pre-initialized values.

> Source/WebCore/platform/graphics/avfoundation/objc/ImageDecoderAVFObjC.mm:361
> -    if (!supportsMediaType(MediaType::Video))
> -        return nullptr;
> +    if (AVFoundationMIMETypeCache::singleton().types().contains(mimeType))
> +        return true;

I would change this sightly:

    ContentType type  { mimeType };
    if (!AVFoundationMIMETypeCache::singleton().types().contains(type.contentType()))
        return flase;

That way, if the content-type portion of the MIME isn't an AVFoundation supported type, we don't bother asking the AVURLAsset for something which it's not going to support.

[EWS Watchlist]

Comment on attachment 340895 [details]
Patch

Attachment 340895 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/7757570

Number of test failures exceeded the failure limit.

[EWS Watchlist]

Created attachment 340907 [details]
Archive of layout-test-results from ews103 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews103  Port: mac-sierra  Platform: Mac OS X 10.12.6

[EWS Watchlist]

Comment on attachment 340895 [details]
Patch

Attachment 340895 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/7757505

Number of test failures exceeded the failure limit.

[EWS Watchlist]

Created attachment 340908 [details]
Archive of layout-test-results from ews113 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-sierra  Platform: Mac OS X 10.12.6

[EWS Watchlist]

Comment on attachment 340895 [details]
Patch

Attachment 340895 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/7757907

New failing tests:
fast/images/animated-image-mp4.html
http/tests/images/image-supports-video.html
fast/images/animated-image-mp4-crash.html

[EWS Watchlist]

Created attachment 340915 [details]
Archive of layout-test-results from ews107 for mac-sierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews107  Port: mac-sierra-wk2  Platform: Mac OS X 10.12.6

[EWS Watchlist]

Comment on attachment 340895 [details]
Patch

Attachment 340895 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/7757756

New failing tests:
fast/images/animated-image-mp4.html
fast/images/animated-image-mp4-crash.html

[EWS Watchlist]

Created attachment 340919 [details]
Archive of layout-test-results from ews124 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews124  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.4

[Eric Carlson]

*** Bug 185817 has been marked as a duplicate of this bug. ***

[Eric Carlson]

Created attachment 341124 [details]
Patch

[Eric Carlson]

Created attachment 341126 [details]
Patch

[Jer Noble]

Comment on attachment 341126 [details]
Patch

r=me with a general nit. Rather than check whether the m_cache.isEmpty() to see whether it's been initialized, it might be better to have a "bool m_isInitialized" or change m_cache to an std::optional. That way, you can disambiguate between "the list of supported mime types is empty" and "the cache hasn't been initialized yet".

[EWS Watchlist]

Comment on attachment 341126 [details]
Patch

Attachment 341126 [details] did not pass mac-ews (mac):
Output: http://webkit-queues.webkit.org/results/7780928

New failing tests:
fast/images/animated-image-mp4-crash.html
http/tests/images/image-supports-video.html
fast/images/animated-image-mp4.html

[EWS Watchlist]

Created attachment 341132 [details]
Archive of layout-test-results from ews100 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-ews.
Bot: ews100  Port: mac-sierra  Platform: Mac OS X 10.12.6

[EWS Watchlist]

Comment on attachment 341126 [details]
Patch

Attachment 341126 [details] did not pass mac-wk2-ews (mac-wk2):
Output: http://webkit-queues.webkit.org/results/7780953

New failing tests:
fast/images/animated-image-mp4.html
fast/images/animated-image-mp4-crash.html
http/tests/images/image-supports-video.html

[EWS Watchlist]

Created attachment 341133 [details]
Archive of layout-test-results from ews104 for mac-sierra-wk2

The attached test failures were seen while running run-webkit-tests on the mac-wk2-ews.
Bot: ews104  Port: mac-sierra-wk2  Platform: Mac OS X 10.12.6

[EWS Watchlist]

Comment on attachment 341126 [details]
Patch

Attachment 341126 [details] did not pass ios-sim-ews (ios-simulator-wk2):
Output: http://webkit-queues.webkit.org/results/7781188

New failing tests:
fast/images/animated-image-mp4-crash.html
fast/images/animated-image-mp4.html

[EWS Watchlist]

Created attachment 341138 [details]
Archive of layout-test-results from ews126 for ios-simulator-wk2

The attached test failures were seen while running run-webkit-tests on the ios-sim-ews.
Bot: ews126  Port: ios-simulator-wk2  Platform: Mac OS X 10.13.4

[EWS Watchlist]

Comment on attachment 341126 [details]
Patch

Attachment 341126 [details] did not pass mac-debug-ews (mac):
Output: http://webkit-queues.webkit.org/results/7781239

New failing tests:
fast/images/animated-image-mp4-crash.html
fast/images/animated-image-mp4.html

[EWS Watchlist]

Created attachment 341139 [details]
Archive of layout-test-results from ews113 for mac-sierra

The attached test failures were seen while running run-webkit-tests on the mac-debug-ews.
Bot: ews113  Port: mac-sierra  Platform: Mac OS X 10.12.6

[Eric Carlson]

Created attachment 341142 [details]
Patch for landing

[WebKit Commit Bot]

Comment on attachment 341142 [details]
Patch for landing

Clearing flags on attachment: 341142

Committed r232136: <https://trac.webkit.org/changeset/232136>

[EWS Watchlist]

Comment on attachment 341142 [details]
Patch for landing

Attachment 341142 [details] did not pass win-ews (win):
Output: http://webkit-queues.webkit.org/results/7782529

New failing tests:
http/tests/security/contentSecurityPolicy/userAgentShadowDOM/allow-video.html

[EWS Watchlist]

Created attachment 341152 [details]
Archive of layout-test-results from ews200 for win-future

The attached test failures were seen while running run-webkit-tests on the win-ews.
Bot: ews200  Port: win-future  Platform: CYGWIN_NT-6.1-2.9.0-0.318-5-3-x86_64-64bit

[Simon Fraser (smfr)]

Could you address my comment #3 please?

[Eric Carlson]

(In reply to Simon Fraser (smfr) from comment #30)
> Could you address my comment #3 please?

Please read the patch.