Bug 198555 – [WinCairo][MediaFoundation] MediaPlayerPrivateMediaFoundation::naturalSize is going to return larger and larger sizes gradually in high DPI

• Component: Platform
• Status: NEW

[Fujii Hironori]

[WinCairo][MediaFoundation] MediaPlayerPrivateMediaFoundation::naturalSize is going to return larger and larger sizes gradually

1. Start WebKitBuild\Debug\bin64\MiniBrowser.exe --wk1 
2. Open LayoutTests/media/video-controls.html
3. The video element is quickly getting very large size

[Fujii Hironori]

I applied a following logging patch to MediaPlayerPrivateMediaFoundation::naturalSize.

> diff --git a/Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.cpp b/Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.cpp
> index 00b0d0884fc..790ec6b4a43 100644
> --- a/Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.cpp
> +++ b/Source/WebCore/platform/graphics/win/MediaPlayerPrivateMediaFoundation.cpp
> @@ -45,6 +45,8 @@
>  #include <wtf/MainThread.h>
>  #include <wtf/NeverDestroyed.h>
>  
> +#include "Logging.h"
> +
>  SOFT_LINK_LIBRARY(Mf);
>  SOFT_LINK_OPTIONAL(Mf, MFCreateSourceResolver, HRESULT, STDAPICALLTYPE, (IMFSourceResolver**));
>  SOFT_LINK_OPTIONAL(Mf, MFCreateMediaSession, HRESULT, STDAPICALLTYPE, (IMFAttributes*, IMFMediaSession**));
> @@ -208,6 +210,7 @@ bool MediaPlayerPrivateMediaFoundation::supportsFullscreen() const
>  FloatSize MediaPlayerPrivateMediaFoundation::naturalSize() const
>  {
>      LockHolder locker(m_cachedNaturalSizeLock);
> +    LOG(Media, "MediaPlayerPrivateMediaFoundation::naturalSize %fx%f", m_cachedNaturalSize.width(), m_cachedNaturalSize.height());
>      return m_cachedNaturalSize;
>  }
>  

Here is the result:

> MediaPlayerPrivateMediaFoundation::naturalSize 450.000000x225.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 450.000000x225.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 450.000000x225.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 450.000000x225.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 450.000000x225.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 450.000000x225.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 675.000000x337.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 675.000000x337.000000
> The thread 0x3e04 has exited with code 0 (0x0).
> MediaPlayerPrivateMediaFoundation::naturalSize 1012.000000x505.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 1012.000000x505.000000
> The thread 0x3c04 has exited with code 0 (0x0).
> MediaPlayerPrivateMediaFoundation::naturalSize 1518.000000x757.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 1518.000000x757.000000
> The thread 0x2f54 has exited with code 0 (0x0).
> MediaPlayerPrivateMediaFoundation::naturalSize 2277.000000x1135.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 2277.000000x1135.000000
> The thread 0x1d88 has exited with code 0 (0x0).
> MediaPlayerPrivateMediaFoundation::naturalSize 3415.000000x1702.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 3415.000000x1702.000000
> The thread 0x33a4 has exited with code 0 (0x0).
> MediaPlayerPrivateMediaFoundation::naturalSize 5122.000000x2553.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 5122.000000x2553.000000
> The thread 0x3aa8 has exited with code 0 (0x0).
> MediaPlayerPrivateMediaFoundation::naturalSize 7683.000000x3829.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 7683.000000x3829.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 7683.000000x3829.000000
> The thread 0xa40 has exited with code 0 (0x0).
> MediaPlayerPrivateMediaFoundation::naturalSize 11524.000000x5743.000000
> The thread 0x478c has exited with code 0 (0x0).
> MediaPlayerPrivateMediaFoundation::naturalSize 11524.000000x4366.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 11524.000000x4366.000000
> The thread 0x3678 has exited with code 0 (0x0).
> MediaPlayerPrivateMediaFoundation::naturalSize 11524.000000x4366.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 11524.000000x4366.000000
> MediaPlayerPrivateMediaFoundation::naturalSize 11524.000000x4366.000000
(...)

[Fujii Hironori]

I confirmed otter-browser-win64-1.0.81-weekly272-setup.exe also has the same issue.

https://shapeshed.com/examples/HTML5-video-element/

[Konstantin Tokarev]

Could you please add m_size to log?

BTW, I believe that to make video larger something needs to invoke  MediaPlayerPrivateMediaFoundation::setSize(). Could you put a breakpoint there and show backtrace?

[Fujii Hironori]

Sure. https://gist.github.com/fujii/42c9bf9d01069b2c51f409b68d1ad0c9

[Fujii Hironori]

I put a breakpoint at setNaturalSize, and got a interasting backtrace.

setSize → renegotiateMediaType → setNaturalSize

> WebKit.dll!WebCore::MediaPlayerPrivateMediaFoundation::setNaturalSize(const WebCore::FloatSize & size) Line 747	C++
> WebKit.dll!WebCore::MediaPlayerPrivateMediaFoundation::CustomVideoPresenter::setMediaType(IMFMediaType * mediaType) Line 1642	C++
> WebKit.dll!WebCore::MediaPlayerPrivateMediaFoundation::CustomVideoPresenter::renegotiateMediaType() Line 1691	C++
> WebKit.dll!WebCore::MediaPlayerPrivateMediaFoundation::CustomVideoPresenter::SetVideoPosition(const MFVideoNormalizedRect * pnrcSource, tagRECT * const prcDest) Line 1426	C++
> WebKit.dll!WebCore::MediaPlayerPrivateMediaFoundation::setSize(const WebCore::IntSize & size) Line 392	C++
> WebKit.dll!WebCore::MediaPlayer::setSize(const WebCore::IntSize & size) Line 875	C++
> WebKit.dll!WebCore::RenderVideo::updatePlayer() Line 259	C++
> WebKit.dll!WebCore::RenderVideo::layout() Line 223	C++
> WebKit.dll!WebCore::RenderElement::layoutIfNeeded() Line 122	C++
> WebKit.dll!WebCore::RenderBlockFlow::layoutLineBoxes(bool relayoutChildren, WebCore::LayoutUnit & repaintLogicalTop, WebCore::LayoutUnit & repaintLogicalBottom) Line 1709	C++
> WebKit.dll!WebCore::RenderBlockFlow::layoutInlineChildren(bool relayoutChildren, WebCore::LayoutUnit & repaintLogicalTop, WebCore::LayoutUnit & repaintLogicalBottom) Line 678	C++
> WebKit.dll!WebCore::RenderBlockFlow::layoutBlock(bool relayoutChildren, WebCore::LayoutUnit pageLogicalHeight) Line 508	C++
> WebKit.dll!WebCore::RenderBlock::layout() Line 603	C++
> WebKit.dll!WebCore::RenderBlockFlow::layoutBlockChild(WebCore::RenderBox & child, WebCore::RenderBlockFlow::MarginInfo & marginInfo, WebCore::LayoutUnit & previousFloatLogicalBottom, WebCore::LayoutUnit & maxFloatLogicalBottom) Line 738	C++
> WebKit.dll!WebCore::RenderBlockFlow::layoutBlockChildren(bool relayoutChildren, WebCore::LayoutUnit & maxFloatLogicalBottom) Line 637	C++
> WebKit.dll!WebCore::RenderBlockFlow::layoutBlock(bool relayoutChildren, WebCore::LayoutUnit pageLogicalHeight) Line 511	C++
> WebKit.dll!WebCore::RenderBlock::layout() Line 603	C++
> WebKit.dll!WebCore::RenderBlockFlow::layoutBlockChild(WebCore::RenderBox & child, WebCore::RenderBlockFlow::MarginInfo & marginInfo, WebCore::LayoutUnit & previousFloatLogicalBottom, WebCore::LayoutUnit & maxFloatLogicalBottom) Line 738	C++
> WebKit.dll!WebCore::RenderBlockFlow::layoutBlockChildren(bool relayoutChildren, WebCore::LayoutUnit & maxFloatLogicalBottom) Line 637	C++
> WebKit.dll!WebCore::RenderBlockFlow::layoutBlock(bool relayoutChildren, WebCore::LayoutUnit pageLogicalHeight) Line 511	C++
> WebKit.dll!WebCore::RenderBlock::layout() Line 603	C++
> WebKit.dll!WebCore::RenderBlockFlow::layoutBlockChild(WebCore::RenderBox & child, WebCore::RenderBlockFlow::MarginInfo & marginInfo, WebCore::LayoutUnit & previousFloatLogicalBottom, WebCore::LayoutUnit & maxFloatLogicalBottom) Line 738	C++
> WebKit.dll!WebCore::RenderBlockFlow::layoutBlockChildren(bool relayoutChildren, WebCore::LayoutUnit & maxFloatLogicalBottom) Line 637	C++
> WebKit.dll!WebCore::RenderBlockFlow::layoutBlock(bool relayoutChildren, WebCore::LayoutUnit pageLogicalHeight) Line 511	C++
> WebKit.dll!WebCore::RenderBlock::layout() Line 603	C++
> WebKit.dll!WebCore::RenderView::layout() Line 186	C++
> WebKit.dll!WebCore::FrameViewLayoutContext::layout() Line 212	C++
> WebKit.dll!WebCore::Document::updateLayout() Line 2079	C++
> WebKit.dll!WebCore::Document::updateLayoutIgnorePendingStylesheets(WebCore::Document::RunPostLayoutTasks runPostLayoutTasks) Line 2094	C++
> WebKit.dll!WebCore::Element::boundingClientRect() Line 1464	C++
> WebKit.dll!WebCore::Element::getBoundingClientRect() Line 1476	C++
> WebKit.dll!WebCore::jsElementPrototypeFunctionGetBoundingClientRectBody(JSC::ExecState * state, WebCore::JSElement * castedThis, JSC::ThrowScope & throwScope) Line 3923	C++
> WebKit.dll!WebCore::IDLOperation<WebCore::JSElement>::call<&WebCore::jsElementPrototypeFunctionGetBoundingClientRectBody,WebCore::CastedThisErrorBehavior::Throw>(JSC::ExecState & state, const char * operationName) Line 53	C++
> WebKit.dll!WebCore::jsElementPrototypeFunctionGetBoundingClientRect(JSC::ExecState * state) Line 3928	C++
> [External Code]

[Fujii Hironori]

Oh, I forgot to mention the important information I'm using 150% scale factor high DPI display.

> MediaPlayerPrivateMediaFoundation::naturalSize m_size=450x225 m_cachedNaturalSize=675.0x337.0

(/ 675.0 450)
→ 1.5

(/ 337.0 225)
→ 1.4977777777777779

Surprisingly, it still happens after I changed display scale factor to 100% manually.

[Konstantin Tokarev]

It seems this is happening because naturalSize is larger than size, see m_size=300x150 m_cachedNaturalSize=450.0x225.0 in the beginning. It cause relayout to adjust size of <video>, which makes video frame larger and naturalSize grows futher, and so on

[Fujii Hironori]

Yup.
My 'deviceScaleFactor' is 1.5 in MediaPlayerPrivateMediaFoundation::setSize.

[Konstantin Tokarev]

So, it seems that we need to scale naturalSize down by deviceScaleFactor. Could you prepare the patch?

[Fujii Hironori]

I think so. I will try.

[Fujii Hironori]

Created attachment 371379 [details]
WIP patch

* Scale naturalSize down by deviceScaleFactor.

This patch actually fixes the video size bloating issue, but the video size is too small. 🤔