[GStreamer] clarify playback errors with MediaError.message

• Component: Media
• Status: RESOLVED FIXED

[Enrique Ocaña]

It would be interesting to expose the internal GStreamer playback errors to the upper layer using the MediaError.message API.

[Enrique Ocaña]

Created attachment 454143 [details]
Patch

[Enrique Ocaña]

Created attachment 454229 [details]
Patch

[Enrique Ocaña]

This patch seems to cause important regressions related to MSE on Mac:

Regressions: Unexpected image-only failures (1)
  compositing/iframes/border-uneven-radius-composited-frame.html [ ImageOnlyFailure ]
Regressions: Unexpected timeouts (14)
  fast/canvas/webgl/texImage2D-mse-flipY-false.html [ Timeout ]
  fast/canvas/webgl/texImage2D-mse-flipY-true.html [ Timeout ]
  fast/history/page-cache-media-source-closed-2.html [ Timeout ]
  fast/history/page-cache-removed-source-buffer.html [ Timeout ]
  http/tests/appcache/top-frame-1.html [ Timeout ]
  http/tests/media/media-source/mediasource-play-then-seek-back-with-remote-control.html [ Timeout ]
  http/tests/media/media-source/mediasource-rvfc.html [ Timeout ]
  http/tests/resourceLoadStatistics/prevalent-resource-unhandled-keydown.html [ Timeout ]
  http/tests/security/mixedContent/secure-redirect-to-secure-redirect-to-basic-auth-secure-image-allowCrossOriginSubresourcesToAskForCredentials.https.html [ Timeout ]
  imported/w3c/web-platform-tests/css/css-grid/layout-algorithm/flex-sizing-rows-min-max-height-001.html [ Timeout ]
  imported/w3c/web-platform-tests/css/css-overflow/scrollable-overflow-transform-dynamic-006.html [ Timeout ]
  imported/w3c/web-platform-tests/css/css-shadow-parts/chaining-invalid-selector.html [ Timeout ]
  imported/w3c/web-platform-tests/html/browsers/browsing-the-web/navigating-across-documents/javascript-url-query-fragment-components.html [ Timeout ]
  imported/w3c/web-platform-tests/html/browsers/history/the-history-interface/combination_history_006.html [ Timeout ]
Regressions: Unexpected text-only failures (15)
  fast/scrolling/mac/adjust-scroll-snap-during-gesture.html [ Failure ]
  http/tests/media/modern-media-controls/macos-fullscreen-media-controls/macos-fullscreen-media-controls-live-broadcast.html [ Failure ]
  http/wpt/fetch/response-opaque-clone.html [ Failure ]
  imported/w3c/web-platform-tests/html/cross-origin-opener-policy/iframe-popup-same-origin-allow-popups-to-unsafe-none.https.html [ Failure ]
  imported/w3c/web-platform-tests/media-source/mediasource-activesourcebuffers.html [ Failure ]
  imported/w3c/web-platform-tests/media-source/mediasource-addsourcebuffer-mode.html [ Failure ]
  imported/w3c/web-platform-tests/media-source/mediasource-appendwindow.html [ Failure ]
  imported/w3c/web-platform-tests/media-source/mediasource-attach-stops-delaying-load-event.html [ Failure ]
  imported/w3c/web-platform-tests/media-source/mediasource-avtracks.html [ Failure ]
  imported/w3c/web-platform-tests/media-source/mediasource-buffered-seek.html [ Failure ]
  imported/w3c/web-platform-tests/media-source/mediasource-buffered.html [ Failure ]
  imported/w3c/web-platform-tests/media-source/mediasource-changetype.html [ Failure ]
  imported/w3c/web-platform-tests/media-source/mediasource-closed.html [ Failure ]
  imported/w3c/web-platform-tests/service-workers/service-worker/fetch-request-no-freshness-headers.https.html [ Failure ]
  imported/w3c/web-platform-tests/webstorage/event_case_sensitive.html [ Failure ]

Unfortunately, I don't access to a working Mac build now to debug the cause, but I suspect it may be strongly related to the player private selection changes in MediaPlayer::networkStateChanged(). I'll try to prove it by submitting several temporary patches and see how EWS behaves.

[Enrique Ocaña]

Created attachment 454358 [details]
Patch

[Enrique Ocaña]

Created attachment 454479 [details]
Patch

[Philippe Normand]

Comment on attachment 454479 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=454479&action=review

> Source/WebCore/html/HTMLMediaElement.cpp:2218
> +    const auto getErrorMessage = [&] (String &&defaultMessage) {

String&& defaultMessage

> Source/WebCore/html/HTMLMediaElement.cpp:2220
> +        if (m_player && !m_player->lastErrorMessage().isEmpty()) {

if (!m_player)
    return message;

auto lastErrorMessage = m_player->lastErrorMessage();
if (!lastErrorMessage)
    return message;

return makeString(message, ": ", lastErrorMessage);

> Source/WebCore/platform/graphics/MediaPlayer.cpp:1918
> +    return m_lastErrorMessage;

Maybe we directly chain to the private player? return m_private->errorMessage(); ?

[Enrique Ocaña]

Comment on attachment 454479 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=454479&action=review

> Source/WebCore/html/HTMLMediaElement.cpp:2221
> +            message.append(": ");

Thanks!

>> Source/WebCore/platform/graphics/MediaPlayer.cpp:1918
>> +    return m_lastErrorMessage;
> 
> Maybe we directly chain to the private player? return m_private->errorMessage(); ?

No. Caching the result is a key piece of the patch. Line 1337...

  if (!m_activeEngineIdentifier && installedMediaEngines().size() > 1 && (m_contentType.isEmpty() || nextBestMediaEngine(m_currentMediaEngine))) {
      m_reloadTimer.startOneShot(0_s);

...reloads the player private in case of error, so the error isn't there when the upper layer tries to ask for it (assuming the chaining you suggest).

That was waht the original version of the patch https://bugs.webkit.org/attachment.cgi?id=454143&action=prettypatch did, but for that to "work", the code that regenerates the player private (line 1337) had to be changed so the old player private remained alive. While this worked fine in WebKitGTK, it caused a lot of regressions on Mac ports. I had to use this new approach of caching the player private error in the player and keep the original code to regenerate the player private in order to avoid the regressions.

[Enrique Ocaña]

Created attachment 454583 [details]
Patch

[Enrique Ocaña]

Comment on attachment 454583 [details]
Patch

Landing the patch, as the editing/spelling/spellcheck-api-crash.html failure in the iOS-15-Simulator-WK2-Tests-EWS bot is unrelated to multimedia stuff.

[EWS]

Tools/Scripts/svn-apply failed to apply attachment 454583 [details] to trunk.
Please resolve the conflicts and upload a new patch.

[Enrique Ocaña]

Created attachment 454697 [details]
Patch

[Enrique Ocaña]

Created attachment 454698 [details]
Patch

[EWS]

Committed r291290 (248429@main): <https://commits.webkit.org/248429@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 454698 [details].

[Radar WebKit Bug Importer]

<rdar://problem/90306817>