Bug 253085 – REGRESSION(260881@main): [GStreamer] DMABuf sink caused several unexpected crashes on the WPE Release bot

• Component: WPE WebKit
• Status: RESOLVED DUPLICATE

[Carlos Alberto Lopez Perez]

The WPE Release bot has started to exit early due to more than 50 crashes since 260881@main

This issue is not reproducible on the other WPE test bots that we have because the post-commit release bot has a different configuration

* All WPE test bots run without a GPU configured (no /dev/dri)

But

* The WPE Release post-commit bot runs inside a weston session and has a real GPU (Intel i7-3770) that is accesible at /dev/dri

You can check the crashes and the backtraces here:  https://build.webkit.org/#/builders/40/builds/9223

I have checked that the crash don't happen if I set WEBKIT_GST_DMABUF_SINK_DISABLED=1 on the environment

This is the relevant info about the environment

bb-wpe-release-test-64 ~ # lscpu | grep ^"Model name"
Model name:                      Intel(R) Core(TM) i7-3770 CPU @ 3.40GHz

bb-wpe-release-test-64 ~ # lspci | grep VGA
00:02.0 VGA compatible controller: Intel Corporation Xeon E3-1200 v2/3rd Gen Core processor Graphics Controller (rev 09)

bb-wpe-release-test-64 ~ # ls  /dev/dri/
by-path  card0  renderD128

bb-wpe-release-test-64 ~ # XDG_RUNTIME_DIR=/run/user/1000 WAYLAND_DISPLAY=wayland-0 /home/buildbot/waylandes2info | curl -F 'sprunge=<-' http://sprunge.us
http://sprunge.us/RgCE9p

bb-wpe-release-test-64 ~ # modetest | curl -F 'sprunge=<-' http://sprunge.us
http://sprunge.us/gIsNu0

[Carlos Alberto Lopez Perez]

I have uploaded here the results of running:

Tools/Scripts/run-webkit-tests --no-show-results --no-new-test-results --no-sample-on-timeout --no-build --results-directory layout-test-results-delme --debug-rwt-logging --release --wpe --no-retry-failures -f fast/canvas/webgl/texImage2D-mse-flipY-false.html fast/canvas/webgl/texImage2D-mse-flipY-true.html fast/canvas/webgl/texImage2D-video-flipY-false.html fast/canvas/webgl/texImage2D-video-flipY-true.html fast/mediastream/RTCSessionDescription.html fast/mediastream/getUserMedia-media-element-display-none.html http/tests/media/track/track-webvtt-slow-loading-2.html http/tests/webrtc/muted-video-mediastream-invisible-autoplay.html http/tests/webrtc/paused-video-mediastream-invisible-autoplay.html imported/w3c/web-platform-tests/css/css-contain/contain-inline-size-replaced.html imported/w3c/web-platform-tests/html/browsers/browsing-the-web/unloading-documents/prompt/001.html imported/w3c/web-platform-tests/html/canvas/element/manual/imagebitmap/createImageBitmap-serializable.html imported/w3c/web-platform-tests/html/infrastructure/urls/resolving-urls/query-encoding/attributes.sub.html?encoding=x-cp1251 imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/autoplay-allowed-by-feature-policy.https.sub.html imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/autoplay-default-feature-policy.https.sub.html imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/autoplay-disabled-by-feature-policy.https.sub.html imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/autoplay-with-broken-track.html imported/w3c/web-platform-tests/html/semantics/embedded-content/the-iframe-element/sandbox_004.htm imported/w3c/web-platform-tests/html/user-activation/activation-trigger-pointerevent.html?touch imported/w3c/web-platform-tests/media-source/mediasource-changetype-play.html imported/w3c/web-platform-tests/mediacapture-fromelement/capture.html imported/w3c/web-platform-tests/navigation-timing/nav2_test_attributes_exist.html imported/w3c/web-platform-tests/remote-playback/cancel-watch-availability.html imported/w3c/web-platform-tests/video-rvfc/request-video-frame-callback-parallel.html imported/w3c/web-platform-tests/webcodecs/videoFrame-canvasImageSource.html imported/w3c/web-platform-tests/webrtc-priority/RTCRtpParameters-encodings.html media/modern-media-controls/media-documents/background-color-and-centering.html media/modern-media-controls/scrubber-support/scrubber-support-drag.html media/track/track-remove-by-setting-innerHTML.html media/video-aspect-ratio.html


* 16 crashes by default
https://people.igalia.com/clopez/wkbug/253085/layout-test-results/default/results.html

* zero crashes when setting WEBKIT_GST_DMABUF_SINK_DISABLED=1  env var
https://people.igalia.com/clopez/wkbug/253085/layout-test-results/dmabuf-sink-disabled/results.html

This were run with a build at 260940@main

[Carlos Alberto Lopez Perez]

Note: I have configured the WPE Release test post-commit bot to run with the environment var WEBKIT_GST_DMABUF_SINK_DISABLED=1 until this issue is fixed.

The first build running with that environment var is https://build.webkit.org/#/builders/40/builds/9225

[Philippe Normand]

bt for imported/w3c/web-platform-tests/media-source/mediasource-duration.html

So we have a GPU but the crash is in swrast. :confused:

Thread 1 (Thread 0x7f09bcff9640 (LWP 523632)):
#0  0x00007f0be120cdee in u_transfer_helper_transfer_unmap () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
#1  0x00007f0be0ca68c2 in tc_call_texture_unmap () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
#2  0x00007f0be0ca6c41 in tc_batch_execute () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
#3  0x00007f0be0ca7d66 in _tc_sync.constprop.0 () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
#4  0x00007f0be0caf128 in tc_flush () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
#5  0x00007f0be07d92ae in st_context_flush () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
#6  0x00007f0be070e688 in dri_flush () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
#7  0x00007f0c47538359 in WebCore::MediaPlayerPrivateGStreamer::pushDMABufToCompositor() () at /app/webkit/WebKitBuild/Release/lib/libWPEWebKit-2.0.so.1
#8  0x00007f0c47525efc in webKitVideoSinkSetMediaPlayerPrivate(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*)::{lambda(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*)#2}::_FUN(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*) () at /app/webkit/WebKitBuild/Release/lib/libWPEWebKit-2.0.so.1
#9  0x00007f0c3d47e056 in ffi_call_unix64 () at ../src/x86/unix64.S:104
#10 0x00007f0c3d47cadd in ffi_call_int (cif=<optimized out>, fn=<optimized out>, rvalue=<optimized out>, avalue=<optimized out>, closure=<optimized out>) at ../src/x86/ffi64.c:673
#11 0x00007f0c3d47d2b3 in ffi_call (cif=cif@entry=0x7f09bcff7780, fn=fn@entry=0x7f0c47525db0 <webKitVideoSinkSetMediaPlayerPrivate(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*)::{lambda(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*)#2}::_FUN(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*)>, rvalue=rvalue@entry=0x7f09bcff76e0, avalue=avalue@entry=0x7f09bcff76a0) at ../src/x86/ffi64.c:710
#12 0x00007f0c3fc344c0 in g_cclosure_marshal_generic_va (closure=<optimized out>, return_value=<optimized out>, instance=<optimized out>, args_list=<optimized out>, marshal_data=<optimized out>, n_params=<optimized out>, param_types=<optimized out>) at ../gobject/gclosure.c:1648
#13 0x00007f0c3fc4c480 in _g_closure_invoke_va (param_types=<optimized out>, n_params=<optimized out>, args=0x7f09bcff7990, instance=0x555f7b6f3c10, return_value=<optimized out>, closure=0x555f7b6f3f30) at ../gobject/gclosure.c:893
#14 g_signal_emit_valist (instance=<optimized out>, signal_id=<optimized out>, detail=<optimized out>, var_args=var_args@entry=0x7f09bcff7990) at ../gobject/gsignal.c:3440
#15 0x00007f0c3fc4c5b3 in g_signal_emit (instance=instance@entry=0x555f7b6f3c10, signal_id=<optimized out>, detail=detail@entry=0) at ../gobject/gsignal.c:3587
#16 0x00007f0c4061348d in gst_app_sink_render_common (psink=0x555f7b6f3c10 [GstBaseSink|webkit-dmabuf-video-appsink], data=<optimized out>, is_list=<optimized out>) at ../gst-libs/gst/app/gstappsink.c:1103
#17 0x00007f0c405ea05e in gst_base_sink_chain_unlocked.constprop.0 (basesink=0x555f7b6f3c10 [GstBaseSink|webkit-dmabuf-video-appsink], obj=0x7f0b40013c60, is_list=<optimized out>, pad=<optimized out>) at ../libs/gst/base/gstbasesink.c:3952
#18 0x00007f0c405b7150 in gst_base_sink_chain_main (basesink=0x555f7b6f3c10 [GstBaseSink|webkit-dmabuf-video-appsink], pad=<optimized out>, obj=0x7f0b40013c60, is_list=0) at ../libs/gst/base/gstbasesink.c:4078
#19 0x00007f0c435a9775 in gst_pad_chain_data_unchecked (pad=pad@entry=0x7f0b50023570 [GstPad|sink], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
#20 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f0b1c051630 [GstPad|proxypad724], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4739
#21 0x00007f0c435ad4de in gst_pad_push (pad=0x7f0b1c051630 [GstPad|proxypad724], buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
#22 0x00007f0c43592aeb in gst_proxy_pad_chain_default (pad=<optimized out>, parent=<optimized out>, buffer=0x7f0b40013c60 [None]) at ../gst/gstghostpad.c:127
#23 0x00007f0c435a9775 in gst_pad_chain_data_unchecked (pad=pad@entry=0x555f7b593640 [GstPad|sink], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
#24 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09c00238b0 [GstPad|src], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4739
#25 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09c00238b0 [GstPad|src], buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
#26 0x00007f0c43592aeb in gst_proxy_pad_chain_default (pad=<optimized out>, parent=<optimized out>, buffer=0x7f0b40013c60 [None]) at ../gst/gstghostpad.c:127
#27 0x00007f0c435a9775 in gst_pad_chain_data_unchecked (pad=pad@entry=0x7f09c401a7b0 [GstPad|proxypad811], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
#28 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09b802b690 [GstPad|src], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4739
#29 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09b802b690 [GstPad|src], buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
#30 0x00007f0c405d331b in gst_base_transform_chain (pad=<optimized out>, parent=0x7f09c403ef00 [GstObject|conv2], buffer=<optimized out>) at ../libs/gst/base/gstbasetransform.c:2391
#31 0x00007f0c435a9775 in gst_pad_chain_data_unchecked (pad=pad@entry=0x7f09b802b8e0 [GstPad|sink], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
#32 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09b802b440 [GstPad|src], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4739
#33 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09b802b440 [GstPad|src], buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
#34 0x00007f0c405d331b in gst_base_transform_chain (pad=<optimized out>, parent=0x7f09c401c0d0 [GstObject|videobalance], buffer=<optimized out>) at ../libs/gst/base/gstbasetransform.c:2391
#35 0x00007f0c435a9775 in gst_pad_chain_data_unchecked (pad=pad@entry=0x7f09b802b1f0 [GstPad|sink], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
#36 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09b8033810 [GstPad|src], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4739
#37 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09b8033810 [GstPad|src], buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
#38 0x00007f0c405d331b in gst_base_transform_chain (pad=<optimized out>, parent=0x7f09c403ea20 [GstObject|scale], buffer=<optimized out>) at ../libs/gst/base/gstbasetransform.c:2391
#39 0x00007f0c435a9775 in gst_pad_chain_data_unchecked (pad=pad@entry=0x7f09b8033a60 [GstPad|sink], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
#40 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09c4040080 [GstPad|src], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4739
#41 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09c4040080 [GstPad|src], buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
#42 0x00007f0c405d331b in gst_base_transform_chain (pad=<optimized out>, parent=0x7f09c403d9b0 [GstObject|conv], buffer=<optimized out>) at ../libs/gst/base/gstbasetransform.c:2391
#43 0x00007f0c435a9775 in gst_pad_chain_data_unchecked (pad=pad@entry=0x7f09b8033cb0 [GstPad|sink], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
#44 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09c401a550 [GstPad|proxypad810], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4739
#45 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09c401a550 [GstPad|proxypad810], buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
#46 0x00007f0c43592aeb in gst_proxy_pad_chain_default (pad=<optimized out>, parent=<optimized out>, buffer=0x7f0b40013c60 [None]) at ../gst/gstghostpad.c:127
#47 0x00007f0c435a9775 in gst_pad_chain_data_unchecked (pad=pad@entry=0x7f09c0023640 [GstPad|sink], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
#48 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09b802ab00 [GstPad|src], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4739
#49 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09b802ab00 [GstPad|src], buffer=buffer@entry=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
#50 0x00007f0bd40545ec in gst_queue_push_one (queue=0x7f09f000c760 [GstQueue|vqueue]) at ../plugins/elements/gstqueue.c:1388
#51 gst_queue_loop (pad=<optimized out>) at ../plugins/elements/gstqueue.c:1541
#52 0x00007f0c435d5d41 in gst_task_func (task=0x7f09d001fcb0 [GstTask|vqueue:src]) at ../gst/gsttask.c:384
#53 0x00007f0c437e2692 in g_thread_pool_thread_proxy (data=<optimized out>) at ../glib/gthreadpool.c:350
#54 0x00007f0c437df722 in g_thread_proxy (data=0x7f0b5c014640) at ../glib/gthread.c:827
#55 0x00007f0c4304e1da in start_thread (arg=<optimized out>) at pthread_create.c:442
#56 0x00007f0c430d6f44 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100

[Philippe Normand]

In some logs there's a steady flow of:

STDERR: warning: Can't open file anon_inode:i915.gem which was expanded to anon_inode:i915.gem during file-backed mapping note processing

[Carlos Alberto Lopez Perez]

(In reply to Philippe Normand from comment #3)
> bt for imported/w3c/web-platform-tests/media-source/mediasource-duration.html
> 
> So we have a GPU but the crash is in swrast. :confused:
> 

Maybe the test environment is forcing the software renderer?

(In reply to Philippe Normand from comment #3)
> bt for imported/w3c/web-platform-tests/media-source/mediasource-duration.html
> 
> So we have a GPU but the crash is in swrast. :confused:
> 
> Thread 1 (Thread 0x7f09bcff9640 (LWP 523632)):
> #0  0x00007f0be120cdee in u_transfer_helper_transfer_unmap () at
> /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
> #1  0x00007f0be0ca68c2 in tc_call_texture_unmap () at
> /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
> #2  0x00007f0be0ca6c41 in tc_batch_execute () at
> /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
> #3  0x00007f0be0ca7d66 in _tc_sync.constprop.0 () at
> /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
> #4  0x00007f0be0caf128 in tc_flush () at
> /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
> #5  0x00007f0be07d92ae in st_context_flush () at
> /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
> #6  0x00007f0be070e688 in dri_flush () at
> /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/swrast_dri.so
> #7  0x00007f0c47538359 in
> WebCore::MediaPlayerPrivateGStreamer::pushDMABufToCompositor() () at
> /app/webkit/WebKitBuild/Release/lib/libWPEWebKit-2.0.so.1
> #8  0x00007f0c47525efc in webKitVideoSinkSetMediaPlayerPrivate(_GstElement*,
> WebCore::MediaPlayerPrivateGStreamer*)::{lambda(_GstElement*,
> WebCore::MediaPlayerPrivateGStreamer*)#2}::_FUN(_GstElement*,
> WebCore::MediaPlayerPrivateGStreamer*) () at
> /app/webkit/WebKitBuild/Release/lib/libWPEWebKit-2.0.so.1
> #9  0x00007f0c3d47e056 in ffi_call_unix64 () at ../src/x86/unix64.S:104
> #10 0x00007f0c3d47cadd in ffi_call_int (cif=<optimized out>, fn=<optimized
> out>, rvalue=<optimized out>, avalue=<optimized out>, closure=<optimized
> out>) at ../src/x86/ffi64.c:673
> #11 0x00007f0c3d47d2b3 in ffi_call (cif=cif@entry=0x7f09bcff7780,
> fn=fn@entry=0x7f0c47525db0
> <webKitVideoSinkSetMediaPlayerPrivate(_GstElement*,
> WebCore::MediaPlayerPrivateGStreamer*)::{lambda(_GstElement*,
> WebCore::MediaPlayerPrivateGStreamer*)#2}::_FUN(_GstElement*,
> WebCore::MediaPlayerPrivateGStreamer*)>, rvalue=rvalue@entry=0x7f09bcff76e0,
> avalue=avalue@entry=0x7f09bcff76a0) at ../src/x86/ffi64.c:710
> #12 0x00007f0c3fc344c0 in g_cclosure_marshal_generic_va (closure=<optimized
> out>, return_value=<optimized out>, instance=<optimized out>,
> args_list=<optimized out>, marshal_data=<optimized out>, n_params=<optimized
> out>, param_types=<optimized out>) at ../gobject/gclosure.c:1648
> #13 0x00007f0c3fc4c480 in _g_closure_invoke_va (param_types=<optimized out>,
> n_params=<optimized out>, args=0x7f09bcff7990, instance=0x555f7b6f3c10,
> return_value=<optimized out>, closure=0x555f7b6f3f30) at
> ../gobject/gclosure.c:893
> #14 g_signal_emit_valist (instance=<optimized out>, signal_id=<optimized
> out>, detail=<optimized out>, var_args=var_args@entry=0x7f09bcff7990) at
> ../gobject/gsignal.c:3440
> #15 0x00007f0c3fc4c5b3 in g_signal_emit
> (instance=instance@entry=0x555f7b6f3c10, signal_id=<optimized out>,
> detail=detail@entry=0) at ../gobject/gsignal.c:3587
> #16 0x00007f0c4061348d in gst_app_sink_render_common (psink=0x555f7b6f3c10
> [GstBaseSink|webkit-dmabuf-video-appsink], data=<optimized out>,
> is_list=<optimized out>) at ../gst-libs/gst/app/gstappsink.c:1103
> #17 0x00007f0c405ea05e in gst_base_sink_chain_unlocked.constprop.0
> (basesink=0x555f7b6f3c10 [GstBaseSink|webkit-dmabuf-video-appsink],
> obj=0x7f0b40013c60, is_list=<optimized out>, pad=<optimized out>) at
> ../libs/gst/base/gstbasesink.c:3952
> #18 0x00007f0c405b7150 in gst_base_sink_chain_main (basesink=0x555f7b6f3c10
> [GstBaseSink|webkit-dmabuf-video-appsink], pad=<optimized out>,
> obj=0x7f0b40013c60, is_list=0) at ../libs/gst/base/gstbasesink.c:4078
> #19 0x00007f0c435a9775 in gst_pad_chain_data_unchecked
> (pad=pad@entry=0x7f0b50023570 [GstPad|sink], type=type@entry=4112,
> data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
> #20 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f0b1c051630
> [GstPad|proxypad724], type=type@entry=4112, data=data@entry=0x7f0b40013c60)
> at ../gst/gstpad.c:4739
> #21 0x00007f0c435ad4de in gst_pad_push (pad=0x7f0b1c051630
> [GstPad|proxypad724], buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
> #22 0x00007f0c43592aeb in gst_proxy_pad_chain_default (pad=<optimized out>,
> parent=<optimized out>, buffer=0x7f0b40013c60 [None]) at
> ../gst/gstghostpad.c:127
> #23 0x00007f0c435a9775 in gst_pad_chain_data_unchecked
> (pad=pad@entry=0x555f7b593640 [GstPad|sink], type=type@entry=4112,
> data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
> #24 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09c00238b0
> [GstPad|src], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at
> ../gst/gstpad.c:4739
> #25 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09c00238b0 [GstPad|src],
> buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
> #26 0x00007f0c43592aeb in gst_proxy_pad_chain_default (pad=<optimized out>,
> parent=<optimized out>, buffer=0x7f0b40013c60 [None]) at
> ../gst/gstghostpad.c:127
> #27 0x00007f0c435a9775 in gst_pad_chain_data_unchecked
> (pad=pad@entry=0x7f09c401a7b0 [GstPad|proxypad811], type=type@entry=4112,
> data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
> #28 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09b802b690
> [GstPad|src], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at
> ../gst/gstpad.c:4739
> #29 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09b802b690 [GstPad|src],
> buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
> #30 0x00007f0c405d331b in gst_base_transform_chain (pad=<optimized out>,
> parent=0x7f09c403ef00 [GstObject|conv2], buffer=<optimized out>) at
> ../libs/gst/base/gstbasetransform.c:2391
> #31 0x00007f0c435a9775 in gst_pad_chain_data_unchecked
> (pad=pad@entry=0x7f09b802b8e0 [GstPad|sink], type=type@entry=4112,
> data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
> #32 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09b802b440
> [GstPad|src], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at
> ../gst/gstpad.c:4739
> #33 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09b802b440 [GstPad|src],
> buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
> #34 0x00007f0c405d331b in gst_base_transform_chain (pad=<optimized out>,
> parent=0x7f09c401c0d0 [GstObject|videobalance], buffer=<optimized out>) at
> ../libs/gst/base/gstbasetransform.c:2391
> #35 0x00007f0c435a9775 in gst_pad_chain_data_unchecked
> (pad=pad@entry=0x7f09b802b1f0 [GstPad|sink], type=type@entry=4112,
> data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
> #36 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09b8033810
> [GstPad|src], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at
> ../gst/gstpad.c:4739
> #37 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09b8033810 [GstPad|src],
> buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
> #38 0x00007f0c405d331b in gst_base_transform_chain (pad=<optimized out>,
> parent=0x7f09c403ea20 [GstObject|scale], buffer=<optimized out>) at
> ../libs/gst/base/gstbasetransform.c:2391
> #39 0x00007f0c435a9775 in gst_pad_chain_data_unchecked
> (pad=pad@entry=0x7f09b8033a60 [GstPad|sink], type=type@entry=4112,
> data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
> #40 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09c4040080
> [GstPad|src], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at
> ../gst/gstpad.c:4739
> #41 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09c4040080 [GstPad|src],
> buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
> #42 0x00007f0c405d331b in gst_base_transform_chain (pad=<optimized out>,
> parent=0x7f09c403d9b0 [GstObject|conv], buffer=<optimized out>) at
> ../libs/gst/base/gstbasetransform.c:2391
> #43 0x00007f0c435a9775 in gst_pad_chain_data_unchecked
> (pad=pad@entry=0x7f09b8033cb0 [GstPad|sink], type=type@entry=4112,
> data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
> #44 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09c401a550
> [GstPad|proxypad810], type=type@entry=4112, data=data@entry=0x7f0b40013c60)
> at ../gst/gstpad.c:4739
> #45 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09c401a550
> [GstPad|proxypad810], buffer=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
> #46 0x00007f0c43592aeb in gst_proxy_pad_chain_default (pad=<optimized out>,
> parent=<optimized out>, buffer=0x7f0b40013c60 [None]) at
> ../gst/gstghostpad.c:127
> #47 0x00007f0c435a9775 in gst_pad_chain_data_unchecked
> (pad=pad@entry=0x7f09c0023640 [GstPad|sink], type=type@entry=4112,
> data=data@entry=0x7f0b40013c60) at ../gst/gstpad.c:4463
> #48 0x00007f0c435ad114 in gst_pad_push_data (pad=pad@entry=0x7f09b802ab00
> [GstPad|src], type=type@entry=4112, data=data@entry=0x7f0b40013c60) at
> ../gst/gstpad.c:4739
> #49 0x00007f0c435ad4de in gst_pad_push (pad=0x7f09b802ab00 [GstPad|src],
> buffer=buffer@entry=0x7f0b40013c60 [None]) at ../gst/gstpad.c:4858
> #50 0x00007f0bd40545ec in gst_queue_push_one (queue=0x7f09f000c760
> [GstQueue|vqueue]) at ../plugins/elements/gstqueue.c:1388
> #51 gst_queue_loop (pad=<optimized out>) at
> ../plugins/elements/gstqueue.c:1541
> #52 0x00007f0c435d5d41 in gst_task_func (task=0x7f09d001fcb0
> [GstTask|vqueue:src]) at ../gst/gsttask.c:384
> #53 0x00007f0c437e2692 in g_thread_pool_thread_proxy (data=<optimized out>)
> at ../glib/gthreadpool.c:350
> #54 0x00007f0c437df722 in g_thread_proxy (data=0x7f0b5c014640) at
> ../glib/gthread.c:827
> #55 0x00007f0c4304e1da in start_thread (arg=<optimized out>) at
> pthread_create.c:442
> #56 0x00007f0c430d6f44 in clone () at
> ../sysdeps/unix/sysv/linux/x86_64/clone.S:100

Yes, its is strange.

I have checked that the wayland environment works inside the flatpak SDK.

📦🌐🐱 org.webkit.Sdk WPE@Release build]$ XDG_RUNTIME_DIR=/run/user/1000 WAYLAND_DISPLAY=wayland-0 /home/buildbot/waylandes2info | curl -F 'sprunge=<-' http://sprunge.us
http://sprunge.us/Qll6fh

You can see how it prints a different Mesa version than the previous one (that one was with the Mesa from the system)

But I don't know why it triggers rendering with swrast. The env var LIBGL_ALWAYS_SOFTWARE is only defined for the GTK port with the xvfb/weston display servers, but not for WPE

Please drop me a line if you want access to the server to debug it.

[Carlos Garcia Campos]

We run tests with LIBGL_ALWAYS_SOFTWARE to force swrast.

[Philippe Normand]

(In reply to Carlos Garcia Campos from comment #6)
> We run tests with LIBGL_ALWAYS_SOFTWARE to force swrast.

For GTK yes. For WPE, I'm not so sure about that.

[Carlos Alberto Lopez Perez]

(In reply to Philippe Normand from comment #7)
> (In reply to Carlos Garcia Campos from comment #6)
> > We run tests with LIBGL_ALWAYS_SOFTWARE to force swrast.
> 
> For GTK yes. For WPE, I'm not so sure about that.

For WPE we don't.

This are the env vars that the WPEWebProcess has on a run on this wpe release test bot right now

bb-wpe-release-test-64 ~ # cat /proc/$(pidof WPEWebProcess|awk '{print $1}')/environ|strings|sort
ASAN_OPTIONS=allocator_may_return_null=1
AT_SPI_BUS_ADDRESS=unix:path=/tmp/webkit-flatpak-a11y-sockets-xld0_v1j/tmpvtu70rpf
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
DIRHELPER_USER_DIR_SUFFIX=WebKitTestRunners-gev2x2w2
DUMPRENDERTREE_TEMP=/tmp/WebKitTestRunners-gev2x2w2
DYLD_FRAMEWORK_PATH=/app/webkit/WebKitBuild/Release
DYLD_LIBRARY_PATH=/app/webkit/WebKitBuild/Release
EGL_PLATFORM=wayland
G_DEBUG=fatal-criticals
GSETTINGS_BACKEND=memory
GST_PLUGIN_FEATURE_RANK=fakeaudiosink:max
HOME=/tmp/WebKitTestRunners-gev2x2w2/home
JSC_maxPerThreadStackUsage=1572864
JSC_useKernTCSM=false
JSC_useSharedArrayBuffer=1
LANG=en_US.UTF-8
LD_LIBRARY_PATH=/app/webkit/WebKitBuild/Release/lib
LOCAL_RESOURCE_ROOT=/app/webkit/LayoutTests
PATH=/usr/lib/sdk/llvm14/bin:/usr/bin:/usr/lib/sdk/rust-stable/bin/
SQLITE_EXEMPT_PATH_FROM_VNODE_GUARDS=/
TERM=dumb
TEST_RUNNER_INJECTED_BUNDLE_FILENAME=/app/webkit/WebKitBuild/Release/lib/libTestRunnerInjectedBundle.so
TEST_RUNNER_TEST_PLUGIN_PATH=/app/webkit/WebKitBuild/Release/lib/plugins
TMPDIR=/tmp/WebKitTestRunners-gev2x2w2
TZ=US/Pacific
WEBKIT_A11Y_BUS_ADDRESS=
WEBKIT_DISABLE_MEMORY_PRESSURE_MONITOR=1
WEBKIT_EXEC_PATH=/app/webkit/WebKitBuild/Release/bin
WEBKIT_FONTS_CONF_DIR=/app/webkit/Tools/WebKitTestRunner/gtk/fonts
WEBKIT_GST_DMABUF_SINK_DISABLED=1
WEBKIT_JHBUILD=0
WPE_USE_HEADLESS_VIEW_BACKEND=1
XDG_CACHE_HOME=/tmp/WebKitTestRunners-gev2x2w2/appcache
XDG_DATA_DIRS=/app/share:/usr/share:/usr/share/runtime/share:/run/host/share
XDG_RUNTIME_DIR=/run/user/1000
__XPC_ASAN_OPTIONS=allocator_may_return_null=1
__XPC_DYLD_FRAMEWORK_PATH=/app/webkit/WebKitBuild/Release
__XPC_DYLD_LIBRARY_PATH=/app/webkit/WebKitBuild/Release
__XPC_JSC_maxPerThreadStackUsage=1572864
__XPC_JSC_useKernTCSM=false
__XPC_JSC_useSharedArrayBuffer=1
__XPC_SQLITE_EXEMPT_PATH_FROM_VNODE_GUARDS=/




# Note: WEBKIT_GST_DMABUF_SINK_DISABLED=1 was added by me later on https://bugs.webkit.org/show_bug.cgi?id=253085#c2

[Carlos Garcia Campos]

But WTR uses the headless view backend on WPE.

[Carlos Alberto Lopez Perez]

(In reply to Carlos Garcia Campos from comment #9)
> But WTR uses the headless view backend on WPE.

that is right... but the headless view backend doesn't need the swrast mesa DRI driver, or does it?

I mean: if you only have swrast it would use it, but if you have a real GPU with its own DRI driver shouldn't use that one instead of swrast?


If I run from the debian system I get that it uses the i965_dri.so driver

bb-wpe-release-test-64 ~ # XDG_RUNTIME_DIR=/run/user/1000 WAYLAND_DISPLAY=wayland-0 strace -f /home/buildbot/waylandes2info 2>&1|grep dri.so
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/dri/tls/i965_dri.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/dri/i965_dri.so", O_RDONLY|O_CLOEXEC) = 5


And from the flatpak environment it using the crocus_dri.so driver which seems to be the new DRI driver for Intel hardware on Mesa.

[📦🌐🐱 org.webkit.Sdk WPE@Release build]$ XDG_RUNTIME_DIR=/run/user/1000 WAYLAND_DISPLAY=wayland-0 strace -f /home/buildbot/waylandes2info 2>&1|grep dri.*so
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libxcb-dri2.so.0", O_RDONLY|O_CLOEXEC) = 4
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/libxcb-dri3.so.0", O_RDONLY|O_CLOEXEC) = 4
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/GL/default/lib/dri/tls/crocus_dri.so", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
openat(AT_FDCWD, "/usr/lib/x86_64-linux-gnu/GL/default/lib/dri/crocus_dri.so", O_RDONLY|O_CLOEXEC) = 5


I can see it using that crocus DRI driver in this other crash log https://people.igalia.com/clopez/wkbug/253085/layout-test-results/default/imported/w3c/web-platform-tests/html/semantics/embedded-content/media-elements/autoplay-with-broken-track-crash-log.txt

[Carlos Alberto Lopez Perez]

(In reply to Carlos Alberto Lopez Perez from comment #10)
> 
> I can see it using that crocus DRI driver in this other crash log
> https://people.igalia.com/clopez/wkbug/253085/layout-test-results/default/
> imported/w3c/web-platform-tests/html/semantics/embedded-content/media-
> elements/autoplay-with-broken-track-crash-log.txt

Or not.. it is calling functions that start with crocus_ inside swrast_dri.so 🤔

[Philippe Normand]

(In reply to Carlos Garcia Campos from comment #9)
> But WTR uses the headless view backend on WPE.

Right, and checking HeadlessViewBackendFdo I see the rendering is done with Cairo from SHM data, so... seems like rendering done in CPU indeed.

I wonder how we can detect this and make the player fallback to the GL sink in this situation...

[Carlos Alberto Lopez Perez]

(In reply to Philippe Normand from comment #12)
> (In reply to Carlos Garcia Campos from comment #9)
> > But WTR uses the headless view backend on WPE.
> 
> Right, and checking HeadlessViewBackendFdo I see the rendering is done with
> Cairo from SHM data, so... seems like rendering done in CPU indeed.
> 
> I wonder how we can detect this and make the player fallback to the GL sink
> in this situation...


The easiest one is to check for the env var WPE_USE_HEADLESS_VIEW_BACKEND=1 that Tools/Scripts/webkitpy/port/headlessdriver.py defines

[Carlos Alberto Lopez Perez]

(In reply to Carlos Alberto Lopez Perez from comment #13)
> (In reply to Philippe Normand from comment #12)
> 
> The easiest one is to check for the env var WPE_USE_HEADLESS_VIEW_BACKEND=1
> that Tools/Scripts/webkitpy/port/headlessdriver.py defines

But maybe there is a better way of detecting this.

I wonder if this would be also an issue with other backends like Cog headless or the ones from WPEBackend-rdk

[Philippe Normand]

I don't think cog headless is affected by this, because it performs no rendering whatsoever.

[Philippe Normand]

Pull request: https://github.com/WebKit/WebKit/pull/10867

[Philippe Normand]

(In reply to Philippe Normand from comment #16)
> Pull request: https://github.com/WebKit/WebKit/pull/10867

Can you test on the bot please?

[Zan Dobersek]

Same but different, observed under Debian Unstable, running WKTR over LayoutTests/media/video-aspect-ratio.html:

#0  0x00007f30b6cc887c in si_switch_compute_shader (prefetch=<synthetic pointer>, offset=<optimized out>, code_object=0x0, shader=0x7f14ac12bba0, program=<optimized out>, sctx=0x7f14ac013590) at ../src/gallium/drivers/radeonsi/si_compute.c:562
        cs = 0x7f14ac013a20
        config = 0x7f14ac12bda0
        shader_va = <optimized out>
        __cs = <optimized out>
        __cs_num_initial = <optimized out>
        __cs_buf = <optimized out>
        inline_config = {num_sgprs = 0, num_vgprs = 0, num_shared_vgprs = 0, spilled_sgprs = 0, spilled_vgprs = 0, lds_size = 0, spi_ps_input_ena = 0, spi_ps_input_addr = 0, float_mode = 0, scratch_bytes_per_wave = 0, rsrc1 = 0, rsrc2 = 0, rsrc3 = 0}
        tmpring_size = 1152
        __cs_num = <optimized out>
        sctx = 0x7f14ac013590
        sscreen = <optimized out>
        program = <optimized out>
        code_object = 0x0
        i = <optimized out>
        cs_regalloc_hang = <optimized out>
        prefetch = false
        display_dcc_store_mask = <optimized out>
#1  si_launch_grid (ctx=0x7f14ac013590, info=0x7f14d5fd8b00) at ../src/gallium/drivers/radeonsi/si_compute.c:1007
        sctx = 0x7f14ac013590
        sscreen = <optimized out>
        program = <optimized out>
        code_object = 0x0
        i = <optimized out>
        cs_regalloc_hang = <optimized out>
        prefetch = false
        display_dcc_store_mask = <optimized out>
#2  0x00007f30b6cca094 in si_launch_grid_internal (sctx=sctx@entry=0x7f14ac013590, info=info@entry=0x7f14d5fd8b00, shader=shader@entry=0x7f14a00094b0, flags=flags@entry=47) at ../src/gallium/drivers/radeonsi/si_compute_blit.c:198
        saved_cs = 0x0
#3  0x00007f30b6cca584 in si_launch_grid_internal_images (sctx=sctx@entry=0x7f14ac013590, images=images@entry=0x7f14d5fd8ad0, num_images=num_images@entry=2, info=info@entry=0x7f14d5fd8b00, shader=0x7f14a00094b0, flags=flags@entry=15) at ../src/gallium/drivers/radeonsi/si_compute_blit.c:615
        saved_image = {{resource = 0x0, format = PIPE_FORMAT_NONE, access = 0, shader_access = 0, u = {tex = {first_layer = 0, last_layer = 0, level = 0}, buf = {offset = 0, size = 0}}}, {resource = 0x0, format = PIPE_FORMAT_NONE, access = 0, shader_access = 0, u = {tex = {first_layer = 0, last_layer = 0, level = 0}, buf = {offset = 0, size = 0}}}}
#4  0x00007f30b6ccb8df in si_compute_copy_image (sctx=sctx@entry=0x7f14ac013590, dst=dst@entry=0x7f14a000e180, dst_level=dst_level@entry=0, src=src@entry=0x7f14a0010380, src_level=src_level@entry=0, dstx=dstx@entry=0, dsty=<optimized out>, dstz=0, src_box=<optimized out>, flags=15) at ../src/gallium/drivers/radeonsi/si_compute_blit.c:789
        ssrc = 0x7f14a0010380
        sdst = 0x7f14a000e180
        src_format = <optimized out>
        dst_format = <optimized out>
        is_linear = true
        new_box = {x = 5, y = 0, z = 0, width = 0, height = 0, depth = 0}
        src_access = 0
        dst_access = 0
        image = {{resource = 0x7f14a0010380, format = PIPE_FORMAT_R8_UNORM, access = 1, shader_access = 1, u = {tex = {first_layer = 0, last_layer = 0, level = 0}, buf = {offset = 0, size = 0}}}, {resource = 0x7f14a000e180, format = PIPE_FORMAT_R8_UNORM, access = 2, shader_access = 2, u = {tex = {first_layer = 0, last_layer = 0, level = 0}, buf = {offset = 0, size = 0}}}}
        info = {pc = 0, input = 0x0, work_dim = 0, block = {64, 1, 1}, last_block = {0, 0, 0}, grid = {5, 240, 1}, grid_base = {0, 0, 0}, indirect = 0x0, indirect_offset = 0}
        dst_is_1d = <optimized out>
        src_is_1d = <optimized out>
        block_x = <optimized out>
        block_y = <optimized out>
        block_z = 1
        copy_image_cs_ptr = <optimized out>
#5  0x00007f30b6cc1a58 in si_resource_copy_region (ctx=ctx@entry=0x7f14ac013590, dst=dst@entry=0x7f14a000e180, dst_level=dst_level@entry=0, dstx=0, dsty=0, dstz=0, src=0x7f14a0010380, src_level=0, src_box=0x7f14d5fd8d00) at ../src/gallium/drivers/radeonsi/si_blit.c:915
        sctx = 0x7f14ac013590
        ssrc = 0x7f14a0010380
        dst_view = <optimized out>
        dst_templ = {reference = {count = -538976289}, format = 4112, writable = 0, texture = 0x10101010dfdfdfdf, context = 0x10101010dfdfdfdf, width = 57311, height = 57311, nr_samples = 16, u = {tex = {level = 3755991007, first_layer = 4112, last_layer = 4112}, buf = {first_element = 3755991007, last_element = 269488144}}}
        src_templ = {{__cl_space = "\337\337\337\337\020\020\020\020\337\337\337\337\020\020\020\020", '\000' <repeats 32 times>, "\337\337\337\337\020\020\020\020\337\337\337\337\020\020\020\020", reference = {count = -538976289}}, format = 24543, target = 31, swizzle_r = 5, swizzle_g = 7, swizzle_b = 7, swizzle_a = 6, texture = 0x10101010dfdfdfdf, context = 0x6969694b1b0c691b, u = {tex = {first_layer = 27080, last_layer = 26985, first_level = 105, last_level = 105}, buf = {offset = 1768516040, size = 1768515945}}}
        src_view = <optimized out>
        dstbox = {x = -572596256, y = 4112, z = 4112, width = -538976289, height = 4112, depth = 4112}
#6  0x00007f30b6c93b5b in si_copy_from_staging_texture (stransfer=0x7f14a00102e0, ctx=0x7f14ac013590) at ../src/gallium/drivers/radeonsi/si_texture.c:127
        transfer = 0x7f14a00102e0
        dst = 0x7f14a000e180
        src = 0x7f14a0010380
        sbox = {x = 0, y = 0, z = 0, width = 320, height = 240, depth = 1}
        sctx = 0x7f14ac013590
#7  si_texture_transfer_unmap (ctx=0x7f14ac013590, transfer=0x7f14a00102e0) at ../src/gallium/drivers/radeonsi/si_texture.c:1983
        sctx = 0x7f14ac013590
#8  0x00007f30b6a23fce in tc_call_texture_unmap (pipe=<optimized out>, call=<optimized out>, last=<optimized out>) at ../src/gallium/auxiliary/util/u_threaded_context.c:2884
        p = <optimized out>
#9  0x00007f30b6a242fa in batch_execute (parsing=false, last=0x7f14ac058730, pipe=0x7f14ac013590, batch=0x7f14ac0586f8) at ../src/gallium/auxiliary/util/u_threaded_context.c:312
        call = 0x7f14ac058720
        iter = 0x7f14ac058720
        first = <optimized out>
        batch = 0x7f14ac0586f8
        pipe = 0x7f14ac013590
        last = 0x7f14ac058730
        tc = <optimized out>
        fence = <optimized out>
#10 tc_batch_execute (job=job@entry=0x7f14ac0586f8, gdata=gdata@entry=0x0, thread_index=thread_index@entry=0) at ../src/gallium/auxiliary/util/u_threaded_context.c:356
        batch = 0x7f14ac0586f8
        pipe = 0x7f14ac013590
        last = 0x7f14ac058730
        tc = <optimized out>
        fence = <optimized out>
#11 0x00007f30b6a25336 in _tc_sync (tc=tc@entry=0x7f14ac0566a0, func=<optimized out>, info=<optimized out>) at ../src/gallium/auxiliary/util/u_threaded_context.c:575
        last = <optimized out>
        next = 0x7f14ac0586f8
        synced = <optimized out>
#12 0x00007f30b6a2c258 in tc_flush (_pipe=0x7f14ac0566a0, fence=0x0, flags=1) at ../src/gallium/auxiliary/util/u_threaded_context.c:3452
        tc = 0x7f14ac0566a0
        pipe = 0x7f14ac013590
        screen = 0x556623ef6200
#13 0x00007f30b6576946 in st_context_flush (stctxi=0x7f14ac0e50a0, flags=3, fence=0x0, before_flush_cb=0x0, args=0x7f14d5fd8e70) at ../src/mesa/state_tracker/st_manager.c:813
        st = 0x7f14ac0e50a0
        pipe_flags = 1
#14 0x00007f30b64b2c99 in dri_flush (cPriv=<optimized out>, dPriv=<optimized out>, flags=<optimized out>, reason=<optimized out>) at ../src/gallium/frontends/dri/dri_drawable.c:537
        ctx = 0x7f14ac010170
        drawable = 0x0
        st = 0x7f14ac0e50a0
        flush_flags = <optimized out>
        args = {ctx = 0x0, drawable = 0x0, flags = 0, reason = __DRI2_THROTTLE_SWAPBUFFER, swap_msaa_buffers = false}
#15 0x00007f311b5eefcc in WebCore::MediaPlayerPrivateGStreamer::pushDMABufToCompositor() () at /build/webkit/build-unstable/build-webkit/lib/libWPEWebKit-1.0.so.3
#16 0x00007f311b5dfc2d in webKitVideoSinkSetMediaPlayerPrivate(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*)::{lambda(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*)#3}::_FUN(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*) () at /build/webkit/build-unstable/build-webkit/lib/libWPEWebKit-1.0.so.3
#17 ...

[Philippe Normand]

Looks like some Mesa bug?

[Carlos Alberto Lopez Perez]

I can reproduce the failure also on my laptop (Intel Iris GPU) with Debian+flatpak

And I can also reproduce it if I open with Cog the html file LayoutTests/media/video-aspect-ratio.html directly

With Cog (under wayland) I get this backtrace when opening the html of the test:

Thread 1 (Thread 0x7f9dd5fef640 (LWP 156)):
#0  0x00007fb40520cdee in u_transfer_helper_transfer_unmap () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/iris_dri.so
#1  0x00007fb404ca68c2 in tc_call_texture_unmap () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/iris_dri.so
#2  0x00007fb404ca6c41 in tc_batch_execute () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/iris_dri.so
#3  0x00007fb404ca7d66 in _tc_sync.constprop.0 () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/iris_dri.so
#4  0x00007fb404caf128 in tc_flush () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/iris_dri.so
#5  0x00007fb4047d92ae in st_context_flush () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/iris_dri.so
#6  0x00007fb40470e688 in dri_flush () at /usr/lib/x86_64-linux-gnu/GL/default/lib/dri/iris_dri.so
#7  0x00007fb46bbc4ab9 in WebCore::MediaPlayerPrivateGStreamer::pushDMABufToCompositor() () at /app/webkit/WebKitBuild/Release/lib/libWPEWebKit-2.0.so.1
#8  0x00007fb46bbb264c in webKitVideoSinkSetMediaPlayerPrivate(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*)::{lambda(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*)#2}::_FUN(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*) () at /app/webkit/WebKitBuild/Release/lib/libWPEWebKit-2.0.so.1
#9  0x00007fb461afb056 in ffi_call_unix64 () at ../src/x86/unix64.S:104


full bt: http://sprunge.us/ZD6I6x


You can see how with Cog it uses iris_dri.so instead of swrast_dri.so but still crashes

If I set WEBKIT_GST_DMABUF_SINK_DISABLED=1 then it works

[Philippe Normand]

I don't think we can do much here, these crashes look like issues in Mesa...

Maybe we enabled this sink too early? We can revert this but we'd also need some Mesa bugreport and investigation...

[Carlos Alberto Lopez Perez]

(In reply to Philippe Normand from comment #21)
> I don't think we can do much here, these crashes look like issues in Mesa...
> 
> Maybe we enabled this sink too early? We can revert this but we'd also need
> some Mesa bugreport and investigation...

Can you reproduce the issue or is only us?

Wondering if maybe it has to be with the Kernel version (perhaps this something that has been only implemented on newer kernels, don't know). I only tested with kernel 5.10 so far.

[Carlos Alberto Lopez Perez]

(In reply to Carlos Alberto Lopez Perez from comment #20)
> And I can also reproduce it if I open with Cog the html file
> LayoutTests/media/video-aspect-ratio.html directly
> 
> With Cog (under wayland) I get this backtrace when opening the html of the
> test:

The crash doesn't happen if I use the MiniBrowser instead of Cog to open that file (by setting env var WPE_BROWSER=minibrowser before calling run-minibrowser script)

[Carlos Alberto Lopez Perez]

(In reply to Carlos Alberto Lopez Perez from comment #23)
> (In reply to Carlos Alberto Lopez Perez from comment #20)
> > And I can also reproduce it if I open with Cog the html file
> > LayoutTests/media/video-aspect-ratio.html directly
> > 
> > With Cog (under wayland) I get this backtrace when opening the html of the
> > test:
> 
> The crash doesn't happen if I use the MiniBrowser instead of Cog to open
> that file (by setting env var WPE_BROWSER=minibrowser before calling
> run-minibrowser script)

Forget that.

It also happens, but much less often. With Cog also it doesn't happen always.
Let's say: of each try that I did Cog crashes 90% of the times and MB 10%.
I also can't get a coredump when running with MB for some unknown reason

[Zan Dobersek]

From my side, the bug is easily reproducible also in MiniBrowser, on content where multiple video players are active.

On my setup, an AMD GPU with software media decoding, the problem seems to be uploading of decoded content into mapped gbm_bo objects. With multiple pipelines acting at the same time, this ends up being done concurrently from different threads. And given it's faulty in MiniBrowser, it's not exclusively an issue of swrast, since I end up using the radeonsi driver. It can end up crashing the GPU too, which is fun.

[Zan Dobersek]

The crash I'm observing occurs when two or more pipelines end up mapping separate gbm_bo objects on separate threads through the same gbm_device.

The single gbm_device (handled in the GBMDevice class) is necessary, since the object created through it travel across threads. Except for this mapping operation in the GStreamer pipeline, other operations done through gbm_device are simple, but a more thorough exploration about thread-safety guarantees of libgbm in general is needed.

The crashes are avoidable by setting up a global map/unmap lock and locking it for each occasion of copying software-decoded data into the gbm_bo object.

[Zan Dobersek]

Created attachment 465257 [details]
Locking patch

[Philippe Normand]

Comment on attachment 465257 [details]
Locking patch

Great! Can you send a PR?

[Zan Dobersek]

Yes, of course, I just pasted it here if others want to try. I still have to try it out on WKTR and ideally also the layout test suite.

[Philippe Normand]

(In reply to Philippe Normand from comment #17)
> (In reply to Philippe Normand from comment #16)
> > Pull request: https://github.com/WebKit/WebKit/pull/10867
> 
> Can you test on the bot please?

I think we still need this PR too ^^

[Zan Dobersek]

(In reply to Philippe Normand from comment #30)
> (In reply to Philippe Normand from comment #17)
> > (In reply to Philippe Normand from comment #16)
> > > Pull request: https://github.com/WebKit/WebKit/pull/10867
> > 
> > Can you test on the bot please?
> 
> I think we still need this PR too ^^

Let's see how locking affects this. My thinking is this is the same problem, and more video tests would fail if there was a general issue of dmabuf-outputting playback in WKTR in headless mode.

[Carlos Alberto Lopez Perez]

(In reply to Zan Dobersek from comment #27)
> Created attachment 465257 [details]
> Locking patch

Tested on my latop: Intel GPU, Debian system, Wayland environment, using flatpak build.

It fixes the crash when running Cog/MB over LayoutTests/media/video-aspect-ratio.html

But I still get crashes when running Cog/MB with this two html files

LayoutTests/fast/canvas/webgl/texImage2D-video-flipY-false.html
LayoutTests/fast/canvas/webgl/texImage2D-video-flipY-true.html

* texImage2D-video-flipY-false crash log :

Thread 1 (Thread 0x7f7138fce640 (LWP 92)):
#0  0x00007f7243f43ecc in WebCore::MediaPlayerPrivateGStreamer::pushDMABufToCompositor() () at /app/webkit/WebKitBuild/Release/lib/libWPEWebKit-2.0.so.1
#1  0x00007f7243f31e3c in webKitVideoSinkSetMediaPlayerPrivate(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*)::{lambda(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*)#2}::_FUN(_GstElement*, WebCore::MediaPlayerPrivateGStreamer*) () at /app/webkit/WebKitBuild/Release/lib/libWPEWebKit-2.0.so.1
#2  0x00007f7239e75056 in ffi_call_unix64 () at ../src/x86/unix64.S:104
#3  0x00007f7239e73add in ffi_call_int (cif=<optimized out>, fn=<optimized out>, rvalue=<optimized out>, avalue=<optimized out>, closure=<optimized out>) at ../src/x86/ffi64.c:673

Full BT -> http://sprunge.us/GrIvUJ

* texImage2D-video-flipY-true  crash log : 

#0  0x00007f10a37780cd in __GI___poll (fds=0x7f0ea8003980, nfds=2, timeout=-1) at ../sysdeps/unix/sysv/linux/poll.c:29
        sc_ret = -516
        sc_cancel_oldtype = 0
#1  0x00007f10a3eb2f9e in g_main_context_poll (priority=<optimized out>, n_fds=2, fds=0x7f0ea8003980, timeout=<optimized out>, context=0x7f0ea8000b70) at ../glib/gmain.c:4518
        ret = <optimized out>
        errsv = <optimized out>
        poll_func = 0x7f10a3e62720 <g_poll>
        max_priority = 2147483647
        timeout = -1
        some_ready = <optimized out>
        nfds = 2
        allocated_nfds = <optimized out>
        fds = 0x7f0ea8003980
#2  g_main_context_iterate.constprop.0 (context=0x7f0ea8000b70, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4208
        max_priority = 2147483647
        timeout = -1
        some_ready = <optimized out>
        nfds = 2
        allocated_nfds = <optimized out>
        fds = 0x7f0ea8003980
#3  0x00007f10a3e5c2af in g_main_loop_run (loop=0x7f0ea8002b50) at ../glib/gmain.c:4413

Full BT -> http://sprunge.us/yGf0Nh


And when running layout tests with the command from https://bugs.webkit.org/show_bug.cgi?id=253085#c1 (with headlessview backend on a system with GPU; my laptop with Intel GPU) I get this crash logs:

https://people.igalia.com/clopez/wkbug/253085/layout-test-results/patch_id_bugzilla_465257/results.html

Most of them have a backtrace very similar to texImage2D-video-flipY-false above (crashing at WebCore::MediaPlayerPrivateGStreamer::pushDMABufToCompositor()). They not longer are calling into swrast_dri.so

[Zan Dobersek]

Created attachment 465260 [details]
Null swapchain buffer

[Carlos Alberto Lopez Perez]

(In reply to Zan Dobersek from comment #33)
> Created attachment 465260 [details]
> Null swapchain buffer

Amazing!

With this patch on top everything works fine to me now :)

I can't not longer reproduce any crash with Cog or running the layout tests.

I get a few random crashes on imported/w3c/web-platform-tests/webcodecs/videoFrame-canvasImageSource.html and imported/w3c/web-platform-tests/html/user-activation/activation-trigger-pointerevent.html?touch but those also that happens with WEBKIT_GST_DMABUF_SINK_DISABLED=1 and looking at wktesthunter results it looks this have been happening since a while. So unrelated to this, those just need to be gardened. Also the back-trace on those is very different.

[Zan Dobersek]

(In reply to Zan Dobersek from comment #33)
> Created attachment 465260 [details]
> Null swapchain buffer

This early-returns when a null buffer is returned from the swapchain, i.e. all the buffers are consumed. It's healthy to do it, but not exactly expected.

It happens with the given tests because the actual video-associated layer isn't composited, so the proxy object isn't active, so the dmabuf is never pushed through for display, and subsequently never released. To avoid this, I'll also move the is-proxy-active check before any buffer is retrieved from the swapchain.

[Carlos Alberto Lopez Perez]

(In reply to Carlos Alberto Lopez Perez from comment #2)
> Note: I have configured the WPE Release test post-commit bot to run with the
> environment var WEBKIT_GST_DMABUF_SINK_DISABLED=1 until this issue is fixed.
> 
> The first build running with that environment var is
> https://build.webkit.org/#/builders/40/builds/9225

Just removed that env var from the bot environment

First run without it will be https://build.webkit.org/#/builders/40/builds/9331

[Philippe Normand]

Seems like it went "well"? Can this be closed?

[Philippe Normand]

(In reply to Philippe Normand from comment #37)
> Seems like it went "well"? Can this be closed?

Ping

[Carlos Alberto Lopez Perez]

(In reply to Philippe Normand from comment #38)
> (In reply to Philippe Normand from comment #37)
> > Seems like it went "well"? Can this be closed?
> 
> Ping

Yes.

Closing

*** This bug has been marked as a duplicate of bug 253244 ***