Bug 308119 – [GStreamer] WebKitGLVideoSink disposed from wrong thread

• Component: WPE WebKit
• Status: NEW

[Claudio Saavedra]

Seen in EWS WPE (assertions enabled) with fast/mediastream/mediastream-gc.html

https://ews-build.s3-us-west-2.amazonaws.com/WPE-WK2-Tests-EWS/7567acfe-111794/results.html

STDERR: ASSERTION FAILED: isMainThread()
STDERR: ../../../Source/WebCore/platform/graphics/gstreamer/GLVideoSinkGStreamer.cpp(58) : _WebKitGLVideoSinkPrivate::~_WebKitGLVideoSinkPrivate()
STDERR: CONSOLE MESSAGE: A capture MediaStreamTrack was destroyed without having been stopped explicitly
STDERR: CONSOLE MESSAGE: A capture MediaStreamTrack was destroyed without having been stopped explicitly
STDERR: CONSOLE MESSAGE: A capture MediaStreamTrack was destroyed without having been stopped explicitly
STDERR: CONSOLE MESSAGE: A capture MediaStreamTrack was destroyed without having been stopped explicitly
STDERR: 0:00:02.412831514 1144765 0x555aed7b35b0 ERROR             video-info video-info.c:595:gst_video_info_from_caps: no format given
STDERR: CONSOLE MESSAGE: A capture MediaStreamTrack was destroyed without having been stopped explicitly
STDERR: CONSOLE MESSAGE: A capture MediaStreamTrack was destroyed without having been stopped explicitly
STDERR: CONSOLE MESSAGE: A capture MediaStreamTrack was destroyed without having been stopped explicitly
STDERR: CONSOLE MESSAGE: A capture MediaStreamTrack was destroyed without having been stopped explicitly
STDERR: 1   0x7fd7abd0df5c webkit_gl_video_sink_finalize(_GObject*)
STDERR: 2   0x7fd79f38661d g_object_unref
STDERR: 3   0x7fd5b474596a gst_play_bin3_finalize
STDERR: 4   0x7fd79f38661d g_object_unref
STDERR: 5   0x7fd7a002b469 gst_element_call_async_func
STDERR: 6   0x7fd79f2aa422 ???
STDERR: 7   0x7fd79f2a4e62 ???
STDERR: 8   0x7fd7a028aaa4 start_thread
STDERR: 9   0x7fd7a0317c6c clone3

[Philippe Normand]

This is quite odd, hints to a refcount issue... The pipeline shouldn't be disposed within the call_async callback...

[Philippe Normand]

Also the pipeline is reffed before sending to the task pool... see https://gitlab.freedesktop.org/gstreamer/gstreamer/-/blob/1.26/subprojects/gstreamer/gst/gstelement.c?ref_type=heads#L3875

[Claudio Saavedra]

Does this thread tell you anything? from  https://ews-build.s3-us-west-2.amazonaws.com/WPE-WK2-Tests-EWS/7567acfe-111794/results.html

Thread 23 (Thread 0x7fd798b20540 (LWP 1144765)):
#0  __strlen_evex () at ../sysdeps/x86_64/multiarch/strlen-evex-base.S:84
#1  0x00007fd79f29148d in g_strdup () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#2  0x00007fd79f39b0b9 in ??? () at /lib/x86_64-linux-gnu/libgobject-2.0.so.0
#3  0x00007fd7a00ab990 in gst_value_intersect (dest=0x555aedc472a8, value1=0x555aedc51268, value2=0x555aedbeb618) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstvalue.c:6423
#4  0x00007fd7a00ac3f1 in gst_value_intersect_list_list (value2=<optimized out>, value1=<optimized out>, dest=<optimized out>) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstvalue.c:5157
#5  gst_value_intersect_list (dest=0x7ffcaedd5d10, value1=<optimized out>, value2=0x555aedbef190) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstvalue.c:5208
#6  0x00007fd7a00842b7 in gst_structure_intersect (struct1=struct1@entry=0x555aedc4dd60, struct2=struct2@entry=0x555aedbef0d0) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gststructure.c:4344
#7  0x00007fd7a001a329 in gst_caps_intersect_first (caps2=0x555aedb689b0, caps1=0x555aedc39ce0) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstcaps.c:2078
#8  gst_caps_intersect_full (caps1=caps1@entry=0x555aedc39ce0, caps2=caps2@entry=0x555aedb689b0, mode=mode@entry=GST_CAPS_INTERSECT_FIRST) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstcaps.c:2133
#9  0x00007fd79ea57801 in gst_base_transform_query_caps (filter=0x0, pad=0x555aedbd2510, trans=0x555aedc50930) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/libs/gst/base/gstbasetransform.c:716
#10 gst_base_transform_default_query (trans=0x555aedc50930, direction=<optimized out>, query=0x555aedc52850) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/libs/gst/base/gstbasetransform.c:1605
#11 0x00007fd7a0056f2e in gst_pad_query (pad=pad@entry=0x555aedbd2510, query=query@entry=0x555aedc52850) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstpad.c:4293
#12 0x00007fd7a0057a5e in gst_pad_peer_query (pad=pad@entry=0x555aedc30fa0, query=0x555aedc52850) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstpad.c:4430
#13 0x00007fd7a009808c in query_caps_func (pad=pad@entry=0x555aedc30fa0, data=data@entry=0x7ffcaedd6090) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstutils.c:2815
#14 0x00007fd7a005593a in gst_pad_forward (pad=pad@entry=0x555aedc3fa80, forward=forward@entry=0x7fd7a0098060 <query_caps_func>, user_data=user_data@entry=0x7ffcaedd6090) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstpad.c:3123
#15 0x00007fd7a009a910 in gst_pad_proxy_query_caps (pad=pad@entry=0x555aedc3fa80, query=query@entry=0x555aedc52850) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstutils.c:2867
#16 0x00007fd7a0055c49 in gst_pad_query_caps_default (query=0x555aedc52850, pad=0x555aedc3fa80) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstpad.c:3309
#17 gst_pad_query_default (pad=0x555aedc3fa80, parent=<optimized out>, query=0x555aedc52850) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstpad.c:3537
#18 0x00007fd7a0056f2e in gst_pad_query (pad=pad@entry=0x555aedc3fa80, query=query@entry=0x555aedc52850) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstpad.c:4293
#19 0x00007fd7a009b09b in gst_pad_query_caps (pad=pad@entry=0x555aedc3fa80, filter=filter@entry=0x0) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstutils.c:3117
#20 0x00007fd7a009bacb in gst_element_get_compatible_pad (element=element@entry=0x555aedb9e9f0, pad=pad@entry=0x555aedc3fa80, caps=caps@entry=0x0) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstutils.c:1192
#21 0x00007fd7a009cb1d in gst_element_link_pads_full (src=0x555aedc4fc70, srcpadname=0x0, dest=0x555aedb9e9f0, destpadname=0x0, flags=GST_PAD_LINK_CHECK_DEFAULT) at ../../../../../jhbuild/checkout/gstreamer/subprojects/gstreamer/gst/gstutils.c:1941
#22 0x00007fd7abeaeb20 in WebCore::GStreamerVideoCapturer::createConverter() () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#23 0x00007fd7abe9ab2b in WebCore::GStreamerCapturer::setupPipeline() () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#24 0x00007fd7abeb8765 in WebCore::GStreamerVideoCapturer::setupPipeline() () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#25 0x00007fd7abec26df in WebCore::MockRealtimeVideoSourceGStreamer::MockRealtimeVideoSourceGStreamer(WTF::String&&, WTF::AtomString&&, WebCore::MediaDeviceHashSalts&&, std::optional<WTF::ObjectIdentifierGeneric<WebCore::PageIdentifierType, WTF::ObjectIdentifierMainThreadAccessTraits<unsigned long>, unsigned long> >) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#26 0x00007fd7abec32f4 in WebCore::MockRealtimeVideoSource::create(WTF::String&&, WTF::AtomString&&, WebCore::MediaDeviceHashSalts&&, WebCore::MediaConstraints const*, std::optional<WTF::ObjectIdentifierGeneric<WebCore::PageIdentifierType, WTF::ObjectIdentifierMainThreadAccessTraits<unsigned long>, unsigned long> >) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#27 0x00007fd7abed5d2f in WebCore::MockRealtimeVideoSourceFactory::createVideoCaptureSource(WebCore::CaptureDevice const&, WebCore::MediaDeviceHashSalts&&, WebCore::MediaConstraints const*, std::optional<WTF::ObjectIdentifierGeneric<WebCore::PageIdentifierType, WTF::ObjectIdentifierMainThreadAccessTraits<unsigned long>, unsigned long> >) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#28 0x00007fd7abe6946b in WebCore::RealtimeMediaSourceCenter::getUserMediaDevices(WebCore::MediaStreamRequest const&, WebCore::MediaDeviceHashSalts&&, WTF::Vector<WebCore::RealtimeMediaSourceCenter::DeviceInfo, 0ul, WTF::CrashOnOverflow, 16ul, WTF::FastMalloc>&, WTF::Vector<WebCore::RealtimeMediaSourceCenter::DeviceInfo, 0ul, WTF::CrashOnOverflow, 16ul, WTF::FastMalloc>&, WebCore::MediaConstraintType&) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#29 0x00007fd7abe6964b in WebCore::RealtimeMediaSourceCenter::validateRequestConstraintsAfterEnumeration(WebCore::MediaStreamRequest const&, WebCore::MediaDeviceHashSalts const&) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#30 0x00007fd7abe69dde in WTF::Detail::CallableWrapper<WebCore::RealtimeMediaSourceCenter::validateRequestConstraints(WTF::CompletionHandler<void (std::experimental::fundamentals_v3::expected<WebCore::RealtimeMediaSourceCenter::ValidDevices, WebCore::MediaConstraintType>&&)>&&, WebCore::MediaStreamRequest const&, WebCore::MediaDeviceHashSalts&&)::{lambda()#1}, void>::call() () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#31 0x00007fd7a542c2ae in WTF::CompletionHandler<void ()>::operator()() () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#32 0x00007fd7a55257e8 in WTF::Ref<WTF::CallbackAggregatorOnThread<(WTF::DestructionThread)0>, WTF::RawPtrTraits<WTF::CallbackAggregatorOnThread<(WTF::DestructionThread)0> >, WTF::DefaultRefDerefTraits<WTF::CallbackAggregatorOnThread<(WTF::DestructionThread)0> > >::~Ref() () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#33 0x00007fd7abe572cd in WebCore::RealtimeMediaSourceCenter::enumerateDevices(bool, bool, bool, bool, WTF::CompletionHandler<void ()>&&) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#34 0x00007fd7abe5b224 in WebCore::RealtimeMediaSourceCenter::validateRequestConstraints(WTF::CompletionHandler<void (std::experimental::fundamentals_v3::expected<WebCore::RealtimeMediaSourceCenter::ValidDevices, WebCore::MediaConstraintType>&&)>&&, WebCore::MediaStreamRequest const&, WebCore::MediaDeviceHashSalts&&) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#35 0x00007fd7a51c9dd4 in void IPC::handleMessageAsync<Messages::UserMediaCaptureManager::ValidateUserMediaRequestConstraints, IPC::Connection, WebKit::UserMediaCaptureManager, WebKit::UserMediaCaptureManager, void (WebCore::MediaStreamRequest const&, WebCore::MediaDeviceHashSalts&&, WTF::CompletionHandler<void (std::experimental::fundamentals_v3::expected<WebCore::RealtimeMediaSourceCenter::ValidDevices, WebCore::MediaConstraintType>&&)>&&)>(IPC::Connection&, IPC::Decoder&, WebKit::UserMediaCaptureManager*, void (WebKit::UserMediaCaptureManager::*)(WebCore::MediaStreamRequest const&, WebCore::MediaDeviceHashSalts&&, WTF::CompletionHandler<void (std::experimental::fundamentals_v3::expected<WebCore::RealtimeMediaSourceCenter::ValidDevices, WebCore::MediaConstraintType>&&)>&&)) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#36 0x00007fd7a51cc258 in non-virtual thunk to WebKit::UserMediaCaptureManager::didReceiveMessage(IPC::Connection&, IPC::Decoder&) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#37 0x00007fd7a57b5e83 in IPC::MessageReceiverMap::dispatchMessage(IPC::Connection&, IPC::Decoder&) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#38 0x00007fd7a57c6523 in WebKit::AuxiliaryProcess::dispatchMessage(IPC::Connection&, IPC::Decoder&) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#39 0x00007fd7a50532f6 in WebKit::AuxiliaryProcess::didReceiveMessage(IPC::Connection&, IPC::Decoder&) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#40 0x00007fd7a57ac7a0 in IPC::Connection::dispatchMessage(IPC::Decoder&) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#41 0x00007fd7a57adc35 in IPC::Connection::dispatchMessage(WTF::UniqueRef<IPC::Decoder>) [clone .part.0] () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#42 0x00007fd7a57ae2ff in IPC::Connection::dispatchOneIncomingMessage() () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#43 0x00007fd7a836a3f1 in WTF::RunLoop::performWork() () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#44 0x00007fd7a847896d in WTF::RunLoop::RunLoop()::{lambda(void*)#1}::_FUN(void*) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#45 0x00007fd7a847eb8a in WTF::RunLoop::{lambda(_GSource*, int (*)(void*), void*)#1}::_FUN(_GSource*, int (*)(void*), void*) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#46 0x00007fd79f27645e in ??? () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#47 0x00007fd79f2766d0 in g_main_context_dispatch () at /lib/x86_64-linux-gnu/libglib-2.0.so.0
#48 0x00007fd7a847b664 in WTF::RunLoop::runGLibMainLoopIteration(WTF::RunLoop::MayBlock) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#49 0x00007fd7a847bc72 in WTF::RunLoop::run() () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#50 0x00007fd7a60ac186 in int WebKit::AuxiliaryProcessMain<WebKit::WebProcessMainWPE>(int, char**) () at /sdk/webkit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#51 0x00007fd7a02181ca in __libc_start_call_main (main=main@entry=0x555ac1dc67d0 <main>, argc=argc@entry=4, argv=argv@entry=0x7ffcaedd88c8) at ../sysdeps/nptl/libc_start_call_main.h:58
#52 0x00007fd7a021828b in __libc_start_main_impl (main=0x555ac1dc67d0 <main>, argc=4, argv=0x7ffcaedd88c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffcaedd88b8) at ../csu/libc-start.c:360
#53 0x0000555ac1dc6705 in _start ()

[Philippe Normand]

That doesn't look related to the actual crash, I suppose when the assert triggered, some capture pipeline was negotiating caps. The assert happens in another pipeline.

[Philippe Normand]

I was able to reproduce this using stress-ng...

So it seems to be a race condition where the pipeline is teared down while the video sink is mid-flight in an async state change. I'm not sure though, this is quite a mess to debug :(