Bug 211572 – [GStreamer][STABLE] Crash in WebCore::MediaPlayer::createResourceLoader

• Component: Media
• Status: RESOLVED WORKSFORME

[Michael Catanzaro]

Ephy Tech Preview has started crashing when visiting my MP4 test page https://www.reddit.com/r/StLouis/comments/fvnqpb/gotta_love_our_zoo/. The page crashes immediately on load 100% of the time. It was working *very* recently, so probably something has changed somewhere in the runtime, but the crash is WebKit's problem so:

(gdb) bt full
#0  0x00007f01150bd631 in WebCore::MediaPlayer::createResourceLoader()
    (this=<optimized out>)
    at ../Source/WebCore/platform/graphics/MediaPlayer.h:636
#1  0x00007f0113f4a72c in <lambda()>::operator()(void) const
    (__closure=0x7f005406e008)
    at ../Source/WebCore/platform/graphics/gstreamer/WebKitWebSourceGStreamer.cpp:694
        priv = 0x7efeec007c40
        loadOptions = <optimized out>
        notifyAsyncCompletion = true
        src = 0x7efeec007de0 [WebKitWebSrc]
        request = 
                {<WebCore::ResourceRequestBase> = {m_url = {m_string = {static MaxLength = 2147483647, m_impl = {static isRefPtr = <error reading variable: Missing ELF symbol "WTF::RefPtr<WTF::StringImpl, WTF::DumbPtrTraits<WTF::StringImpl> >::isRefPtr".>, m_ptr = 0x7f007426c080}}, m_isValid = 1, m_protocolIsInHTTPFamily = 1, m_cannotBeABaseURL = 0, m_portLength = 0, static maxPortLength = 7, static maxSchemeLength = 67108863, m_schemeEnd = 5, m_userStart = 8, m_userEnd = 8, m_passwordEnd = 8, m_hostEnd = 17, m_pathAfterLastSlash = 32, m_pathEnd = 42, m_queryEnd = 42}, m_timeoutInterval = 0, m_firstPartyForCookies = {m_string = {static MaxLength = 2147483647, m_impl = {static isRefPtr = <error reading variable: Missing ELF symbol "WTF::RefPtr<WTF::StringImpl, WTF::DumbPtrTraits<WTF::StringImpl> >::isRefPtr".>, m_ptr = 0x7f007426c080}}, m_isValid = 1, m_protocolIsInHTTPFamily = 1, m_cannotBeABaseURL = 0, m_portLength = 0, static maxPortLength = 7, static maxSchemeLength = 67108863, m_schemeEnd = 5, m_userStart = 8, m_userEnd = 8, m_passwordEnd = 8, m_hostEnd = 17, m_pathAfterLastSlash = 32, m_pathEnd = 42, m_queryEnd = 42}, m_httpMethod = {static MaxLength = 2147483647, m_impl = {static isRefPtr = <error reading variable: Missing ELF symbol "WTF::RefPtr<WTF::StringImpl, WTF::DumbPtrTraits<WTF::StringImpl> >::isRefPtr".>, m_ptr = 0x7eff0647b000}}, m_initiatorIdentifier = {static MaxLength = 2147483647, m_impl = {static isRefPtr = <error reading variable: Missing ELF symbol "WTF::RefPtr<WTF::StringImpl, WTF::DumbPtrTraits<WTF::StringImpl> >::isRefPtr".>, m_ptr = 0x0}}, m_cachePartition = {static MaxLength = 2147483647, m_impl = {static isRefPtr = <error reading variable: Missing ELF symbol "WTF::RefPtr<WTF::StringImpl, WTF::DumbPtrTraits<WTF::StringImpl> >::isRefPtr".>, m_ptr = 0x7f0111c3fce0 <WTF::StringImpl::s_emptyAtomString>}}, m_httpHeaderFields = {m_commonHeaders = {<WTF::VectorBuffer<WebCore::HTTPHeaderMap::CommonHeader, 0, WTF::FastMalloc>> = {<WTF::VectorBufferBase<WebCore::HTTPHeaderMap::CommonHeader, WTF::FastMalloc>> = {m_buffer = 0x7f0054029000, m_capacity = 6, m_size = 2}, <No data fields>}, <No data fields>}, m_uncommonHeaders = {<WTF::VectorBuffer<WebCore::HTTPHeaderMap::UncommonHeader, 0, WTF::FastMalloc>> = {<WTF::VectorBufferBase<WebCore::HTTPHeaderMap::UncommonHeader, WTF::FastMalloc>> = {m_buffer = 0x0, m_capacity = 0, m_size = 0}, <No data fields>}, <No data fields>}}, m_responseContentDispositionEncodingFallbackArray = {<WTF::VectorBuffer<WTF::String, 0, WTF::FastMalloc>> = {<WTF::VectorBufferBase<WTF::String, WTF::FastMalloc>> = {m_buffer = 0x0, m_capacity = 0, m_size = 0}, <No data fields>}, <No data fields>}, m_httpBody = {static isRefPtr = <error reading variable: Missing ELF symbol "WTF::RefPtr<WebCore::FormData, WTF::DumbPtrTraits<WebCore::FormData> >::isRefPtr".>, m_ptr = 0x0}, m_cachePolicy = WebCore::ResourceRequestCachePolicy::UseProtocolCachePolicy, m_sameSiteDisposition = WebCore::ResourceRequestBase::SameSiteDisposition::Unspecified, m_priority = WebCore::ResourceLoadPriority::Low, m_requester = WebCore::ResourceRequestBase::Requester::Unspecified, m_inspectorInitiatorNodeIdentifier = {<WTF::constexpr_Optional_base<int>> = {init_ = false, storage_ = {dummy_ = 0 '\000', value_ = 0}}, <No data fields>}, m_allowCookies = true, m_resourceRequestUpdated = true, m_platformRequestUpdated = false, m_resourceRequestBodyUpdated = true, m_platformRequestBodyUpdated = false, m_hiddenFromInspector = false, m_isTopSite = false, static s_defaultTimeoutInterval = 0}, m_acceptEncoding = false, m_soupFlags = (unknown: 0), m_initiatingPageID = {<WTF::constexpr_Optional_base<unsigned long>> = {init_ = false, storage_ = {dummy_ = 0 '\000', value_ = 0}}, <No data fields>}}
        protector = {m_ptr = 0x7efeec007de0 [WebKitWebSrc]}
#2  0x00007f01118b095c in WTF::Function<void ()>::operator()() const (this=<synthetic pointer>) at ../Source/WTF/wtf/Function.h:81
        function = {m_callableWrapper = std::unique_ptr<WTF::Detail::CallableWrapperBase<void>> = {get() = 0x7f0054047050}}
        functionsHandled = 6
        functionsToHandle = 10
#3  0x00007f01118b095c in WTF::RunLoop::performWork() (this=0x7f0106dfa000) at ../Source/WTF/wtf/RunLoop.cpp:124
        function = {m_callableWrapper = std::unique_ptr<WTF::Detail::CallableWrapperBase<void>> = {get() = 0x7f0054047050}}
        functionsHandled = 6
        functionsToHandle = 10
#4  0x00007f01118ff41d in WTF::RunLoop::<lambda(gpointer)>::operator() (__closure=0x0, userData=<optimized out>) at ../Source/WTF/wtf/glib/RunLoopGLib.cpp:68
#5  0x00007f01118ff41d in WTF::RunLoop::<lambda(gpointer)>::_FUN(gpointer) () at ../Source/WTF/wtf/glib/RunLoopGLib.cpp:70
#6  0x00007f0112037bce in g_main_dispatch (context=0x55964ecc7e20) at ../glib/gmain.c:3309
        dispatch = 0x7f01118ff430 <WTF::<lambda(GSource*, GSourceFunc, gpointer)>::_FUN(GSource *, GSourceFunc, gpointer)>
        prev_source = 0x0
        was_in_call = 0
        user_data = 0x7f0106dfa000
        callback = 0x7f01118ff410 <WTF::RunLoop::<lambda(gpointer)>::_FUN(gpointer)>
        cb_funcs = 0x7f011210e280 <g_source_callback_funcs>
        cb_data = 0x55964ee341c0
        need_destroy = <optimized out>
        source = 0x55964ee28cb0
        current = 0x55964ecd1430
        i = 0
        __func__ = "g_main_dispatch"
#7  0x00007f0112037bce in g_main_context_dispatch (context=context@entry=0x55964ecc7e20) at ../glib/gmain.c:3974
#8  0x00007f0112037f80 in g_main_context_iterate (context=0x55964ecc7e20, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4047
        max_priority = 100
        timeout = 0
        some_ready = 1
        nfds = <optimized out>
        allocated_nfds = <optimized out>
        fds = 0x55964f3d3580
#9  0x00007f0112038273 in g_main_loop_run (loop=0x55964ed90860) at ../glib/gmain.c:4241
        __func__ = "g_main_loop_run"
#10 0x00007f01118ffeb0 in WTF::RunLoop::run() () at ../Source/WTF/wtf/glib/RunLoopGLib.cpp:96
        runLoop = @0x7f0106dfa000: {<WTF::FunctionDispatcher> = {<WTF::ThreadSafeRefCounted<WTF::FunctionDispatcher, (WTF::DestructionThread)0>> = {<WTF::ThreadSafeRefCountedBase> = {m_refCount = {<std::__atomic_base<unsigned int>> = {static _S_alignment = 4, _M_i = 38}, static is_always_lock_free = true}}, <No data fields>}, _vptr.FunctionDispatcher = 0x7f0111c0c240 <vtable for WTF::RunLoop+16>}, m_functionQueueLock = {static isHeldBit = 1 '\001', static hasParkedBit = 2 '\002', m_byte = {value = {<std::__atomic_base<unsigned char>> = {static _S_alignment = 1, _M_i = 0 '\000'}, static is_always_lock_free = true}}}, m_functionQueue = {m_start = 86, m_end = 90, m_buffer = {<WTF::VectorBufferBase<WTF::Function<void()>, WTF::FastMalloc>> = {m_buffer = 0x7f009414aa80, m_capacity = 108, m_size = 0}, <No data fields>}}, m_mainContext = {m_ptr = 0x55964ecc7e20}, m_mainLoops = {<WTF::VectorBuffer<WTF::GRefPtr<_GMainLoop>, 0, WTF::FastMalloc>> = {<WTF::VectorBufferBase<WTF::GRefPtr<_GMainLoop>, WTF::FastMalloc>> = {m_buffer = 0x7f0106df9000, m_capacity = 16, m_size = 1}, <No data fields>}, <No data fields>}, m_source = {m_ptr = 0x55964ee28cb0}}
        mainContext = 0x55964ecc7e20
        innermostLoop = 0x55964ed90860
        nestedMainLoop = <optimized out>
#11 0x00007f0113f4145f in WebKit::AuxiliaryProcessMain<WebKit::WebProcess, WebKit::WebProcessMainGtk>(int, char**) (argc=3, argv=<optimized out>) at ../Source/WebKit/Shared/AuxiliaryProcessMain.h:49
        auxiliaryMain = {<WebKit::AuxiliaryProcessMainBase> = {_vptr.AuxiliaryProcessMainBase = 0x7f0116384110 <vtable for WebKit::WebProcessMainGtk+16>, m_parameters = {uiProcessName = {static MaxLength = 2147483647, m_impl = {static isRefPtr = <error reading variable: Missing ELF symbol "WTF::RefPtr<WTF::StringImpl, WTF::DumbPtrTraits<WTF::StringImpl> >::isRefPtr".>, m_ptr = 0x0}}, clientIdentifier = {static MaxLength = 2147483647, m_impl = {static isRefPtr = <error reading variable: Missing ELF symbol "WTF::RefPtr<WTF::StringImpl, WTF::DumbPtrTraits<WTF::StringImpl> >::isRefPtr".>, m_ptr = 0x0}}, processIdentifier = {<WTF::constexpr_Optional_base<WTF::ObjectIdentifier<WebCore::ProcessIdentifierType> >> = {init_ = true, storage_ = {dummy_ = 75 'K', value_ = {<WTF::ObjectIdentifierBase> = {<No data fields>}, m_identifier = 75, static m_generationProtected = false}}}, <No data fields>}, connectionIdentifier = 94, extraInitializationData = {m_impl = {static smallMaxLoadNumerator = <optimized out>, static smallMaxLoadDenominator = <optimized out>, static largeMaxLoadNumerator = <optimized out>, static largeMaxLoadDenominator = <optimized out>, static maxSmallTableCapacity = <optimized out>, static minLoad = <optimized out>, static tableSizeOffset = <optimized out>, static tableSizeMaskOffset = <optimized out>, static keyCountOffset = <optimized out>, static deletedCountOffset = <optimized out>, static metadataSize = <optimized out>, m_table = 0x0}}, processType = WebKit::AuxiliaryProcess::ProcessType::WebContent}}, <No data fields>}
#12 0x00007f0112f87043 in __libc_start_main (main=0x55964d59c820 <main(int, char**)>, argc=3, argv=0x7ffd4fd9cd88, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffd4fd9cd78) at ../csu/libc-start.c:308
        result = <optimized out>
        unwind_buf = {cancel_jmp_buf = {{jmp_buf = {94104031185280, 1728232511868871156, 94104031185024, 140725943127424, 0, 0, -1728971161859415564, -1587009104334315020}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x3, 0x7ffd4fd9cd88}, data = {prev = 0x0, cleanup = 0x0, canceltype = 3}}}
        not_first_call = <optimized out>
#13 0x000055964d59c8ae in _start () at ../sysdeps/x86_64/start.S:120

[Alicia Boya García]

The presence of `notifyAsyncCompletion` calls my attention. That argument (along with async start) has been removed in the WebKitWebSrc rework, which is most likely the element behind this bug and has been largely modified since.

[Michael Catanzaro]

Hmm, is it safe to backport that work to 2.28? (Probably not?)

BTW, maybe time for a 2.29 release? It's now a really long time for Tech Preview to not be following trunk.

[Philippe Normand]

(In reply to Michael Catanzaro from comment #2)
> Hmm, is it safe to backport that work to 2.28? (Probably not?)
> 

As it provides stability fixes and no new features, I would backport it, if it's not too much trouble.

[Alicia Boya García]

(In reply to Michael Catanzaro from comment #2)
> Hmm, is it safe to backport that work to 2.28? (Probably not?)

It's still a bit risky since it's a recent land with substantial changes, but it fixes issues.

[Michael Catanzaro]

Probably safest to release 2.29.1 first, so we can start testing it. Then, once we have had a couple weeks to do basic testing, then maybe consider backporting.

But if you don't want to investigate a stable branch crash, then we need to backport no matter what, because this crash is 100%.

[Philippe Normand]

This is the 3rd bug report about createResourceLoader()... Are we sure there's no duplicate? See bug 204035 and bug 204161.

[Alicia Boya García]

*** Bug 204161 has been marked as a duplicate of this bug. ***

[Alicia Boya García]

*** Bug 204035 has been marked as a duplicate of this bug. ***

[Michael Catanzaro]

I guess what

[Michael Catanzaro]

Sorry, it seems pressing tab by mistake shifts focus to Save Changes, and then pressing Space on the Save Changes button submits the comment. <_<

I'm not sure why I failed to find those bugs before reporting this. I remember being surprised when I searched Bugzilla and couldn't find any dups....

Anyway, what's different now is that, when I reported the first two bugs, the crash was random and non-reproducible. Now it's no longer possible to load the affected webpage without hitting the crash.

Also note Phil had an interesting comment at: https://bugs.webkit.org/show_bug.cgi?id=204161#c1

[Alicia Boya García]

(In reply to Michael Catanzaro from comment #10)
> Also note Phil had an interesting comment at:
> https://bugs.webkit.org/show_bug.cgi?id=204161#c1

This is Phil's comment:
> This might be a regression introduced by r247215 ...
> Capturing the src pointer in the lambda of webKitWebSrcMakeRequest() doesn't look right; protector should be used instead.

After the WebKitWebSrc rework, webKitWebSrcStart() (the function modified in that commit) has been almost emptied, all lambdas use protectors, and care has taken to cancel pending callbacks when the pipeline is flushed or torn down. These bugs are no longer relevant with the WebKitWebSrc rework.

[Michael Catanzaro]

As of today, the page is, incredibly, no longer crashing in Tech Preview. Who knows what has changed.

(In reply to Alicia Boya García from comment #11)
> After the WebKitWebSrc rework, webKitWebSrcStart() (the function modified in
> that commit) has been almost emptied, all lambdas use protectors, and care
> has taken to cancel pending callbacks when the pipeline is flushed or torn
> down. These bugs are no longer relevant with the WebKitWebSrc rework.

Alicia, maybe you should make a decision as to whether your commits should be backported (and if so, which ones), or whether this requires investigation on the current stable branch?

[Alicia Boya García]

(In reply to Michael Catanzaro from comment #12)
> As of today, the page is, incredibly, no longer crashing in Tech Preview.
> Who knows what has changed.
> 
> (In reply to Alicia Boya García from comment #11)
> > After the WebKitWebSrc rework, webKitWebSrcStart() (the function modified in
> > that commit) has been almost emptied, all lambdas use protectors, and care
> > has taken to cancel pending callbacks when the pipeline is flushed or torn
> > down. These bugs are no longer relevant with the WebKitWebSrc rework.
> 
> Alicia, maybe you should make a decision as to whether your commits should
> be backported (and if so, which ones), or whether this requires
> investigation on the current stable branch?

My decision is to backport. In fact, it's already in the backport list: https://trac.webkit.org/wiki/WebKitGTK/2.28.x

[Michael Catanzaro]

(In reply to Michael Catanzaro from comment #5)
> Probably safest to release 2.29.1 first, so we can start testing it. Then,
> once we have had a couple weeks to do basic testing, then maybe consider
> backporting.

Thanks for this release, Carlos. We can begin testing Alicia's changes soon... let's give in a couple weeks to see if users report regressions before backporting.

[Michael Catanzaro]

So far, I've noticed some video playback hangs on YouTube that seem to be newly-introduced, which may or may not be related. But seek is definitely working much better than it was before (and the seeking bugs were more severe IMO).

[Alicia Boya García]

(In reply to Michael Catanzaro from comment #15)
> So far, I've noticed some video playback hangs on YouTube that seem to be
> newly-introduced, which may or may not be related. But seek is definitely
> working much better than it was before (and the seeking bugs were more
> severe IMO).

YouTube uses MSE so therefore it doesn't use WebKitWebSrc or createResourceLoader(). Any perceived improvement or regression is unrelated to the WebKitWebSrc threading rework.

[Xabier Rodríguez Calvar]

Is this bug reproduciable? I could not.

[Michael Catanzaro]

It was 100% reproducible at the time, but it looks like it stopped crashing about a year ago, see comment #5. Whatever.