Bug 278090 – [GTK][WPE?][2.44.3] WebKitWebProcess crashes using Geary on on Arch Linux

• Component: Layout and Rendering
• Status: RESOLVED DUPLICATE

[Adrian Perez]

Created attachment 472152 [details]
Output from "thread apply all bt" in GDB

Reported in the WebKitGTK chat room:

  So, the 2.44.3 update landed on Arch a moments ago (Updated from
  2.44.2) and now, at least for me, Geary (Email Client) is failing
  to render web content. I know Geary is mostly unmaintained, but
  this kind of updated should break anything right?

  Geary logs shows the next message:

  *[wrn] 08:56:00.0695 geary:components-web-view.vala:618: Web process crashed: WEBKIT_WEB_PROCESS_CRASHED

I asked for a full backtrace (attached, yay coredumpctl+debuginfod!),
this is the relevant bit leading to the crash:

Thread 1 (Thread 0x7c8e7b68d300 (LWP 2)):
#0  memcpy () at /usr/include/bits/string_fortified.h:29
#1  uninitializedCopy () at WTF/Headers/wtf/Vector.h:190
#2  uninitializedCopy () at WTF/Headers/wtf/Vector.h:284
#3  operator= () at WTF/Headers/wtf/Vector.h:1044
#4  translateIntersectionPointsToSkipInkBoundaries () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/rendering/TextDecorationPainter.cpp:144
#5  operator()<WebCore::TextDecorationLine, WebCore::TextDecorationStyle, WebCore::Color const, WebCore::FloatRect> () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/rendering/TextDecorationPainter.cpp:219
#6  0x00007c8e8fd45b15 in paintBackgroundDecorations () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/rendering/TextDecorationPainter.cpp:287
#7  0x00007c8e8fd42747 in paintBackgroundDecorations () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/rendering/TextBoxPainter.cpp:683
#8  paintForegroundAndDecorations () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/rendering/TextBoxPainter.cpp:391
#9  0x00007c8e8fd3faf4 in paint () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/rendering/TextBoxPainter.cpp:134
#10 0x00007c8e8f578f35 in paintDisplayBox () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/layout/integration/inline/LayoutIntegrationInlineContentPainter.cpp:96
#11 0x00007c8e8f57f6cb in paint () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/layout/integration/inline/LayoutIntegrationInlineContentPainter.cpp:135
#12 paint () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/layout/integration/inline/LayoutIntegrationLineLayout.cpp:950
#13 0x00007c8e8fb5733d in paintContents () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/rendering/RenderBlock.cpp:1117
#14 paintObject () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/rendering/RenderBlock.cpp:1300
#15 0x00007c8e8fb4efa0 in paint () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/rendering/RenderBlock.cpp:1099
#16 0x00007c8e8fb68843 in paintChild () at /usr/src/debug/webkit2gtk-4.1/webkitgtk-2.44.3/Source/WebCore/rendering/RenderBlock.cpp:1176

[Adrian Perez]

The reported has also filed a bug in the Geary issue tracker:

  https://gitlab.gnome.org/GNOME/geary/-/issues/1632

[Adrian Perez]

The backtrace going through TextDecorationPainter makes me think this
might be one more manifestation of the issue behind bug #274780 but I
am not completely sure because in one of my computers I get the following
crash inside JSC instead using the same WebKit package from Arch Linux,
which seems unrelated -- what is true is that the Arch package has a crashy
WebKitWebProcess:

#0  __pthread_kill_implementation (threadid=<optimized out>, signo=signo@entry=6, no_tid=no_tid@entry=0) at pthread_kill.c:44
#1  0x000078ae292a5463 in __pthread_kill_internal (threadid=<optimized out>, signo=6) at pthread_kill.c:78
#2  0x000078ae2924c120 in __GI_raise (sig=sig@entry=6) at ../sysdeps/posix/raise.c:26
#3  0x000078ae292334c3 in __GI_abort () at abort.c:79
#4  0x000078ae27687dfc in WTFCrashWithInfo(int, char const*, char const*, int) () at WTF/Headers/wtf/Assertions.h:780
#5  0x000078ae28af928c in asAddress () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/JavaScriptCore/wasm/WasmBBQJIT.cpp:208
#6  0x000078ae28af928c in emitMoveMemory () at /usr/lib/libjavascriptcoregtk-6.0.so.1
#7  0x000078ae28a79fbd in returnValuesFromCall<8ul> () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/JavaScriptCore/wasm/WasmBBQJIT.cpp:3978
#8  0x000078ae28a79667 in addCall () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/JavaScriptCore/wasm/WasmBBQJIT.cpp:4017
#9  0x000078ae28a93424 in parseExpression () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/JavaScriptCore/wasm/WasmFunctionParser.h:2949
#10 0x000078ae28a7eb4f in parseBody () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/JavaScriptCore/wasm/WasmFunctionParser.h:501
#11 parse () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/JavaScriptCore/wasm/WasmFunctionParser.h:454
#12 0x000078ae28be296b in parseAndCompileBBQ () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/JavaScriptCore/wasm/WasmBBQJIT.cpp:4665
#13 compileFunction () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/JavaScriptCore/wasm/WasmBBQPlan.cpp:271
#14 0x000078ae28bdc9de in work () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/JavaScriptCore/wasm/WasmBBQPlan.cpp:145
#15 0x000078ae28d6b7d3 in work () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/JavaScriptCore/wasm/WasmWorklist.cpp:119
#16 0x000078ae28fb87e4 in operator() () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/WTF/wtf/AutomaticThread.cpp:229
#17 call () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/WTF/wtf/Function.h:53
#18 0x000078ae2906150e in operator() () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/WTF/wtf/Function.h:82
#19 entryPoint () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/WTF/wtf/Threading.cpp:258
#20 wtfThreadEntryPoint () at /usr/src/debug/webkitgtk-6.0/webkitgtk-2.44.3/Source/WTF/wtf/posix/ThreadingPOSIX.cpp:247
#21 0x000078ae292a339d in start_thread (arg=<optimized out>) at pthread_create.c:447
#22 0x000078ae293282a4 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100

[Adrian Perez]

The Arch package changes are only changing the version and the
expected checksum for the release tarball:

  https://gitlab.archlinux.org/archlinux/packaging/packages/webkit2gtk-4.1/-/commit/c222be91fbc595e376c106e6533ae18e2e0d1572

So we can discard a change in how they package WebKitGTK.

[Jan Alexander Steffens (heftig)]

I pasted a buildinfo diff into the comment at https://gitlab.archlinux.org/archlinux/packaging/packages/webkit2gtk-4.1/-/issues/1#note_203150, which is mostly a list of packages that were installed for the build.

The new package was built with Clang 18.1.8-2 and the old one with 17.0.6-2.

[Jim Mason]

(In reply to Adrian Perez from comment #2)

I am getting an almost identical backtrace from a new WebProcess crash in 2.44.3, that I experience when I visit a WASM/Unity demo: https://www.wasm.com.cn/demo/Tanks/

I've included my backtrace below.

My first suspicion is that it was caused by a cherry pick of the commit related to bug #271175 .  I'm rebuilding now with that backed out just to test my hypothesis.  The WASM/Unity tanks demo works fine for me fine in @main, so maybe there are some later commits or other dependencies.

The tanks demo crash backtrace in 2.44.3:

Thread 38 received signal SIGABRT, Aborted.
[Switching to Thread 32 (LWP 32)]
0x00007ffc0b9711aa in __lwp_sigqueue () from /lib/64/libc.so.1
(gdb) bt
#0  0x00007ffc0b9711aa in __lwp_sigqueue () at /lib/64/libc.so.1
#1  0x00007ffc0b9657c1 in thr_kill () at /lib/64/libc.so.1
#2  0x00007ffc0b913d09 in raise () at /lib/64/libc.so.1
#3  0x00007ffc0b8e8df2 in abort () at /lib/64/libc.so.1
#4  0x00007ffc038bb2fb in  () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#5  0x00007ffc0491b2c7 in  () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#6  0x00007ffc049b8294 in JSC::Wasm::BBQJITImpl::BBQJIT::emitMoveMemory(JSC::Wasm::TypeKind, JSC::Wasm::BBQJITImpl::BBQJIT::Location, JSC::Wasm::BBQJITImpl::BBQJIT::Location) () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#7  0x00007ffc04947d16 in void JSC::Wasm::BBQJITImpl::BBQJIT::returnValuesFromCall<8ul>(WTF::Vector<JSC::Wasm::BBQJITImpl::BBQJIT::Value, 8ul, WTF::CrashOnOverflow, 16ul, WTF::FastMalloc>&, JSC::Wasm::FunctionSignature const&, JSC::Wasm::CallInformation const&) () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#8  0x00007ffc0493b709 in JSC::Wasm::BBQJITImpl::BBQJIT::addCall(unsigned int, JSC::Wasm::TypeDefinition const&, WTF::Vector<JSC::Wasm::BBQJITImpl::BBQJIT::Value, 0ul, WTF::CrashOnOverflow, 16ul, WTF::FastMalloc>&, WTF::Vector<JSC::Wasm::BBQJITImpl::BBQJIT::Value, 8ul, WTF::CrashOnOverflow, 16ul, WTF::FastMalloc>&, JSC::CallLinkInfoBase::CallType) () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#9  0x00007ffc04963cd1 in JSC::Wasm::FunctionParser<JSC::Wasm::BBQJITImpl::BBQJIT>::parseExpression() () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#10 0x00007ffc049557cb in JSC::Wasm::FunctionParser<JSC::Wasm::BBQJITImpl::BBQJIT>::parseBody() () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#11 0x00007ffc04948820 in JSC::Wasm::FunctionParser<JSC::Wasm::BBQJITImpl::BBQJI--Type <RET> for more, q to quit, c to continue without paging--c
T>::parse() () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#12 0x00007ffc0493d907 in JSC::Wasm::parseAndCompileBBQ(JSC::Wasm::CompilationContext&, JSC::Wasm::BBQCallee&, JSC::Wasm::FunctionData const&, JSC::Wasm::TypeDefinition const&, WTF::Vector<JSC::Wasm::UnlinkedWasmToWasmCall, 0ul, WTF::CrashOnOverflow, 16ul, WTF::FastMalloc>&, JSC::Wasm::ModuleInformation const&, JSC::MemoryMode, unsigned int, std::__1::optional<bool>, unsigned int, JSC::Wasm::TierUpCount*) () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#13 0x00007ffc049f6a9b in JSC::Wasm::BBQPlan::compileFunction(unsigned int, JSC::Wasm::BBQCallee&, JSC::Wasm::CompilationContext&, WTF::Vector<JSC::Wasm::UnlinkedWasmToWasmCall, 0ul, WTF::CrashOnOverflow, 16ul, WTF::FastMalloc>&, JSC::Wasm::TierUpCount*) () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#14 0x00007ffc049f5c21 in JSC::Wasm::BBQPlan::work(JSC::Wasm::Plan::CompilationEffort) () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#15 0x00007ffc04bd3da6 in JSC::Wasm::Worklist::Thread::work() () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#16 0x00007ffc04ccd1b4 in WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call() () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#17 0x00007ffc04cf7066 in WTF::Thread::entryPoint(WTF::Thread::NewThreadContext*) () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#18 0x00007ffc04d59769 in WTF::wtfThreadEntryPoint(void*) () at /usr/lib/64/libjavascriptcoregtk-4.1.so.0
#19 0x00007ffc0b967ba9 in _thrp_setup () at /lib/64/libc.so.1
#20 0x00007ffc0b967e50 in _lwp_start () at /lib/64/libc.so.1
#21 0x0000000000000000 in  ()

[Michael Catanzaro]

(In reply to Adrian Perez from comment #2)
> The backtrace going through TextDecorationPainter makes me think this
> might be one more manifestation of the issue behind bug #274780 

It's got to be. You can even see from the pkgbuild that arch is using Clang plus LTO. I'm going to mark this as a duplicate.

> but I am not completely sure because in one of my computers I get the following
> crash inside JSC instead using the same WebKit package from Arch Linux,
> which seems unrelated -- what is true is that the Arch package has a crashy
> WebKitWebProcess:

Surely this is an unrelated second crash. Could you or Jim please report a JSC bug for this where we can continue? Jim's theory sounds pretty reasonable to me (thanks for checking it, Jim!).

*** This bug has been marked as a duplicate of bug 274780 ***

[Adrian Perez]

(In reply to Jan Alexander Steffens (heftig) from comment #4)
> I pasted a buildinfo diff into the comment at
> https://gitlab.archlinux.org/archlinux/packaging/packages/webkit2gtk-4.1/-/
> issues/1#note_203150, which is mostly a list of packages that were installed
> for the build.
> 
> The new package was built with Clang 18.1.8-2 and the old one with 17.0.6-2.

For the record, this observation matches comment in bug #274780 about the
issue starting (or re-appearing) with Clang 18.1.x

[Michael Catanzaro]

> For the record, this observation matches comment in bug #274780 about the
> issue starting (or re-appearing) with Clang 18.1.x

In that case, it's probably possibly to bisect Clang to find the commit that broke. It won't be fun, but would likely avoid miscompilations in other packages too, so the value would be high.