``` $ uname -ar Linux orion.dev 5.6.0-0.rc7.git1.2.local.fc33.ppc64le #1 SMP Sun Mar 29 10:28:55 AEDT 2020 ppc64le ppc64le ppc64le GNU/Linux $ rpm -q webkit2gtk3 webkit2gtk3-2.28.0-8.fc32.ppc64le # GNOME web application $ epiphany --version Web 3.36.0 ``` Opening up any website with GNOME Web (aka epiphany) would result in crashing the forked WebKitWebProcess process. Here is stack trace: ``` [tle@orion ~]$ coredumpctl debug 10750 PID: 10750 (WebKitWebProces) UID: 1000 (tle) GID: 1000 (tle) Signal: 6 (ABRT) Timestamp: Sat 2020-04-18 22:28:08 AEST (13min ago) Command Line: /usr/libexec/webkit2gtk-4.0/WebKitWebProcess 11 30 Executable: /usr/libexec/webkit2gtk-4.0/WebKitWebProcess Control Group: /user.slice/user-1000.slice/user@1000.service/apps.slice/apps-org.gnome.Terminal.slice/vte-spawn-14470ceb-8474-455e-9fc4-74a68b385db4.scope Unit: user@1000.service User Unit: vte-spawn-14470ceb-8474-455e-9fc4-74a68b385db4.scope Slice: user-1000.slice Owner UID: 1000 (tle) Boot ID: 29003e772cb44e319c84ccbe6ad7a138 Machine ID: 5632f07729a648c49d05933910ac9490 Hostname: orion.dev Storage: /var/lib/systemd/coredump/core.WebKitWebProces.1000.29003e772cb44e319c84ccbe6ad7a138.10750.1587212888000000000000.lz4 Message: Process 10750 (WebKitWebProces) of user 1000 dumped core. Stack trace of thread 2: #0 0x00007ffff3f79238 __libc_signal_restore_set (libc.so.6 + 0x49238) #1 0x00007ffff3f57c68 __GI_abort (libc.so.6 + 0x27c68) #2 0x00007ffff1fd5eb4 _Z15CRASH_WITH_INFOz (libjavascriptcoregtk-4.0.so.18 + 0x915eb4) #3 0x00007ffff1ff8214 _ZN3JSC8JSObject12ensureLengthERNS_2VMEj (libjavascriptcoregtk-4.0.so.18 + 0x938214) #4 0x00007ffff1fd84bc _ZN3JSC8JSObject38putDirectIndexSlowOrBeyondVectorLengthEPNS_14JSGlobalObjectEjNS_7JSValueEjNS_18PutDirectIndexModeE (libjavascriptcoregtk-4.0.so.18 + 0x9184bc) #5 0x00007ffff1ccfbe4 _ZN3JSC8JSObject14putDirectIndexEPNS_14JSGlobalObjectEjNS_7JSValueEjNS_18PutDirectIndexModeE (libjavascriptcoregtk-4.0.so.18 + 0x60fbe4) #6 0x00007ffff1ca3d34 _ZN3JSC5LLInt5CLoop7executeENS_8OpcodeIDEPvPNS_2VMEPNS_14ProtoCallFrameEb (libjavascriptcoregtk-4.0.so.18 + 0x5e3d34) #7 0x00007ffff1cca380 vmEntryToJavaScript (libjavascriptcoregtk-4.0.so.18 + 0x60a380) #8 0x00007ffff1c80858 _ZN3JSC7JITCode7executeEPNS_2VMEPNS_14ProtoCallFrameE (libjavascriptcoregtk-4.0.so.18 + 0x5c0858) #9 0x00007ffff1e59128 _ZN3JSC4callEPNS_14JSGlobalObjectENS_7JSValueENS_8CallTypeERKNS_8CallDataES2_RKNS_7ArgListE (libjavascriptcoregtk-4.0.so.18 + 0x799128) #10 0x00007ffff1e594e0 _ZN3JSC12profiledCallEPNS_14JSGlobalObjectENS_15ProfilingReasonENS_7JSValueENS_8CallTypeERKNS_8CallDataES3_RKNS_7ArgListE (libjavascriptcoregtk-4.0.so.18 + 0x7994e0) #11 0x00007ffff1fc34b8 _ZN3JSC11JSMicrotask3runEPNS_14JSGlobalObjectE (libjavascriptcoregtk-4.0.so.18 + 0x9034b8) #12 0x00007ffff5c370f8 _ZN7WebCore11JSExecState7runTaskEPN3JSC14JSGlobalObjectERNS1_9MicrotaskE (libwebkit2gtk-4.0.so.37 + 0x19c70f8) #13 0x00007ffff5f72f7c _ZNK3WTF8FunctionIFvvEEclEv (libwebkit2gtk-4.0.so.37 + 0x1d02f7c) #14 0x00007ffff5f96a30 _ZN7WebCore14MicrotaskQueue26performMicrotaskCheckpointEv (libwebkit2gtk-4.0.so.37 + 0x1d26a30) #15 0x00007ffff5f71910 _ZN7WebCore9EventLoop26performMicrotaskCheckpointEv (libwebkit2gtk-4.0.so.37 + 0x1d01910) #16 0x00007ffff5f72060 _ZN7WebCore18EventLoopTaskGroup26performMicrotaskCheckpointEv (libwebkit2gtk-4.0.so.37 + 0x1d02060) #17 0x00007ffff5c4bd24 _ZN7WebCore11JSExecState21didLeaveScriptContextEPN3JSC14JSGlobalObjectE (libwebkit2gtk-4.0.so.37 + 0x19dbd24) #18 0x00007ffff5c4db28 _ZN7WebCore11JSExecStateD4Ev (libwebkit2gtk-4.0.so.37 + 0x19ddb28) #19 0x00007ffff5f779cc _ZN7WebCore11EventTarget25innerInvokeEventListenersERNS_5EventEN3WTF6VectorINS3_6RefPtrINS_23RegisteredEventListenerENS3_13DumbPtrTraitsIS6_EEEELm1ENS3_15CrashOnOverflowELm16ENS3_10FastMallocEEENS0_16EventInvokePhaseE (libwebkit2gtk-4.0.so.37 + 0x1d079cc) #20 0x00007ffff5f7a1c8 _ZN7WebCore11EventTarget18fireEventListenersERNS_5EventENS0_16EventInvokePhaseE (libwebkit2gtk-4.0.so.37 + 0x1d0a1c8) #21 0x00007ffff5f7a794 _ZN7WebCore11EventTarget13dispatchEventERNS_5EventE (libwebkit2gtk-4.0.so.37 + 0x1d0a794) #22 0x00007ffff6f40080 _ZN7WebCore14XMLHttpRequest13dispatchEventERNS_5EventE (libwebkit2gtk-4.0.so.37 + 0x2cd0080) #23 0x00007ffff6f46070 _ZN7WebCore35XMLHttpRequestProgressEventThrottle25dispatchEventWhenPossibleERNS_5EventE (libwebkit2gtk-4.0.so.37 + 0x2cd6070) #24 0x00007ffff6f46408 _ZN7WebCore35XMLHttpRequestProgressEventThrottle21dispatchProgressEventERKN3WTF10AtomStringE (libwebkit2gtk-4.0.so.37 + 0x2cd6408) #25 0x00007ffff6f486d4 _ZN7WebCore14XMLHttpRequest28callReadyStateChangeListenerEv (libwebkit2gtk-4.0.so.37 + 0x2cd86d4) #26 0x00007ffff6f487f8 _ZN7WebCore14XMLHttpRequest11changeStateENS0_5StateE (libwebkit2gtk-4.0.so.37 + 0x2cd87f8) #27 0x00007ffff6f49890 _ZN7WebCore14XMLHttpRequest16didFinishLoadingEm (libwebkit2gtk-4.0.so.37 + 0x2cd9890) #28 0x00007ffff644b744 _ZN7WebCore24DocumentThreadableLoader16didFinishLoadingEm (libwebkit2gtk-4.0.so.37 + 0x21db744) #29 0x00007ffff644b9ac _ZThn16_N7WebCore24DocumentThreadableLoader14notifyFinishedERNS_14CachedResourceE (libwebkit2gtk-4.0.so.37 + 0x21db9ac) #30 0x00007ffff651728c _ZN7WebCore14CachedResource11checkNotifyEv (libwebkit2gtk-4.0.so.37 + 0x22a728c) #31 0x00007ffff6515f24 _ZN7WebCore14CachedResource13finishLoadingEPNS_12SharedBufferE (libwebkit2gtk-4.0.so.37 + 0x22a5f24) #32 0x00007ffff6520508 _ZN7WebCore17CachedRawResource13finishLoadingEPNS_12SharedBufferE (libwebkit2gtk-4.0.so.37 + 0x22b0508) #33 0x00007ffff64c9400 _ZN7WebCore17SubresourceLoader16didFinishLoadingERKNS_18NetworkLoadMetricsE (libwebkit2gtk-4.0.so.37 + 0x2259400) #34 0x00007ffff50d6490 _ZN6WebKit17WebResourceLoader21didFinishResourceLoadERKN7WebCore18NetworkLoadMetricsE (libwebkit2gtk-4.0.so.37 + 0xe66490) #35 0x00007ffff4a724c8 _ZN3IPC22callMemberFunctionImplIN6WebKit17WebResourceLoaderEMS2_FvRKN7WebCore18NetworkLoadMetricsEESt5tupleIJS4_EEJLm0EEEEvPT_T0_OT1_St16integer_sequenceImJXspT2_EEE (libwebkit2gtk-4.0.so.37 + 0x8024c8) #36 0x00007ffff4a71058 _ZN6WebKit17WebResourceLoader34didReceiveWebResourceLoaderMessageERN3IPC10ConnectionERNS1_7DecoderE (libwebkit2gtk-4.0.so.37 + 0x801058) #37 0x00007ffff50c1db8 _ZN6WebKit24NetworkProcessConnection17didReceiveMessageERN3IPC10ConnectionERNS1_7DecoderE (libwebkit2gtk-4.0.so.37 + 0xe51db8) #38 0x00007ffff4bebd48 _ZN3IPC10Connection15dispatchMessageERNS_7DecoderE (libwebkit2gtk-4.0.so.37 + 0x97bd48) #39 0x00007ffff4bedaa0 _ZN3IPC10Connection15dispatchMessageESt10unique_ptrINS_7DecoderESt14default_deleteIS2_EE (libwebkit2gtk-4.0.so.37 + 0x97daa0) #40 0x00007ffff4bee2dc _ZN3IPC10Connection26dispatchOneIncomingMessageEv (libwebkit2gtk-4.0.so.37 + 0x97e2dc) #41 0x00007ffff4bee854 operator() (libwebkit2gtk-4.0.so.37 + 0x97e854) #42 0x00007ffff222736c _ZNK3WTF8FunctionIFvvEEclEv (libjavascriptcoregtk-4.0.so.18 + 0xb6736c) #43 0x00007ffff228f508 operator() (libjavascriptcoregtk-4.0.so.18 + 0xbcf508) #44 0x00007ffff228f590 operator() (libjavascriptcoregtk-4.0.so.18 + 0xbcf590) #45 0x00007ffff2b4e9d8 g_main_context_dispatch (libglib-2.0.so.0 + 0x6e9d8) #46 0x00007ffff2b4eeb8 g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0x6eeb8) #47 0x00007ffff2b4f3ec g_main_loop_run (libglib-2.0.so.0 + 0x6f3ec) #48 0x00007ffff22907d4 _ZN3WTF7RunLoop3runEv (libjavascriptcoregtk-4.0.so.18 + 0xbd07d4) #49 0x00007ffff51d8a14 _ZN6WebKit20AuxiliaryProcessMainINS_10WebProcessENS_17WebProcessMainGtkEEEiiPPc (libwebkit2gtk-4.0.so.37 + 0xf68a14) #50 0x00007ffff51d7f78 _ZN6WebKit14WebProcessMainEiPPc (libwebkit2gtk-4.0.so.37 + 0xf67f78) #51 0x00000001000007d0 main (WebKitWebProcess + 0x7d0) #52 0x00007ffff3f580cc generic_start_main (libc.so.6 + 0x280cc) #53 0x00007ffff3f58290 __libc_start_main (libc.so.6 + 0x28290) Stack trace of thread 8: #0 0x00007ffff405c55c __GI___poll (libc.so.6 + 0x12c55c) #1 0x00007ffff2b66528 g_poll (libglib-2.0.so.0 + 0x86528) #2 0x00007ffff2b4ee34 g_main_context_iterate.constprop.0 (libglib-2.0.so.0 + 0x6ee34) #3 0x00007ffff2b4f3ec g_main_loop_run (libglib-2.0.so.0 + 0x6f3ec) #4 0x00007ffff22907d4 _ZN3WTF7RunLoop3runEv (libjavascriptcoregtk-4.0.so.18 + 0xbd07d4) #5 0x00007ffff228b140 operator() (libjavascriptcoregtk-4.0.so.18 + 0xbcb140) #6 0x00007ffff2229ba0 _ZNK3WTF8FunctionIFvvEEclEv (libjavascriptcoregtk-4.0.so.18 + 0xb69ba0) #7 0x00007ffff2292b28 wtfThreadEntryPoint (libjavascriptcoregtk-4.0.so.18 + 0xbd2b28) #8 0x00007ffff0fe9618 start_thread (libpthread.so.0 + 0x9618) #9 0x00007ffff406cf64 __clone (libc.so.6 + 0x13cf64) GNU gdb (GDB) Fedora 9.1-3.fc32 Copyright (C) 2020 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "ppc64le-redhat-linux-gnu". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://www.gnu.org/software/gdb/bugs/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/libexec/webkit2gtk-4.0/WebKitWebProcess... Reading symbols from /usr/lib/debug/usr/libexec/webkit2gtk-4.0/WebKitWebProcess-2.28.0-8.fc32.ppc64le.debug... [New LWP 2] [New LWP 8] [New LWP 3] [New LWP 9] [New LWP 4] [New LWP 7] [New LWP 11] [New LWP 5] [New LWP 6] [New LWP 17] [New LWP 16] [New LWP 15] [New LWP 14] [New LWP 13] [New LWP 18] [New LWP 12] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Missing separate debuginfo for /usr/lib64/epiphany/web-process-extensions/libephywebprocessextension.so Try: dnf --enablerepo='*debug*' install /usr/lib/debug/.build-id/b6/ddcda13a0c5cba27b8f99ea69a12d150125b4d.debug Missing separate debuginfo for /usr/lib64/epiphany/libephymisc.so Try: dnf --enablerepo='*debug*' install /usr/lib/debug/.build-id/46/8d25e73a32897b806246db860ae10e19b73dbd.debug Core was generated by `/usr/libexec/webkit2gtk-4.0/WebKitWebProcess 11 30 '. Program terminated with signal SIGABRT, Aborted. #0 0x00007ffff3f79238 in __libc_signal_restore_set (set=0x7fffffffc158) at ../sysdeps/unix/sysv/linux/internal-signals.h:86 86 INTERNAL_SYSCALL_CALL (rt_sigprocmask, err, SIG_SETMASK, set, NULL, Missing separate debuginfos, use: dnf debuginfo-install glib2-2.64.2-1.fc32.ppc64le --Type <RET> for more, q to quit, c to continue without paging-- [Current thread is 1 (Thread 0x7fffec5404a0 (LWP 2))] (gdb) bt #0 0x00007ffff3f79238 in __libc_signal_restore_set (set=0x7fffffffc158) at ../sysdeps/unix/sysv/linux/internal-signals.h:86 #1 __GI_raise (sig=<optimized out>) at ../sysdeps/unix/sysv/linux/raise.c:48 #2 0x00007ffff3f57c68 in __GI_abort () at abort.c:79 #3 0x00007ffff1fd5eb4 in CRASH_WITH_INFO(...) () at DerivedSources/ForwardingHeaders/wtf/Assertions.h:660 #4 JSC::StructureIDTable::get () at ../Source/JavaScriptCore/runtime/StructureIDTable.h:175 #5 JSC::VM::getStructure () at ../Source/JavaScriptCore/runtime/VM.h:897 #6 JSC::JSCell::structure () at ../Source/JavaScriptCore/runtime/JSCellInlines.h:129 #7 JSC::JSObject::ensureLengthSlow () at ../Source/JavaScriptCore/runtime/JSObject.cpp:3389 #8 0x00007ffff1ff8214 in JSC::JSObject::ensureLength () at ../Source/JavaScriptCore/runtime/JSObject.h:1029 #9 JSC::JSObject::putByIndexBeyondVectorLengthWithoutAttributes<(unsigned char)8> () at ../Source/JavaScriptCore/runtime/JSObject.cpp:2813 #10 0x00007ffff1fd84bc in JSC::JSObject::putDirectIndexSlowOrBeyondVectorLength () at ../Source/JavaScriptCore/runtime/JSObject.cpp:3144 #11 0x00007ffff1ccfbe4 in JSC::JSObject::putDirectIndex () at ../Source/JavaScriptCore/runtime/JSObject.h:246 #12 llint_slow_path_put_by_val_direct () at ../Source/JavaScriptCore/llint/LLIntSlowPaths.cpp:1065 #13 0x00007ffff1ca3d34 in JSC::LLInt::CLoop::execute () at DerivedSources/JavaScriptCore/LLIntAssembly.h:11279 #14 0x00007ffff1cca380 in vmEntryToJavaScript () at ../Source/JavaScriptCore/llint/LLIntThunks.cpp:171 #15 0x00007ffff1c80858 in JSC::JITCode::execute () at ../Source/JavaScriptCore/jit/JITCodeInlines.h:38 #16 JSC::Interpreter::executeCall () at ../Source/JavaScriptCore/interpreter/Interpreter.cpp:910 #17 0x00007ffff1e59128 in JSC::call () at ../Source/JavaScriptCore/runtime/CallData.cpp:59 #18 0x00007ffff1e594e0 in JSC::profiledCall () at ../Source/JavaScriptCore/runtime/CallData.cpp:80 #19 0x00007ffff1fc34b8 in JSC::JSMicrotask::run () at ../Source/JavaScriptCore/runtime/JSMicrotask.cpp:96 #20 0x00007ffff5c370f8 in WebCore::JSExecState::runTask () at ../Source/WebCore/bindings/js/JSExecState.h:91 #21 WebCore::JSMicrotaskCallback::call () at ../Source/WebCore/bindings/js/JSMicrotaskCallback.h:46 #22 operator() () at ../Source/WebCore/bindings/js/JSDOMWindowBase.cpp:214 #23 call () at DerivedSources/ForwardingHeaders/wtf/Function.h:52 #24 0x00007ffff5f72f7c in WTF::Function<void ()>::operator()() const () at DerivedSources/ForwardingHeaders/wtf/Function.h:84 #25 WebCore::EventLoopFunctionDispatchTask::execute () at ../Source/WebCore/dom/EventLoop.cpp:134 #26 0x00007ffff5f96a30 in WebCore::MicrotaskQueue::performMicrotaskCheckpoint () at ../Source/WebCore/dom/Microtasks.cpp:64 #27 0x00007ffff5f71910 in WebCore::EventLoop::performMicrotaskCheckpoint () at ../Source/WebCore/dom/EventLoop.cpp:51 #28 0x00007ffff5f72060 in WebCore::EventLoopTaskGroup::performMicrotaskCheckpoint () at ../Source/WebCore/dom/EventLoop.cpp:155 #29 0x00007ffff5c4bd24 in WebCore::JSExecState::didLeaveScriptContext () at ../Source/WebCore/bindings/js/JSExecState.cpp:42 #30 0x00007ffff5c4db28 in WebCore::JSExecState::~JSExecState () at ../Source/WebCore/bindings/js/JSExecState.h:144 #31 WebCore::JSExecState::profiledCall () at ../Source/WebCore/bindings/js/JSExecState.h:72 #32 WebCore::JSEventListener::handleEvent () at ../Source/WebCore/bindings/js/JSEventListener.cpp:180 #33 0x00007ffff5f779cc in WebCore::EventTarget::innerInvokeEventListeners () at ../Source/WebCore/dom/EventTarget.cpp:308 #34 0x00007ffff5f7a1c8 in WebCore::EventTarget::fireEventListeners () at ../Source/WebCore/dom/EventTarget.cpp:246 #35 0x00007ffff5f7a794 in WebCore::EventTarget::dispatchEvent () at ../Source/WebCore/dom/EventTarget.cpp:204 #36 0x00007ffff6f40080 in WebCore::XMLHttpRequest::dispatchEvent () at ../Source/WebCore/xml/XMLHttpRequest.cpp:1091 #37 0x00007ffff6f46070 in WebCore::XMLHttpRequestProgressEventThrottle::dispatchEventWhenPossible () at ../Source/WebCore/xml/XMLHttpRequestProgressEventThrottle.cpp:91 #38 0x00007ffff6f46408 in WebCore::XMLHttpRequestProgressEventThrottle::dispatchProgressEvent () at ../Source/WebCore/xml/XMLHttpRequestProgressEventThrottle.cpp:105 --Type <RET> for more, q to quit, c to continue without paging-- #39 WebCore::XMLHttpRequestProgressEventThrottle::dispatchProgressEvent () at ../Source/WebCore/xml/XMLHttpRequestProgressEventThrottle.cpp:94 #40 0x00007ffff6f486d4 in WebCore::XMLHttpRequest::callReadyStateChangeListener () at ../Source/WebCore/xml/XMLHttpRequest.cpp:316 #41 0x00007ffff6f487f8 in WebCore::XMLHttpRequest::changeState () at ../Source/WebCore/xml/XMLHttpRequest.cpp:298 #42 WebCore::XMLHttpRequest::changeState () at ../Source/WebCore/xml/XMLHttpRequest.cpp:281 #43 0x00007ffff6f49890 in WebCore::XMLHttpRequest::didFinishLoading () at ../Source/WebCore/xml/XMLHttpRequest.cpp:937 #44 0x00007ffff644b744 in WebCore::DocumentThreadableLoader::didFinishLoading () at ../Source/WebCore/loader/DocumentThreadableLoader.cpp:475 #45 0x00007ffff644b9ac in non-virtual thunk to WebCore::DocumentThreadableLoader::notifyFinished(WebCore::CachedResource&) () at ../Source/WebCore/loader/DocumentThreadableLoader.cpp:448 #46 0x00007ffff651728c in WebCore::CachedResource::checkNotify () at ../Source/WebCore/loader/cache/CachedResource.cpp:371 #47 WebCore::CachedResource::checkNotify () at ../Source/WebCore/loader/cache/CachedResource.cpp:364 #48 0x00007ffff6515f24 in WebCore::CachedResource::finishLoading () at ../Source/WebCore/loader/cache/CachedResource.cpp:387 #49 0x00007ffff6520508 in WebCore::CachedRawResource::finishLoading () at ../Source/WebCore/loader/cache/CachedRawResource.cpp:120 #50 0x00007ffff64c9400 in WebCore::SubresourceLoader::didFinishLoading () at ../Source/WebCore/loader/SubresourceLoader.cpp:701 #51 0x00007ffff50d6490 in WebKit::WebResourceLoader::didFinishResourceLoad () at ../Source/WebKit/WebProcess/Network/WebResourceLoader.cpp:229 #52 0x00007ffff4a724c8 in IPC::callMemberFunctionImpl<WebKit::WebResourceLoader, void (WebKit::WebResourceLoader::*)(WebCore::NetworkLoadMetrics const&), std::tuple<WebCore::NetworkLoadMetrics>, 0ul> () at ../Source/WebKit/Platform/IPC/HandleMessage.h:41 #53 IPC::callMemberFunction<WebKit::WebResourceLoader, void (WebKit::WebResourceLoader::*)(WebCore::NetworkLoadMetrics const&), std::tuple<WebCore::NetworkLoadMetrics>, std::integer_sequence<unsigned long, 0ul> > () at ../Source/WebKit/Platform/IPC/HandleMessage.h:47 #54 IPC::handleMessage<Messages::WebResourceLoader::DidFinishResourceLoad, WebKit::WebResourceLoader, void (WebKit::WebResourceLoader::*)(WebCore::NetworkLoadMetrics const&)> () at ../Source/WebKit/Platform/IPC/HandleMessage.h:120 #55 0x00007ffff4a71058 in WebKit::WebResourceLoader::didReceiveWebResourceLoaderMessage () at DerivedSources/WebKit/WebResourceLoaderMessageReceiver.cpp:66 #56 0x00007ffff50c1db8 in WebKit::NetworkProcessConnection::didReceiveMessage () at ../Source/WebKit/WebProcess/Network/NetworkProcessConnection.cpp:88 #57 0x00007ffff4bebd48 in IPC::Connection::dispatchMessage () at ../Source/WebKit/Platform/IPC/Connection.cpp:1008 #58 0x00007ffff4bedaa0 in IPC::Connection::dispatchMessage () at ../Source/WebKit/Platform/IPC/Connection.cpp:1077 #59 0x00007ffff4bee2dc in IPC::Connection::dispatchOneIncomingMessage () at ../Source/WebKit/Platform/IPC/Connection.cpp:1146 #60 0x00007ffff4bee854 in operator() () at ../Source/WebKit/Platform/IPC/Connection.cpp:985 #61 call () at DerivedSources/ForwardingHeaders/wtf/Function.h:52 #62 0x00007ffff222736c in WTF::Function<void ()>::operator()() const () at ../Source/WTF/wtf/Function.h:84 #63 WTF::RunLoop::performWork () at ../Source/WTF/wtf/RunLoop.cpp:107 #64 0x00007ffff228f508 in operator() () at ../Source/WTF/wtf/glib/RunLoopGLib.cpp:68 #65 _FUN () at ../Source/WTF/wtf/glib/RunLoopGLib.cpp:70 #66 0x00007ffff228f590 in operator() () at ../Source/WTF/wtf/glib/RunLoopGLib.cpp:45 #67 _FUN () at ../Source/WTF/wtf/glib/RunLoopGLib.cpp:46 #68 0x00007ffff2b4e9d8 in g_main_context_dispatch () from /lib64/libglib-2.0.so.0 #69 0x00007ffff2b4eeb8 in g_main_context_iterate.constprop () from /lib64/libglib-2.0.so.0 #70 0x00007ffff2b4f3ec in g_main_loop_run () from /lib64/libglib-2.0.so.0 #71 0x00007ffff22907d4 in WTF::RunLoop::run () at ../Source/WTF/wtf/glib/RunLoopGLib.cpp:96 #72 0x00007ffff51d8a14 in WebKit::AuxiliaryProcessMain<WebKit::WebProcess, WebKit::WebProcessMainGtk> () at ../Source/WebKit/Shared/AuxiliaryProcessMain.h:68 #73 0x00007ffff51d7f78 in WebKit::WebProcessMain () at ../Source/WebKit/WebProcess/gtk/WebProcessMainGtk.cpp:68 --Type <RET> for more, q to quit, c to continue without paging-- #74 0x00000001000007d0 in main () at ../Source/WebKit/WebProcess/EntryPoint/unix/WebProcessMain.cpp:45 ```
*** Bug 210686 has been marked as a duplicate of this bug. ***
The only practical way to debug this is to bisect to find the commit that broke it. Any chance you're able to try your own build from git and see if you can reproduce the crash there? If so, then it's bisectable. $ git clone git://git.webkit.org/WebKit-https.git WebKit Once you've cloned (it will take a while, but don't do a shallow clone because we need the history), follow the instructions "Building WebKitGTK from a release tarball" at [1], NOT the instructions "Building WebKitGTK from git" below that. Except do use git, not a release tarball. I know that's confusing, but it will be fine. ;) If you have trouble, then I can try to bisect it for you with the access to your machine that you offered, but that will go onto my TODO list with no ETA, so quickest way is to try yourself. My guess is that a bisect would require about 10 builds, probably can be done in a long morning or afternoon on your hardware. [1] https://trac.webkit.org/wiki/BuildingGtk#BuildingWebKitGTKfromareleasetarball
I forgot to mention a couple things. First, before you do anything else: $ sudo dnf builddep webkit2gtk3 $ sudo dnf install 'pkgconfig(libsystemd)' And when you run cmake, explicitly enable MiniBrowser so that you can test the result in MiniBrowser. That will be easier than trying to test your build in Epiphany: $ cmake -DPORT=GTK -DCMAKE_BUILD_TYPE=RelWithDebInfo -DENABLE_MINIBROWSER=ON -GNinja
Well I also forgot about bug #1823031. That's going to cause some trouble for bisecting. When bisecting, you have to be very careful to check which crash you're getting and make sure it's not bug #1823031. You'll inevitably hit that crash probably for large portions of the bisect, in which case you'll need to manually apply this patch here and then un-apply before going on to the next revision: https://src.fedoraproject.org/rpms/webkit2gtk3/raw/d2086487babfb8d1e2c4cff00f7b4123262abf89/f/fix-ppc64le.patch That should work for most revisions that you test; there will be a couple weeks where that patch won't apply cleanly, but if you're unlucky and hit one of those revisions it should be pretty easy to adapt it to apply. OK, I know that's a bit more complex that I was hoping for, but it should still be doable... good luck, let me know if you have questions.
Sure, let me try. I will come back to you asap.
(In reply to Michael Catanzaro from comment #4) > Well I also forgot about bug #1823031. That's going to cause some trouble > for bisecting. When bisecting, you have to be very careful to check which > crash you're getting and make sure it's not bug #1823031. You'll inevitably > hit that crash probably for large portions of the bisect, in which case > you'll need to manually apply this patch here and then un-apply before going > on to the next revision: > > https://src.fedoraproject.org/rpms/webkit2gtk3/raw/ > d2086487babfb8d1e2c4cff00f7b4123262abf89/f/fix-ppc64le.patch > > That should work for most revisions that you test; there will be a couple > weeks where that patch won't apply cleanly, but if you're unlucky and hit > one of those revisions it should be pretty easy to adapt it to apply. > > OK, I know that's a bit more complex that I was hoping for, but it should > still be doable... good luck, let me know if you have questions. No luck so far for me. I am trying to find one version that actually works with FC31 before starting the git bisecting process. I am now at 2.27.4 (no luck) and continuing to go further back (let's hope it would work with 2.26.4). Btw, I think this issue is also related to other components that are specific to FC32. I could verify that webkit2gtk3-2.28.0 + epiphany 3.34 runs perfectly on FC31.
EDIT: s/that actually works with FC31/that actually works with FC32
(In reply to Trung LE from comment #6) > Btw, I think this issue is also related to other components that are > specific to FC32. I could verify that webkit2gtk3-2.28.0 + epiphany 3.34 > runs perfectly on FC31. You need to look at the Fedora package revision as well if you're using the packaged version. 2.28.0 is guaranteed to crash on ppc64le due to bug #209236. 2.28.0-7 has my first attempt to fix it (which you had reported broken, but which might not have been since you might have been seeing the second crash instead) and 2.28.0-8 has my second attempt.
> You need to look at the Fedora package revision as well if you're using the packaged version. 2.28.0 is guaranteed to crash on ppc64le due to bug #209236. 2.28.0-7 has my first attempt to fix it (which you had reported broken, but which might not have been since you might have been seeing the second crash instead) and 2.28.0-8 has my second attempt. I fetch the repo with `git clone https://src.fedoraproject.org/rpms/webkit2gtk3.git` then switch to f31 branch with `git checkout f31` and checkout the 2.26.4 commit with `git reset a9158c8285624fe6b1a41691bb079ea963c1b04f && git reset --hard && git clean -df` Next I build the RPM with `fedpkg --release f32 local` Once the RPMs are generated, I then install them with `sudo rpm -i ppc64le/*.rpm`. I open the Minibrowser by `/user/libexec/webkit2gtk-4.0/MiniBrowser`. The app report following error: ``` /usr/libexec/webkit2gtk-4.0/WebKitWebProcess: symbol lookup error: /usr/libexec/webkit2gtk-4.0/WebKitWebProcess: undefined symbol: WebProcessMainUnix ``` Running `/usr/libexec/webkit2gtk-4.0/WebKitWebProcess` directly also yield the same error. Wondering if you could suggest how to work-around this issue. My objective is to confirm that FC32 does not have issue with 2.26.4 Minibrowser.
Hm, I don't know why your build is broken, but I didn't mean to suggest trying to build old packages on new Fedora. It would make more sense to just go back in time and try to figure out: when was the last time this worked? Does WebKitGTK work in F31? Does it work in F30? Does it work in F29? Eventually if you go back far enough, it ought to work, and we'll have a starting point. And if not, then we know this isn't a regression.
(In reply to Michael Catanzaro from comment #10) > Hm, I don't know why your build is broken, but I didn't mean to suggest > trying to build old packages on new Fedora. It would make more sense to just > go back in time and try to figure out: when was the last time this worked? > Does WebKitGTK work in F31? Does it work in F30? Does it work in F29? > Eventually if you go back far enough, it ought to work, and we'll have a > starting point. And if not, then we know this isn't a regression. Sorry for late reply. Here is my finding: On Fedora 31 webkit2gtk3 2.26.0 to 2.28.2 ALL WORK :) On Fedora 32 webkit2gtk3 2.26.4 to 2.28.2 NONE WORK :( I could __not__ find a working starting revision to start the git bisection.
(In reply to Trung LE from comment #11) > On Fedora 31 webkit2gtk3 2.26.0 to 2.28.2 ALL WORK :) > > On Fedora 32 webkit2gtk3 2.26.4 to 2.28.2 NONE WORK :( > > I could __not__ find a working starting revision to start the git bisection. OK, that's weird. So that's proof that this is not a regression in WebKit, but a regression somewhere else. I have absolutely no clue what component to suspect.
(In reply to Michael Catanzaro from comment #12) > (In reply to Trung LE from comment #11) > > On Fedora 31 webkit2gtk3 2.26.0 to 2.28.2 ALL WORK :) > > > > On Fedora 32 webkit2gtk3 2.26.4 to 2.28.2 NONE WORK :( > > > > I could __not__ find a working starting revision to start the git bisection. > > OK, that's weird. So that's proof that this is not a regression in WebKit, > but a regression somewhere else. > > I have absolutely no clue what component to suspect. Again on Fedora 32, I was hopeful that 2.26.4 would run correctly but I ran into the /usr/libexec/webkit2gtk-4.0/WebKitWebProcess: symbol lookup error: /usr/libexec/webkit2gtk-4.0/WebKitWebProcess: undefined symbol: WebProcessMainUnix error which is totally different to the reported JSC one. Is there any chance you could trigger a custom-built RPM of the Fc32 2.26.4 on koji build system?
There's no point. You've already proven that 2.28.2 works fine in F31.
I don't run Fedora, but I can confirm that on Void Linux ppc64le this issue happens when updating to 2.28.2 but not in 2.26.4 (I'm the package maintainer, currently working on updating it). I get that exact StructureIDTable backtrace.
Also, this does not crash always. Simple sites work (even with some JS), only complicated javascript (e.g. on twitter) crashes it.
Anyway, I have the whole Git history cloned, built, managed to reproduce the issue, currently going back in history to catch the exact revision.
I narrowed the crash down to a single commit: http://git.webkit.org/?p=WebKit.git;a=commit;h=2c0c47246d650167ff6c551d5c346f194538e0da This will not build on its own (see the #errors about unknown architecture) so it needs to be combined with a fix that came later: http://git.webkit.org/?p=WebKit.git;a=commit;h=bc5d7c9715a510503a0155b9fdbecbf5c551f559 After that, the crash is reproducible. The commit right before the first change, i.e. http://git.webkit.org/?p=WebKit.git;a=commit;h=2664ebec35d469bd909c9df8c5465c74a6cd49d0 builds and works perfectly fine; for anything after that, I get the StructureIDTable crash.
I will try reverting this on top of master later to verify, as well as try to figure out why exactly this is crashing. It's late here though, so that's for later; if you have a hunch or something else, I'd appreciate that, too.
Great work, thanks Daniel. (In reply to Michael Catanzaro from comment #14) > There's no point. You've already proven that 2.28.2 works fine in F31. Maybe a false negative, since Daniel says the crash does not always occur?
I built WebKit master, which crashes in the same way. I then went on to modify GCMemoryOperations.h and replaced its functions with simply the following: ``` template <typename T> ALWAYS_INLINE void gcSafeMemcpy(T* dst, T* src, size_t bytes) { static_assert(sizeof(T) == sizeof(JSValue)); RELEASE_ASSERT(bytes % 8 == 0); memcpy(dst, src, bytes); } template <typename T> ALWAYS_INLINE void gcSafeMemmove(T* dst, T* src, size_t bytes) { static_assert(sizeof(T) == sizeof(JSValue)); RELEASE_ASSERT(bytes % 8 == 0); memmove(dst, src, bytes); } template <typename T> ALWAYS_INLINE void gcSafeZeroMemory(T* dst, size_t bytes) { static_assert(sizeof(T) == sizeof(JSValue)); RELEASE_ASSERT(bytes % 8 == 0); memset(reinterpret_cast<char*>(dst), 0, bytes); } ``` It no longer crashes when I do that (it effectively reverts the SVN revision, except it makes sure the problem is not in the way these functions are called). So it's definitely that specific rev that's the problem.
I believe I have fixed the problem. I'm currently doing a rebuild to verify that. Will attach a patch once I've confirmed it
Created attachment 398740 [details] Patch to fix the regression To quote the commit message: The problem at hand here is that the control flow is wrong. As it was, we'd do something like: ``` if (bytes <= smallCutoff) { slow path } else if (aarch64 || bytes <= mediumCutoff) { either x86_64 path, aarch64 path or slow path } else { assert(x86_64) do x86_64 path, or nothing on other archs } ``` That means everything on non-x86_64/aarch64 that tried to memcpy more than mediumCutoff would end up doing nothing. Fix the code so that slow path is taken automatically always if running non-x86_64/aarch64 architectures. Remove the #else in the mediumCutoff branch as that is now never taken.
(In reply to Daniel Kolesa from comment #23) > Created attachment 398740 [details] > Patch to fix the regression > > To quote the commit message: > > The problem at hand here is that the control flow is wrong. As it was, we'd > do something like: > > ``` > if (bytes <= smallCutoff) { > slow path > } else if (aarch64 || bytes <= mediumCutoff) { > either x86_64 path, aarch64 path or slow path > } else { > assert(x86_64) > do x86_64 path, or nothing on other archs > } > ``` > > That means everything on non-x86_64/aarch64 that tried to memcpy more than > mediumCutoff would end up doing nothing. > > Fix the code so that slow path is taken automatically always if running > non-x86_64/aarch64 architectures. Remove the #else in the mediumCutoff > branch as that is now never taken. Good catch. I can confirm that patch addresses the regression.
Nice.
(In reply to Daniel Kolesa from comment #23) > That means everything on non-x86_64/aarch64 that tried to memcpy more than > mediumCutoff would end up doing nothing. Well it must be hitting the RELEASE_ASSERT(isX86_64()) there. Oops! Normally a RELEASE_ASSERT() is a smoking gun, but because the function used ALWAYS_INLINE, it didn't show up in the backtrace, which made this a *lot* harder than it should have been. :/
Makes sense. Either way, webkit master now builds and functions properly, which should take care of webkitgtk for a while. I think I'll submit my 32-bit big endian llint patch too while at it...
Comment on attachment 398740 [details] Patch to fix the regression View in context: https://bugs.webkit.org/attachment.cgi?id=398740&action=review If you have a git checkout, please run Tools/Scripts/prepare-ChangeLog -b 210685 to prepare the patch for Bugzilla. If you're working from tarball, probably easiest for me to do that for you. > Source/JavaScriptCore/heap/GCMemoryOperations.h:57 > + if (bytes <= smallCutoff || (!isARM64() && !isX86_64())) Probably best to use build guards here rather than runtime guards. What I would do in your patch is: not touch this line, keep the call to slowPathForwardMemcpy() belong, remove the RELEASE_ASSERT(isX86_64()), and then add another #else that also calls slowPathForwardMemcpy() when this second #if CPU(X86_64) branch is not taken. Sound good? That results in a confusing mess, but importantly it would be parallel to the confusing mess in the other two functions, below, both of which duplicate the fallback path first for non-x86_64/aarch64 and then again for non-GCC/clang. Then I can follow up with a patch to change the guards so that we don't need to write the fallback case twice in a row at the bottom of the function.
(In reply to Michael Catanzaro from comment #28) > What I would do in your patch is: not touch this line, keep the call to > slowPathForwardMemcpy() belong, I meant: "below"
Doing that results in needless branches being taken based on smallCutoff/mediumCutoff, which just falling back to slow path early prevents... runtime guards shouldn't matter too much, these checks are constexpr, and the compiler will fold them away, plus it's shorter and less messy
(In reply to Daniel Kolesa from comment #27) > Makes sense. Either way, webkit master now builds and functions properly, > which should take care of webkitgtk for a while. I think I'll submit my > 32-bit big endian llint patch too while at it... Sounds like you are using a git checkout. If you're bored, you could try Tools/run-jsc-stress-tests and see if it *really* works. :P On internal CI, I'm currently seeing 150 failures on ppc64le and s390x that weren't there in 2.26, but it's entirely plausible you just fixed them. ;)
Created attachment 398760 [details] patch with changelog entry added changelog entry
I can run the stress tests, how do I do that from a built git tree? I've just built it using cmake as usual
(In reply to Daniel Kolesa from comment #30) > Doing that results in needless branches being taken based on > smallCutoff/mediumCutoff, which just falling back to slow path early > prevents... OK, you're right. That would be avoided by my proposed follow-up patch, though. If you prefer, we can do it all in one patch. In WebKit we have an annoying rule that you only attach one patch per bug report, so we have a tendency to solve multiple slightly-related things in the same patch that might be expected to be two separate patches in other projects. We can fix it with build guards and also clean up the guards all in one. Here's what I was thinking: ``` #if USE(JSVALUE64) #if COMPILER(GCC_COMPATIBLE) && (CPU(X86_64) || CPU(ARM64)) if (bytes <= smallCutoff) { slow path } else if (aarch64 || bytes <= mediumCutoff) { #if CPU(X86_64) x86_64 fast path #elif CPU(ARM64) aarch64 fast path #endif } else { assert(x86_64) #if (X86_64) x86_64 fast path #endif } #else // non-x86_64/aarch64 or non-GCC-compatible JSVALUE64 path slow path #else // JSVALUE32 path vanilla memcpy #endif ``` I think that's the simplest we can get it down to (for some value of "simple"). Then we'd make the same change in the other two functions as well, adding && (CPU(X86_64) || CPU(ARM64)) guards at the top of the chain so that we can compress the non-GCC compat and non-x86_64/aarch64 fallbacks together. Do you agree that makes sense? I can upload a patch if you want, but you get your name in the commit message if you try. ;) We can also go with your simple patch here, but I would wind up basically undoing the change in that follow-up.
yeah that looks good to me, I can update the patch
(In reply to Daniel Kolesa from comment #33) > I can run the stress tests, how do I do that from a built git tree? I've > just built it using cmake as usual You probably need to use Tools/Scripts/update-webkitgtk-libs and then Tools/Scripts/build-webkit --gtk first. Come to think of it, that's probably not worth the effort; you're as likely as not to spend the rest of the day debugging the development scripts. I'll find out whether our ppc64le and s390x bots are happy with this change soon enough anyway.
Alright, I have an updated patch (which actually makes it *more* consistent with the other functions, the logic was overall kinda broken) without changing any more than necessary, just doing one more build to check if I haven't messed up and things still work.
Created attachment 398767 [details] updated to use compile-time guards This updates the patch to use compile-time guards. The second USE(JSVALUE64) was not necessary, since the whle thing is wrapped in USE(JSVALUE64) above that, so instead I replaced it with arch guards, which effectively caused the slow path to be taken for other JSVALUE64 archs always (plain memcpy fallback for non-JSVALUE64 works as before).
Comment on attachment 398767 [details] updated to use compile-time guards Exactly, thanks. I'll follow-up to change this in the other two functions as well, since they can benefit from the same change.
Also confirmed functionality on ppc64 (big endian), with webkitgtk 2.28.2.
Committed r261326: <https://trac.webkit.org/changeset/261326> All reviewed patches have been landed. Closing bug and clearing flags on attachment 398767 [details].
<rdar://problem/62988277>
(In reply to Michael Catanzaro from comment #36) > I'll find out whether our ppc64le and > s390x bots are happy with this change soon enough anyway. You fixed 149 out of 150 test failures. Thanks. ;)