Bug 291759 – [Skia] Crashes in GrResourceCache::notifyARefCntReachedZero, GrResourceCache::findAndRefScratchResource

• Status: RESOLVED
• Resolution: FIXED
• Component: WebKitGTK

[Michael Catanzaro]

Created attachment 474957 [details]
Full backtrace

When watching a Kaltura video on an internal Red Hat website using Ephy Tech Preview with WebKitGTK 2.49.1, the web process occasionally crashes with SIGILL, which is a quite unusual way to crash. (I wonder if an illegal instruction is used to implement SkASSERT?)

Program terminated with signal SIGILL, Illegal instruction.
#0  0x00007f66d7a25379 in GrResourceCache::notifyARefCntReachedZero (this=0x5654b90289a0, resource=0x5654ba41b8e0, 
    removedRef=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceCache.cpp:387

warning: 387	/buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceCache.cpp: No such file or directory
[Current thread is 1 (Thread 0x7f66cb837ac0 (LWP 2))]
(gdb) bt
#0  0x00007f66d7a25379 in GrResourceCache::notifyARefCntReachedZero
    (this=0x5654b90289a0, resource=0x5654ba41b8e0, removedRef=<optimized out>)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceCache.cpp:387
#1  0x00007f66d7a2c407 in GrIORef<GrGpuResource>::notifyWillBeZero
    (this=0x5654ba41b8e8, removedRef=GrIORef<GrGpuResource>::LastRemovedRef::kMainRef)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrGpuResource.h:102
#2  GrIORef<GrGpuResource>::unref (this=0x5654ba41b8e8)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrGpuResource.h:65
#3  SkSafeUnref<GrSurface> (obj=0x5654ba41b8e0)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:151
#4  sk_sp<GrSurface>::reset (this=0x5654baef593b, ptr=0x0)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:316
#5  sk_sp<GrSurface>::operator=(decltype(nullptr)) (this=0x5654baef593b)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:260
#6  GrTextureProxy::~GrTextureProxy (this=0x5654baef5910, vtt=<optimized out>)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrTextureProxy.cpp:110
#7  0x00007f66d7a2d8bf in GrTextureRenderTargetProxy::~GrTextureRenderTargetProxy (this=0x5654baef58e0, vtt=<optimized out>)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrTextureRenderTargetProxy.h:44
#8  GrTextureRenderTargetProxy::~GrTextureRenderTargetProxy (this=0x5654baef58e0)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrTextureRenderTargetProxy.h:44
#9  GrTextureRenderTargetProxy::~GrTextureRenderTargetProxy (this=0x5654baef58e0)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrTextureRenderTargetProxy.h:44
#10 0x00007f66d7a2d8bf in virtual thunk to GrTextureRenderTargetProxy::~GrTextureRenderTargetProxy() ()
    at /usr/lib/x86_64-linux-gnu/libwebkitgtk-6.0.so.4

Unfortunately it's 35 frames deep inside Skia. The first WebKit frame is:

#36 0x00007f66d7217a5d in WebCore::createAcceleratedRenderingFence<sk_sp<SkImage> const> (object=...)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/skia/GraphicsContextSkia.cpp:1054

I will attach a full backtrace.

[Michael Catanzaro]

*** Bug 291836 has been marked as a duplicate of this bug. ***

[Michael Catanzaro]

Bug #291836 contains a stack trace for this happening on youtube.com.

[Michael Catanzaro]

Here's a completely different stack trace:

(gdb) bt
#0  skia_private::AutoSTMalloc<8ul, unsigned int, void>::get (this=0x7f4a40e68910)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/private/base/SkTemplates.h:383
#1  skgpu::ResourceKey::operator== (that=..., this=<optimized out>)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ResourceKey.h:102
#2  skgpu::ScratchKey::operator== (that=..., this=<optimized out>)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ResourceKey.h:217
#3  skia_private::THashTable<SkTMultiMap<GrGpuResource, skgpu::ScratchKey, GrResourceCache::ScratchMapTraits>::ValueList*, skgpu::ScratchKey, SkTDynamicHash<SkTMultiMap<GrGpuResource, skgpu::ScratchKey, GrResourceCache::ScratchMapTraits>::ValueList, skgpu::ScratchKey, SkTMultiMap<GrGpuResource, skgpu::ScratchKey, GrResourceCache::ScratchMapTraits>::ValueList>::AdaptedTraits>::find
    (this=0x55b60c637978, key=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/core/SkTHash.h:126
#4  0x00007f4fb2624b44 in skia_private::THashTable<SkTMultiMap<GrGpuResource, skgpu::ScratchKey, GrResourceCache::ScratchMapTraits>::ValueList*, skgpu::ScratchKey, SkTDynamicHash<SkTMultiMap<GrGpuResource, skgpu::ScratchKey, GrResourceCache::ScratchMapTraits>::ValueList, skgpu::ScratchKey, SkTMultiMap<GrGpuResource, skgpu::ScratchKey, GrResourceCache::ScratchMapTraits>::ValueList>::AdaptedTraits>::findOrNull
    (this=0x55b60c637978, key=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/core/SkTHash.h:138
#5  SkTDynamicHash<SkTMultiMap<GrGpuResource, skgpu::ScratchKey, GrResourceCache::ScratchMapTraits>::ValueList, skgpu::ScratchKey, SkTMultiMap<GrGpuResource, skgpu::ScratchKey, GrResourceCache::ScratchMapTraits>::ValueList>::find (this=0x55b60c637978, key=...)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/core/SkTDynamicHash.h:42
#6  SkTMultiMap<GrGpuResource, skgpu::ScratchKey, GrResourceCache::ScratchMapTraits>::find (this=0x55b60c637978, key=...)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/core/SkTMultiMap.h:95
#7  GrResourceCache::findAndRefScratchResource (this=0x55b60c637930, scratchKey=...)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceCache.cpp:216
#8  0x00007f4fb2628346 in GrResourceProvider::findAndRefScratchTexture
    (this=0x55b60c70a290, key=..., label="TextureRenderTarget_FullyLazyProxy")
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceProvider.cpp:336
#9  GrResourceProvider::findAndRefScratchTexture
    (this=<optimized out>, dimensions=..., format=<optimized out>, textureType=<optimized out>, renderable=<optimized out>, renderTargetSampleCnt=<optimized out>, mipmapped=<optimized out>, isProtected=<optimized out>, label="TextureRenderTarget_FullyLazyProxy")
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceProvider.cpp:369
#10 0x00007f4fb26286c0 in GrResourceProvider::getExactScratch
    (this=<optimized out>, dimensions=..., format=<optimized out>, textureType=<optimized out>, renderable=<optimized out>, renderTargetSampleCnt=<optimized out>, budgeted=<optimized out>, mipmapped=<optimized out>, isProtected=<optimized out>, label=Python Exception <class 'gdb.error'>: value has been optimized out
)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceProvider.cpp:140
#11 GrResourceProvider::createTexture
    (this=0x55b60c70a290, dimensions=..., format=..., textureType=GrTextureType::k2D, renderable=<optimized out>, renderTargetSampleCnt=4, mipmapped=<optimized out>, budgeted=<optimized out>, isProtected=<optimized out>, label="TextureRenderTarget_FullyLazyProxy")
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceProvider.cpp:256
#12 0x00007f4fb2872e9a in GrDynamicAtlas::reset(SkISize, GrCaps const&)::$_0::operator()(GrResourceProvider*, GrSurfaceProxy::LazySurfaceDesc const&) const (this=<optimized out>, resourceProvider=0x55b60c70a290, desc=<optimized out>)
    at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrDynamicAtlas.cpp:110
#13 std::__invoke_impl<GrSurfaceProxy::LazyCallbackResult, GrDynamicAtlas::reset(SkISize, GrCaps const&)::$_0&, GrResourceProvider*, GrSurfaceProxy::LazySurfaceDesc const&>(std::__invoke_other, GrDynamicAtlas::reset(SkISize, GrCaps const&)::$_0&, GrResourceProvider*&&, GrSurfaceProxy::LazySurfaceDesc const&) (__f=<optimized out>, __args=<optimized out>, __args=<optimized out>)
    at /usr/bin/../lib/gcc/x86_64-unknown-linux-gnu/14.2.0/../../../../include/c++/14.2.0/bits/invoke.h:61
#14 std::__invoke_r<GrSurfaceProxy::LazyCallbackResult, GrDynamicAtlas::reset(SkISize, GrCaps const&)::$_0&, GrResourceProvider*, GrSurfaceProxy::LazySurfaceDesc const&>(GrDynamicAtlas::reset(SkISize, GrCaps const&)::$_0&, GrResourceProvider*&&, GrSurfaceProxy::LazySurfaceDesc const&) (__fn=<optimized out>, __args=<optimized out>, __args=<optimized out>)
    at /usr/bin/../lib/gcc/x86_64-unknown-linux-gnu/14.2.0/../../../../include/c++/14.2.0/bits/invoke.h:114
#15 std::_Function_handler<GrSurfaceProxy::LazyCallbackResult(GrResourceProvider*, GrSurfaceProxy::LazySurfaceDesc const&), GrDynamicAtlas::reset(SkISize, GrCaps const&)::$_0>::_M_invoke (__functor=<optimized out>, __args=<optimized out>, __args=...)
    at /usr/bin/../lib/gcc/x86_64-unknown-linux-gnu/14.2.0/../../../../include/c++/14.2.0/bits/std_function.h:290

I'll attach the full version. Clearly something is wrong with GrResourceCache.

[Michael Catanzaro]

Created attachment 475083 [details]
Full backtrace for second crash

[Alberto Garcia]

Created attachment 475318 [details]
Full backtrace (Debian bookworm)

It also crashes with Debian bookworm, I'm attaching another backtrace.

I'm having this problem often with 2.48.2 using the Element Matrix client, it was working fine with 2.48.1.

[Philippe Normand]

+1 to Berto's, I was about to report the same issue. Tangram is now on 2.48.2 and Matrix is crashing a lot :(

[Philippe Normand]

Possibly fixed by this cherry-pick, I haven't had time to test yet. https://github.com/WebKit/WebKit/commit/ffa624c2dba1d4e644a1dc970084cf53ed25200b

[Michael Catanzaro]

Another variant, from https://bugzilla.redhat.com/show_bug.cgi?id=2368817:

 #0 GrGpu::getContext at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrGpu.h:75
 #1 get_resource_cache at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrGpuResource.cpp:24
 #2 GrGpuResource::release at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrGpuResource.cpp:57
 #3 GrGpuResource::CacheAccess::release at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrGpuResourceCacheAccess.h:49
 #4 GrResourceCache::releaseAll at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceCache.cpp:183
 #5 GrDirectContext::~GrDirectContext at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrDirectContext.cpp:111
 #7 SkRefCntBase::unref at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/include/core/SkRefCnt.h:78
 #8 SkSafeUnref<GrDirectContext> at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/include/core/SkRefCnt.h:151
 #9 sk_sp<GrDirectContext>::~sk_sp at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/include/core/SkRefCnt.h:256
 #10 WebCore::SkiaGLContext::~SkiaGLContext at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/WebCore/platform/graphics/skia/PlatformDisplaySkia.cpp:127

[Carlos Garcia Campos]

https://github.com/WebKit/WebKit/pull/45990 might fix theses crashes if they still happen in main or 2.48 branch.

[Michael Catanzaro]

Another:

Truncated backtrace:
Thread no. 1 (9 frames)
 #0 SkTDPQueue<GrGpuResource*, &GrResourceCache::CompareTimestamp, &GrResourceCache::AccessResourceIndex>::remove at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceCache.cpp:310
 #1 GrResourceCache::refAndMakeResourceMRU at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceCache.cpp:303
 #2 GrResourceCache::findAndRefScratchResource at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceCache.cpp:219
 #3 GrResourceProvider::createBuffer at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceProvider.cpp:626
 #4 GrBufferAllocPool::getBuffer at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrBufferAllocPool.cpp:425
 #5 GrBufferAllocPool::createBlock at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrBufferAllocPool.cpp:326
 #6 GrBufferAllocPool::makeSpace at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrBufferAllocPool.cpp:229
 #7 GrIndexBufferAllocPool::makeSpace at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrBufferAllocPool.cpp:496
 #8 GrOpFlushState::makeIndexSpace at /usr/src/debug/webkitgtk-2.48.2-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrOpFlushState.cpp:195

[Alberto Garcia]

I confirm that WebKitGTK 2.48.3 fixes the crash for me.

[Michael Catanzaro]

Well that's good news. Must be fixed by 292929@main, then.

[Michael Catanzaro]

*** This bug has been marked as a duplicate of bug 290614 ***

[Michael Catanzaro]

Actually, wait sorry, I'm not sure.

Philippe suggested it might be fixed by that commit, which is present in 2.48.3, but was not sure. Carlos suggested it might be fixed by a different commit that is not present in 2.48.3. So I just assumed Philippe was right. I don't know for certain. I guess it doesn't matter.

[Michael Catanzaro]

Reopening. I just hit this twice using 2.49.2, so not fixed after all. Stack trace is same as in original comment. Unfortunately my reproducer is an internal web page.

[Michael Catanzaro]

Looks like I can reproduce this *somewhat* reliably, in case it would be helpful for me to add prints or check something.

[Michael Catanzaro]

*** Bug 290182 has been marked as a duplicate of this bug. ***

[Michael Catanzaro]

Another stack trace variant from https://bugzilla.redhat.com/show_bug.cgi?id=2373019, indicating memory corruption:

runcated backtrace:
Thread no. 1 (35 frames)
 #0 nouveau_heap_free at ../src/gallium/drivers/nouveau/nouveau_heap.c:113
 #1 nvc0_program_destroy at
../src/gallium/drivers/nouveau/nvc0/nvc0_program.c:985
 #2 nvc0_sp_state_delete at
../src/gallium/drivers/nouveau/nvc0/nvc0_state.c:641
 #3 delete_variant at ../src/mesa/state_tracker/st_program.c:234
 #4 st_release_variants at ../src/mesa/state_tracker/st_program.c:320
 #5 _mesa_delete_program at ../src/mesa/program/program.c:251
 #6 _mesa_reference_program_ at ../src/mesa/program/program.c:338
 #7 _mesa_reference_program at ../src/mesa/program/program.h:93
 #8 _mesa_delete_linked_shader at ../src/mesa/main/shaderobj.c:150
 #9 _mesa_clear_shader_program_data at ../src/mesa/main/shaderobj.c:337
 #10 _mesa_free_shader_program_data at ../src/mesa/main/shaderobj.c:368
 #11 _mesa_delete_shader_program at ../src/mesa/main/shaderobj.c:414
 #12 _mesa_reference_shader_program_ at ../src/mesa/main/shaderobj.c:265
 #13 _mesa_reference_shader_program at ../src/mesa/main/shaderobj.h:82
 #14 delete_shader_program at ../src/mesa/main/shaderapi.c:486
 #15 _mesa_DeleteProgram at ../src/mesa/main/shaderapi.c:1712
 #16 GrGLFunction<void(unsigned int)>::operator() at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/include/gpu/ganesh/gl/GrGLFunctions.h:297
 #17 GrGLProgram::~GrGLProgram at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/gl/GrGLProgram.cpp:103
 #19 SkRefCntBase::unref at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/include/core/SkRefCnt.h:78
 #20 SkSafeUnref<GrGLProgram> at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/include/core/SkRefCnt.h:151
 #21 sk_sp<GrGLProgram>::~sk_sp at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/include/core/SkRefCnt.h:256
 #22 GrGLGpu::ProgramCache::Entry::~Entry at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/gl/GrGLGpuProgramCache.cpp:25
 #23 std::default_delete<GrGLGpu::ProgramCache::Entry>::operator() at
/usr/lib/gcc/x86_64-redhat-linux/15/../../../../include/c++/15/bits/unique_ptr.h:93
 #24 std::unique_ptr<GrGLGpu::ProgramCache::Entry,
std::default_delete<GrGLGpu::ProgramCache::Entry> >::~unique_ptr at
/usr/lib/gcc/x86_64-redhat-linux/15/../../../../include/c++/15/bits/unique_ptr.h:399
 #25 SkLRUCache<GrProgramDesc, std::unique_ptr<GrGLGpu::ProgramCache::Entry,
std::default_delete<GrGLGpu::ProgramCache::Entry> >,
GrGLGpu::ProgramCache::DescHash, SkNoOpPurge>::Entry::~Entry at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/src/core/SkLRUCache.h:26
 #26 SkLRUCache<GrProgramDesc, std::unique_ptr<GrGLGpu::ProgramCache::Entry,
std::default_delete<GrGLGpu::ProgramCache::Entry> >,
GrGLGpu::ProgramCache::DescHash, SkNoOpPurge>::reset at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/src/core/SkLRUCache.h:105
 #27 GrGLGpu::ProgramCache::reset at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/gl/GrGLGpuProgramCache.cpp:53
 #28 GrGLGpu::~GrGLGpu at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/gl/GrGLGpu.cpp:480
 #30 std::default_delete<GrGpu>::operator() at
/usr/lib/gcc/x86_64-redhat-linux/15/../../../../include/c++/15/bits/unique_ptr.h:93
 #31 std::unique_ptr<GrGpu, std::default_delete<GrGpu> >::~unique_ptr at
/usr/lib/gcc/x86_64-redhat-linux/15/../../../../include/c++/15/bits/unique_ptr.h:399
 #32 GrDirectContext::~GrDirectContext at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/src/gpu/ganesh/GrDirectContext.cpp:116
 #34 SkRefCntBase::unref at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/include/core/SkRefCnt.h:78
 #35 SkSafeUnref<GrDirectContext> at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/include/core/SkRefCnt.h:151
 #36 sk_sp<GrDirectContext>::~sk_sp at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/ThirdParty/skia/include/core/SkRefCnt.h:256
 #37 WebCore::SkiaGLContext::~SkiaGLContext at
/usr/src/debug/webkitgtk-2.48.3-1.fc42.x86_64/Source/WebCore/platform/graphics/skia/PlatformDisplaySkia.cpp:127

[Michael Catanzaro]

I think the problem is SkiaPaintingEngine is not actually threadsafe. Clearly it's intended to be safe, but something somewhere is wrong. Perhaps not a very helpful observation, I know.

Just hit this crash on https://www.irs.gov/pub/irs-pdf/f1040.pdf which is WebKit's own PDF.js viewer. There are only two threads using skia. The main thread is calling WebCore::SkiaPaintingEngine::record, while the crash thread is calling WebCore::SkiaPaintingEngine::replay:

Thread 3 (Thread 0x7f6b8346fc40 (LWP 2)):
#0  WTF::WeakRef<WebCore::Node, WebCore::WeakPtrImplWithEventTargetData>::get (this=0x7f6b6ac63518) at WTF/Headers/wtf/WeakRef.h:103
#1  WebCore::RenderObject::document (this=0x7f6b6ac63500) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderObjectInlines.h:38
#2  WebCore::RenderObject::isDocumentElementRenderer (this=0x7f6b6ac63500) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderObjectInlines.h:88
#3  WebCore::RenderBlock::paint(WebCore::PaintInfo&, WebCore::LayoutPoint const&)::$_0::operator()(WebCore::LayoutRect) const (paintingRect=..., this=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderBlock.cpp:1072
#4  WebCore::RenderBlock::paint (this=0x7f6b6ac63500, paintInfo=..., paintOffset=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderBlock.cpp:1084
#5  0x00007f6b8f9a9697 in WebCore::RenderBlock::paintChild (this=0x7f6b6ac5f500, child=..., paintInfo=..., paintOffset=LayoutPoint(0.28125px (18), 0.5625px (36)), paintInfoForChild=..., usePrintRect=<optimized out>, paintType=WebCore::RenderBlock::PaintAsBlock) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderBlock.cpp:1177
#6  0x00007f6b8f9a9370 in WebCore::RenderBlock::paintChildren (this=0x7f6b6ac5f500, paintInfo=..., paintOffset=LayoutPoint(0.28125px (18), 0.5625px (36)), paintInfoForChild=..., usePrintRect=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderBlock.cpp:1134
#7  0x00007f6b8f9a9274 in WebCore::RenderBlock::paintContents (this=0x7f6b6ac5f500, paintInfo=..., paintOffset=LayoutPoint(0.28125px (18), 0.5625px (36))) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderBlock.cpp:1125
#8  0x00007f6b8f9aa1e5 in WebCore::RenderBlock::paintObject (this=0x7f6b6ac5f500, paintInfo=..., paintOffset=LayoutPoint(0.28125px (18), 0.5625px (36))) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderBlock.cpp:1347
#9  0x00007f6b8f9a8b30 in WebCore::RenderBlock::paint (this=0x7f6b6ac5f500, paintInfo=..., paintOffset=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderBlock.cpp:1088
#10 0x00007f6b8fa09f8e in WebCore::paintPhase (phase=WebCore::PaintPhase::ChildBlockBackgrounds, element=<optimized out>, paintInfo=<optimized out>, childPoint=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderElement.cpp:1289
#11 WebCore::RenderElement::paintAsInlineBlock (this=<optimized out>, paintInfo=<optimized out>, childPoint=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderElement.cpp:1302
#12 0x00007f6b8f334fb1 in WebCore::LayoutIntegration::InlineContentPainter::paintDisplayBox (this=0x7ffc161f2538, box=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/layout/integration/inline/LayoutIntegrationInlineContentPainter.cpp:125
#13 0x00007f6b8f33520b in WebCore::LayoutIntegration::InlineContentPainter::paint (this=0x7ffc161f2538) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/layout/integration/inline/LayoutIntegrationInlineContentPainter.cpp:171
#14 0x00007f6b8f33b3c4 in WebCore::LayoutIntegration::LineLayout::paint (this=0x7f68e2914180, paintInfo=..., paintOffset=LayoutPoint(0.28125px (18), 0.5625px (36)), layerRenderer=0x0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/layout/integration/inline/LayoutIntegrationLineLayout.cpp:1168
#15 0x00007f6b8f9aa1e5 in WebCore::RenderBlock::paintObject (this=0x7f68e279ef00, paintInfo=..., paintOffset=LayoutPoint(0.28125px (18), 0.5625px (36))) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderBlock.cpp:1347
#16 0x00007f6b8f9a8b30 in WebCore::RenderBlock::paint (this=0x7f68e279ef00, paintInfo=..., paintOffset=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderBlock.cpp:1088
#17 0x00007f6b8fa60491 in WebCore::RenderLayer::paintForegroundForFragmentsWithPhase (this=0x7f68e27a1600, phase=WebCore::PaintPhase::Foreground, layerFragments=WTF::Vector of length 1, capacity 1 = {...}, context=..., localPaintingInfo=..., paintBehavior=..., subtreePaintRootForRenderer=0x0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderLayer.cpp:4345
#18 0x00007f6b8fa5e687 in WebCore::RenderLayer::paintForegroundForFragments (this=0x7f68e27a1600, layerFragments=WTF::Vector of length 1, capacity 1 = {...}, context=..., contextForTransparencyLayer=..., transparencyPaintDirtyRect=..., haveTransparency=<optimized out>, localPaintingInfo=..., paintBehavior=..., subtreePaintRootForRenderer=0x0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderLayer.cpp:4322
#19 0x00007f6b8fa5b4f0 in WebCore::RenderLayer::paintLayerContents (this=0x7f68e27a1600, context=..., paintingInfo=..., paintFlags=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderLayer.cpp:3885
#20 0x00007f6b8fa7c990 in WebCore::RenderLayerBacking::paintIntoLayer(WebCore::GraphicsLayer const*, WebCore::GraphicsContext&, WebCore::IntRect const&, WTF::OptionSet<WebCore::PaintBehavior>, WebCore::RegionContext*)::$_0::operator()(WebCore::RenderLayer&, WTF::OptionSet<WebCore::RenderLayer::PaintLayerFlag>) const (this=this@entry=0x7ffc161f2c58, layer=..., paintFlags=paintFlags@entry=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderLayerBacking.cpp:3797
#21 0x00007f6b8fa7c501 in WebCore::RenderLayerBacking::paintIntoLayer (this=0x7f68dfd07240, graphicsLayer=0x7f68ea990800, context=..., paintDirtyRect=..., paintBehavior=..., regionContext=0x0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderLayerBacking.cpp:3817
#22 0x00007f6b8fa7cfd1 in WebCore::RenderLayerBacking::paintContents (this=0x7f68dfd07240, graphicsLayer=0x7f68ea990800, context=..., clip=..., layerPaintBehavior=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderLayerBacking.cpp:4126
#23 0x00007f6b8f713fff in WebCore::GraphicsLayer::paintGraphicsLayerContents (this=0x7f68ea990800, context=..., clip=..., layerPaintBehavior=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/GraphicsLayer.cpp:677
#24 0x00007f6b8f83979e in WebCore::SkiaPaintingEngine::paintIntoGraphicsContext (this=<optimized out>, layer=..., context=..., dirtyRect=..., contentsOpaque=<optimized out>, contentsScale=<error reading variable: Value cannot be represented as integer of 8 bytes.>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/skia/SkiaPaintingEngine.cpp:103
#25 0x00007f6b8f83a038 in WebCore::SkiaPaintingEngine::record (this=0x7f6b6a0f6d20, layer=..., recordRect=..., contentsOpaque=false, contentsScale=1) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/skia/SkiaPaintingEngine.cpp:249
#26 0x00007f6b8da7fa64 in WebCore::CoordinatedPlatformLayer::record (this=0x7f691eb400e0, recordRect=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/CoordinatedPlatformLayer.cpp:834
#27 0x00007f6b8da79ae0 in WebCore::CoordinatedBackingStoreProxy::updateIfNeeded (this=0x7f68e8fa40e0, unscaledVisibleRect=<optimized out>, unscaledContentsRect=<optimized out>, shouldCreateAndDestroyTiles=<optimized out>, dirtyRegion=<optimized out>, layer=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/CoordinatedBackingStoreProxy.cpp:172
#28 0x00007f6b8da7f4de in WebCore::CoordinatedPlatformLayer::updateBackingStore (this=0x7f691eb400e0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/CoordinatedPlatformLayer.cpp:722
#29 0x00007f6b8da88f48 in WebCore::GraphicsLayerCoordinated::updateBackingStoreIfNeeded (this=0x7f68ea990800) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1191
#30 WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f68ea990800) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1179
#31 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f68e32b2000) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#32 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f68e32b1400) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#33 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f68ea997400) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#34 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f69b574c000) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#35 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f69b575e000) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#36 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f69b5743000) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#37 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f69b5747800) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#38 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f69b5742400) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#39 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f69b574cc00) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#40 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f6b6b085400) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#41 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f6b6b275000) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#42 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f6b6b082400) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#43 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f6b6b084800) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#44 0x00007f6b8da88f8c in WebCore::GraphicsLayerCoordinated::updateBackingStoresIfNeeded (this=0x7f6b6b081800) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:1183
#45 0x00007f6b8da88b14 in WebCore::GraphicsLayerCoordinated::flushCompositingState (this=0x7f6b6b081800, visibleRect=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/texmap/coordinated/GraphicsLayerCoordinated.cpp:733
#46 0x00007f6b8fa84023 in WebCore::RenderLayerCompositor::flushPendingLayerChanges (this=0x7f6b6aec41a0, isFlushRoot=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/rendering/RenderLayerCompositor.cpp:841
#47 0x00007f6b8f4c09ab in WebCore::LocalFrameView::flushCompositingStateForThisFrame (this=0x7f6b6a124c40, rootFrameForFlush=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/page/LocalFrameView.cpp:934
#48 WebCore::LocalFrameView::flushCompositingStateIncludingSubframes (this=0x7f6b6a124c40) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/page/LocalFrameView.cpp:1138
#49 0x00007f6b8f4fccbd in WebCore::Page::finalizeRenderingUpdateForRootFrame (this=0x7f6b6a001a20, rootFrame=..., flags=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/page/Page.cpp:2389
#50 0x00007f6b8f4fcbe5 in WebCore::Page::finalizeRenderingUpdate (this=0x7f6b6a001a20, flags=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/page/Page.cpp:2369
#51 0x00007f6b8d9c7e06 in WebKit::WebPage::finalizeRenderingUpdate (this=0x7f6b6a0e0100, flags=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebKit/WebProcess/WebPage/WebPage.cpp:5125
#52 0x00007f6b8d9f556b in WebKit::LayerTreeHost::flushLayers (this=0x7f6b6a104100) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebKit/WebProcess/WebPage/CoordinatedGraphics/LayerTreeHost.cpp:199
#53 0x00007f6b8d9f6864 in WebKit::LayerTreeHost::didComposite (this=0x7f6b6a104100, compositionResponseID=69) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebKit/WebProcess/WebPage/CoordinatedGraphics/LayerTreeHost.cpp:526
#54 0x00007f6b8d9fbaa5 in WTF::RunLoop::Timer::Timer<WebKit::ThreadedCompositor>(WTF::Ref<WTF::RunLoop, WTF::RawPtrTraits<WTF::RunLoop>, WTF::DefaultRefDerefTraits<WTF::RunLoop> >&&, WebKit::ThreadedCompositor*, void (WebKit::ThreadedCompositor::*)()) requires WTF::HasRefPtrMemberFunctions<WebKit::ThreadedCompositor>::value::{lambda()#1}::operator()() const (this=<optimized out>) at WTF/Headers/wtf/RunLoop.h:189
#55 WTF::Detail::CallableWrapper<WTF::RunLoop::Timer::Timer<WebKit::ThreadedCompositor>(WTF::Ref<WTF::RunLoop, WTF::RawPtrTraits<WTF::RunLoop>, WTF::DefaultRefDerefTraits<WTF::RunLoop> >&&, WebKit::ThreadedCompositor*, void (WebKit::ThreadedCompositor::*)()) requires WTF::HasRefPtrMemberFunctions<WebKit::ThreadedCompositor>::value::{lambda()#1}, void>::call() (this=<optimized out>) at WTF/Headers/wtf/Function.h:53
#56 0x00007f6b8beb6a95 in WTF::RunLoop::TimerBase::TimerBase(WTF::Ref<WTF::RunLoop, WTF::RawPtrTraits<WTF::RunLoop>, WTF::DefaultRefDerefTraits<WTF::RunLoop> >&&)::$_0::operator()(void*) const (userData=0x7f6b6a02c960, this=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/glib/RunLoopGLib.cpp:177
#57 WTF::RunLoop::TimerBase::TimerBase(WTF::Ref<WTF::RunLoop, WTF::RawPtrTraits<WTF::RunLoop>, WTF::DefaultRefDerefTraits<WTF::RunLoop> >&&)::$_0::__invoke(void*) (userData=0x7f6b6a02c960) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/glib/RunLoopGLib.cpp:169
#58 0x00007f6b8beb5bb1 in WTF::RunLoop::$_0::operator() (source=0x558361874410, callback=0x7f6b8beb6a00 <WTF::RunLoop::TimerBase::TimerBase(WTF::Ref<WTF::RunLoop, WTF::RawPtrTraits<WTF::RunLoop>, WTF::DefaultRefDerefTraits<WTF::RunLoop> >&&)::$_0::__invoke(void*)>, userData=0x7f6b6a02c960, this=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/glib/RunLoopGLib.cpp:53
#59 WTF::RunLoop::$_0::__invoke (source=0x558361874410, callback=0x7f6b8beb6a00 <WTF::RunLoop::TimerBase::TimerBase(WTF::Ref<WTF::RunLoop, WTF::RawPtrTraits<WTF::RunLoop>, WTF::DefaultRefDerefTraits<WTF::RunLoop> >&&)::$_0::__invoke(void*)>, userData=0x7f6b6a02c960) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/glib/RunLoopGLib.cpp:45
#60 0x00007f6b87f044e7 in g_main_dispatch (context=context@entry=0x5583615d8540) at ../glib/gmain.c:3524
#61 0x00007f6b87f07817 in g_main_context_dispatch_unlocked (context=0x5583615d8540) at ../glib/gmain.c:4375
#62 g_main_context_iterate_unlocked (context=0x5583615d8540, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4440
#63 0x00007f6b87f08317 in g_main_loop_run (loop=0x55836161d290) at ../glib/gmain.c:4642
#64 0x00007f6b8beb608d in WTF::RunLoop::run () at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/glib/RunLoopGLib.cpp:108
#65 0x00007f6b8da06534 in WebKit::AuxiliaryProcessMainBase<WebKit::WebProcess, true>::run (this=0x7ffc161f3bd0, argc=<optimized out>, argv=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebKit/Shared/AuxiliaryProcessMain.h:77
#66 WebKit::AuxiliaryProcessMain<WebKit::WebProcessMainGtk> (argc=4, argv=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebKit/Shared/AuxiliaryProcessMain.h:103
#67 0x00007f6b8c62b3c8 in __libc_start_call_main (main=main@entry=0x558336378150 <main(int, char**)>, argc=argc@entry=4, argv=argv@entry=0x7ffc161f3d68) at ../sysdeps/nptl/libc_start_call_main.h:58
#68 0x00007f6b8c62b48b in __libc_start_main_impl (main=0x558336378150 <main(int, char**)>, argc=4, argv=0x7ffc161f3d68, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc161f3d58) at ../csu/libc-start.c:360
#69 0x0000558336378085 in _start () at ../sysdeps/x86_64/start.S:115

Thread 1 (Thread 0x7f69aeffd6c0 (LWP 35)):
#0  0x00007f6b90050149 in GrResourceCache::notifyARefCntReachedZero (this=0x5583619dfb50, resource=0x5583621893d0, removedRef=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceCache.cpp:387
#1  0x00007f6b900571d7 in GrIORef<GrGpuResource>::notifyWillBeZero (this=0x5583621893d8, removedRef=GrIORef<GrGpuResource>::LastRemovedRef::kMainRef) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrGpuResource.h:102
#2  GrIORef<GrGpuResource>::unref (this=0x5583621893d8) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrGpuResource.h:65
#3  SkSafeUnref<GrSurface> (obj=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:151
#4  sk_sp<GrSurface>::reset (this=0x558361fc4d0a, ptr=0x0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:316
#5  sk_sp<GrSurface>::operator=(decltype(nullptr)) (this=0x558361fc4d0a) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:260
#6  GrTextureProxy::~GrTextureProxy (this=0x558361fc4cf0, vtt=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrTextureProxy.cpp:110
#7  0x00007f6b9005868f in GrTextureRenderTargetProxy::~GrTextureRenderTargetProxy (this=0x558361fc4cc0, vtt=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrTextureRenderTargetProxy.h:44
#8  GrTextureRenderTargetProxy::~GrTextureRenderTargetProxy (this=0x558361fc4cc0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrTextureRenderTargetProxy.h:44
#9  GrTextureRenderTargetProxy::~GrTextureRenderTargetProxy (this=0x558361fc4cc0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/GrTextureRenderTargetProxy.h:44
#10 0x00007f6b9005868f in virtual thunk to GrTextureRenderTargetProxy::~GrTextureRenderTargetProxy() () at /usr/lib/x86_64-linux-gnu/libwebkitgtk-6.0.so.4
#11 0x00007f6b902d67ca in SkNVRefCnt<GrSurfaceProxy>::unref (this=0x5583621893d8) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:181
#12 SkSafeUnref<GrSurfaceProxy> (obj=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:151
#13 sk_sp<GrSurfaceProxy>::~sk_sp (this=0x55836206a620) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:256
#14 SkImage_Ganesh::ProxyChooser::~ProxyChooser (this=0x55836206a618) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/image/SkImage_Ganesh.cpp:74
#15 SkImage_Ganesh::~SkImage_Ganesh (this=0x55836206a5e0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/image/SkImage_Ganesh.cpp:237
#16 0x00007f6b902d6812 in SkImage_Ganesh::~SkImage_Ganesh (this=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/gpu/ganesh/image/SkImage_Ganesh.cpp:237
#17 0x00007f6b90115f2c in SkRefCntBase::unref (this=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:78
#18 SkSafeUnref<SkImage> (obj=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:151
#19 sk_sp<SkImage>::~sk_sp (this=0x558361b63f90) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:256
#20 SkImageShader::~SkImageShader (this=0x558361b63f80) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/shaders/SkImageShader.h:28
#21 SkImageShader::~SkImageShader (this=0x558361b63f80) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/shaders/SkImageShader.h:28
#22 0x00007f6b9030e7d2 in SkRefCntBase::unref (this=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:78
#23 SkSafeUnref<SkShader> (obj=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:151
#24 sk_sp<SkShader>::~sk_sp (this=0x5583626efb18) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:256
#25 SkLocalMatrixShader::~SkLocalMatrixShader (this=0x5583626efae0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/shaders/SkLocalMatrixShader.h:29
#26 SkLocalMatrixShader::~SkLocalMatrixShader (this=0x5583626efae0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/shaders/SkLocalMatrixShader.h:29
#27 0x00007f6b8ffe9656 in SkRefCntBase::unref (this=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:78
#28 SkSafeUnref<SkShader> (obj=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:151
#29 sk_sp<SkShader>::~sk_sp (this=0x5583624389c0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:256
#30 SkPaint::~SkPaint (this=0x5583624389b8) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/core/SkPaint.cpp:60
#31 0x00007f6b9000d1df in SkRecord::mutate<SkRecord::Destroyer&> (this=0x55836243cea0, i=2, f=...) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/core/SkRecord.h:55
#32 SkRecord::~SkRecord (this=0x55836243cea0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/core/SkRecord.cpp:15
#33 SkRecord::~SkRecord (this=0x55836243cea0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/core/SkRecord.cpp:12
#34 0x00007f6b9022d278 in SkRefCntBase::unref (this=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:78
#35 SkSafeUnref<SkRecord const> (obj=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:151
#36 sk_sp<SkRecord const>::~sk_sp (this=0x5583625b20e0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:256
#37 SkBigPicture::~SkBigPicture (this=0x5583625b20b0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/core/SkBigPicture.h:27
#38 SkBigPicture::~SkBigPicture (this=0x5583625b20b0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/src/core/SkBigPicture.h:27
#39 0x00007f6b8f83d28e in SkRefCntBase::unref (this=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:78
#40 SkSafeUnref<SkPicture> (obj=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:151
#41 sk_sp<SkPicture>::~sk_sp (this=0x7f691f26f150) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/ThirdParty/skia/include/core/SkRefCnt.h:256
#42 WebCore::SkiaRecordingResult::~SkiaRecordingResult (this=0x7f691f26f140) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/skia/SkiaRecordingResult.cpp:43
#43 WTF::ThreadSafeRefCounted<WebCore::SkiaRecordingResult, (WTF::DestructionThread)0>::deref (this=0x7f691f26f148) at WTF/Headers/wtf/ThreadSafeRefCounted.h:139
#44 WTF::DefaultRefDerefTraits<WebCore::SkiaRecordingResult>::derefIfNotNull (ptr=0x7f691f26f140) at WTF/Headers/wtf/Ref.h:62
#45 WTF::RefPtr<WebCore::SkiaRecordingResult, WTF::RawPtrTraits<WebCore::SkiaRecordingResult>, WTF::DefaultRefDerefTraits<WebCore::SkiaRecordingResult> >::~RefPtr (this=<optimized out>) at WTF/Headers/wtf/RefPtr.h:60
#46 WebCore::SkiaPaintingEngine::replay(WTF::RefPtr<WebCore::SkiaRecordingResult, WTF::RawPtrTraits<WebCore::SkiaRecordingResult>, WTF::DefaultRefDerefTraits<WebCore::SkiaRecordingResult> > const&, WebCore::IntRect const&)::$_0::~$_0() (this=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WebCore/platform/graphics/skia/SkiaPaintingEngine.cpp:270
#47 WTF::Detail::CallableWrapper<WebCore::SkiaPaintingEngine::replay(WTF::RefPtr<WebCore::SkiaRecordingResult, WTF::RawPtrTraits<WebCore::SkiaRecordingResult>, WTF::DefaultRefDerefTraits<WebCore::SkiaRecordingResult> > const&, WebCore::IntRect const&)::$_0, void>::~CallableWrapper (this=<optimized out>) at WTF/Headers/wtf/Function.h:47
#48 WTF::Detail::CallableWrapper<WebCore::SkiaPaintingEngine::replay(WTF::RefPtr<WebCore::SkiaRecordingResult, WTF::RawPtrTraits<WebCore::SkiaRecordingResult>, WTF::DefaultRefDerefTraits<WebCore::SkiaRecordingResult> > const&, WebCore::IntRect const&)::$_0, void>::~CallableWrapper (this=<optimized out>) at WTF/Headers/wtf/Function.h:47
#49 0x00007f6b8be8540e in std::default_delete<WTF::Detail::CallableWrapperBase<void> >::operator() (this=0x7f6b6a0fc338, __ptr=0x5583621893d0) at /usr/bin/../lib/gcc/x86_64-unknown-linux-gnu/14.3.0/../../../../include/c++/14.3.0/bits/unique_ptr.h:93
#50 std::__uniq_ptr_impl<WTF::Detail::CallableWrapperBase<void>, std::default_delete<WTF::Detail::CallableWrapperBase<void> > >::reset (this=0x7f6b6a0fc338, __p=0x0) at /usr/bin/../lib/gcc/x86_64-unknown-linux-gnu/14.3.0/../../../../include/c++/14.3.0/bits/unique_ptr.h:205
#51 std::unique_ptr<WTF::Detail::CallableWrapperBase<void>, std::default_delete<WTF::Detail::CallableWrapperBase<void> > >::reset (this=0x7f6b6a0fc338, __p=0x0) at /usr/bin/../lib/gcc/x86_64-unknown-linux-gnu/14.3.0/../../../../include/c++/14.3.0/bits/unique_ptr.h:504
#52 std::unique_ptr<WTF::Detail::CallableWrapperBase<void>, std::default_delete<WTF::Detail::CallableWrapperBase<void> > >::operator=(decltype(nullptr)) (this=0x7f6b6a0fc338) at /usr/bin/../lib/gcc/x86_64-unknown-linux-gnu/14.3.0/../../../../include/c++/14.3.0/bits/unique_ptr.h:437
#53 WTF::Function<void ()>::operator=(decltype(nullptr)) (this=0x7f6b6a0fc338) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/Function.h:103
#54 WTF::WorkerPool::Worker::work (this=0x7f6b6a0fc300) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/WorkerPool.cpp:54
#55 0x00007f6b8bdc9df4 in WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0::operator()() const (this=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/AutomaticThread.cpp:225
#56 WTF::Detail::CallableWrapper<WTF::AutomaticThread::start(WTF::AbstractLocker const&)::$_0, void>::call (this=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/Function.h:53
#57 0x00007f6b8be518e1 in WTF::Function<void()>::operator() (this=<optimized out>) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/Function.h:82
#58 WTF::Thread::entryPoint (newThreadContext=0x7f69b71fea20) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/Threading.cpp:268
#59 0x00007f6b8beba43d in WTF::wtfThreadEntryPoint (context=0x5583621893d0) at /buildstream/gnome/sdk/webkitgtk-6.0.bst/Source/WTF/wtf/posix/ThreadingPOSIX.cpp:245
#60 0x00007f6b8c699261 in start_thread (arg=<optimized out>) at pthread_create.c:447
#61 0x00007f6b8c71dd14 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:10

[Michael Catanzaro]

OK, https://www.irs.gov/pub/irs-pdf/f1040.pdf is a public reproducer. It crashes about 40-60% of the time. Simply load it a bunch of times in different tabs.

[Nikolas Zimmermann]

Thanks Michael, very valuable information - I'm glad you found a way to reproduce it, I will try to replicate your findings and inspect again the code for thread-safety issues.

[Nikolas Zimmermann]

It's a very important observation that this bug appears when the SkiaRecordingResult is destructed, while the main thread is recording.
This can only happen with multiple layers.

We usually do the following on the main thread:
1. Record the whole layer once, store result in SkiaRecordingResult
2. Replay parts of it ("tiles") in worker threads
3. Proceed to next layer, etc.

Your last backtrace shows that the SkiaRecordingResult is destructed from within the worker thread.
In main we already changed this to ensure the destruction of the SkiaRecordingResult also happens on the main thread.

I was trying: `Tools/Scripts/run-minibrowser --wpe --release 'https://www.irs.gov/pub/irs-pdf/f1040.pdf'` on main and got no crash after loading this N>10 times and scrolling heavily.

Once I reverted the fix in main:

```
-class SkiaRecordingResult final : public ThreadSafeRefCounted<SkiaRecordingResult, WTF::DestructionThread::Main> {
+class SkiaRecordingResult final : public ThreadSafeRefCounted<SkiaRecordingResult> {
```

crashes started to appear (still took N~5 loads until I could reproduce it, but clearly WebProcess crashed).

```
Jun 17 10:35:23 nzimmermann-thinkpad kernel: traps: SkiaGPUWorker[3340019] trap invalid opcode ip:7c727949f07e sp:7c709efff8e0 error:0 in libWPEWebKit-2.0.so.1.6.1[b09e07e,7c7272e68000+6b81000]
```

```
Thread 53 "SkiaGPUWorker" received signal SIGILL, Illegal instruction.
[Switching to Thread 0x7e10658006c0 (LWP 3342982)]
0x00007e10fbc9f07e in GrResourceCache::notifyARefCntReachedZero (this=0x5b6499490850, resource=0x5b6499d823d0, removedRef=<optimized out>)
    at /host/home/nzimmermann/Software/GitRepositories/WebKit/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceCache.cpp:387
387	}
(gdb) bt
#0  0x00007e10fbc9f07e in GrResourceCache::notifyARefCntReachedZero (this=0x5b6499490850, resource=0x5b6499d823d0, removedRef=<optimized out>)
    at /host/home/nzimmermann/Software/GitRepositories/WebKit/Source/ThirdParty/skia/src/gpu/ganesh/GrResourceCache.cpp:387
#1  0x00007e10fbca63b0 in GrIORef<GrGpuResource>::notifyWillBeZero (removedRef=GrIORef<GrGpuResource>::LastRemovedRef::kMainRef, this=<optimized out>)
    at /host/home/nzimmermann/Software/GitRepositories/WebKit/Source/ThirdParty/skia/src/gpu/ganesh/GrGpuResource.h:102
#2  GrIORef<GrGpuResource>::unref (this=0x5b6499d823d8) at /host/home/nzimmermann/Software/GitRepositories/WebKit/Source/ThirdParty/skia/src/gpu/ganesh/GrGpuResource.h:65
#3  SkSafeUnref<GrSurface> (obj=0x5b6499d823d0) at /host/home/nzimmermann/Software/GitRepositories/WebKit/Source/ThirdParty/skia/include/core/SkRefCnt.h:151
#4  sk_sp<GrSurface>::reset (this=0x5b6499db2985, ptr=0x0) at /host/home/nzimmermann/Software/GitRepositories/WebKit/Source/ThirdParty/skia/include/core/SkRefCnt.h:316
#5  sk_sp<GrSurface>::operator=(decltype(nullptr)) (this=0x5b6499db2985) at /host/home/nzimmermann/Software/GitRepositories/WebKit/Source/ThirdParty/skia/include/core/SkRefCnt.h:260
#6  GrTextureProxy::~GrTextureProxy (this=0x5b6499db2970, vtt=<optimized out>)
    at /host/home/nzimmermann/Software/GitRepositories/WebKit/Source/ThirdParty/skia/src/gpu/ganesh/GrTextureProxy.cpp:110
#7  0x00007e10fbca76a6 in GrTextureRenderTargetProxy::~GrTextureRenderTargetProxy (this=0x5b6499db2940, vtt=<optimized out>)
    at /host/home/nzimmermann/Software/GitRepositories/WebKit/Source/ThirdParty/skia/src/gpu/ganesh/GrTextureRenderTargetProxy.h:44
#8  GrTextureRenderTargetProxy::~GrTextureRenderTargetProxy (this=0x5b6499db2940)
    at /host/home/nzimmermann/Software/GitRepositories/WebKit/Source/ThirdParty/skia/src/gpu/ganesh/GrTextureRenderTargetProxy.h:44
#9  GrTextureRenderTargetProxy::~GrTextureRenderTargetProxy (this=0x5b6499db2940)
    at /host/home/nzimmermann/Software/GitRepositories/WebKit/Source/ThirdParty/skia/src/gpu/ganesh/GrTextureRenderTargetProxy.h:44
#10 0x00007e10fbca76a6 in virtual thunk to GrTextureRenderTargetProxy::~GrTextureRenderTargetProxy() ()
    at /host/home/nzimmermann/Software/GitRepositories/WebKit/WebKitBuild/WPE/Release/lib/libWPEWebKit-2.0.so.1
#11 0x00007e10fbd03891 in SkNVRefCnt<GrSurfaceProxy>::unref (this=0x5b6499d823d8)
```

So I think this is finally fixed in main.

[Nikolas Zimmermann]

I requested a backport of the fix in main to the 2.48 series, so that you can test again and confirm the crash is gone.

[Michael Catanzaro]

Actually I'm testing with Epiphany Tech Preview, using WebKitGTK 2.49.2. I see the fix 295939@main is more recent than 2.49.2, so this sounds plausible to me. Let's close this. Thank you!

(Backport to 2.48 will still be helpful, though, since users are reporting the same problem in 2.48.)

[Nikolas Zimmermann]

Excellent, thanks Michael as well :-)