227952 2021-07-14 07:37:51 -0700 Network process memory corruption 2021-08-30 08:10:48 -0700 1 1 1 Unclassified WebKit WebKitGTK WebKit Nightly Build PC Linux RESOLVED WORKSFORME P2 Normal --- 1 mcatanzaro webkit-unassigned bugs-noreply mcatanzaro webkit-bug-importer oldest_to_newest 1776928 0 mcatanzaro 2021-07-14 07:37:51 -0700 (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50 #1 0x00007fe6f7fde855 in __GI_abort () at abort.c:79 #2 0x00007fe6f80392f7 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7fe6f814ae35 "%s\n") at ../sysdeps/posix/libc_fatal.c:155 #3 0x00007fe6f804081c in malloc_printerr (str=str@entry=0x7fe6f8149024 "corrupted size vs. prev_size") at malloc.c:5347 #4 0x00007fe6f8041576 in unlink_chunk (p=p@entry=0x55f4877c8ee0, av=0x7fe6f81819e0 <main_arena>) at malloc.c:1454 #5 0x00007fe6f8041d4b in _int_free (av=0x7fe6f81819e0 <main_arena>, p=0x55f4877c8e40, have_lock=<optimized out>) at malloc.c:4342 #6 0x00007fe6f472da2c in _asn1_delete_structure (e_list=e_list@entry=0x0, structure=structure@entry=0x55f4877f1aa0, flags=flags@entry=0) at structure.c:361 #7 0x00007fe6f472dd80 in asn1_delete_structure (structure=structure@entry=0x55f4877f1aa0) at structure.c:296 #8 0x00007fe68a51344d in gnutls_x509_crt_deinit (cert=0x55f4877f1aa0) at ../../../lib/x509/x509.c:297 #9 0x00007fe6b066df16 in g_tls_certificate_gnutls_finalize (object=0x55f487ba3b00 [GTlsCertificateGnutls]) at ../tls/gnutls/gtlscertificate-gnutls.c:82 #10 0x00007fe6f7c795e2 in g_object_unref (_object=<optimized out>) at ../gobject/gobject.c:3581 #11 g_object_unref (_object=0x55f487ba3b00) at ../gobject/gobject.c:3473 #12 0x00007fe6b066df6e in g_tls_certificate_gnutls_finalize (object=0x7fe690002000 [GTlsCertificateGnutls]) at ../tls/gnutls/gtlscertificate-gnutls.c:88 #13 0x00007fe6f7c795e2 in g_object_unref (_object=<optimized out>) at ../gobject/gobject.c:3581 #14 g_object_unref (_object=0x7fe690002000) at ../gobject/gobject.c:3473 #15 0x00007fe6b067729c in g_tls_connection_base_finalize (object=0x55f4877e7850 [GTlsClientConnectionGnutls]) at ../tls/base/gtlsconnection-base.c:262 #16 0x00007fe6f7c795e2 in g_object_unref (_object=<optimized out>) at ../gobject/gobject.c:3581 #17 g_object_unref (_object=0x55f4877e7850) at ../gobject/gobject.c:3473 #18 0x00007fe6f483577b in soup_io_stream_finalize (object=0x55f487a98b40 [SoupIOStream]) at ../libsoup/soup-io-stream.c:114 #19 0x00007fe6f7c795e2 in g_object_unref (_object=<optimized out>) at ../gobject/gobject.c:3581 #20 g_object_unref (_object=0x55f487a98b40) at ../gobject/gobject.c:3473 #21 0x00007fe6f482fc6f in soup_connection_finalize (object=0x55f487af11f0 [SoupConnection]) at ../libsoup/soup-connection.c:121 #22 0x00007fe6f7c795e2 in g_object_unref (_object=<optimized out>) at ../gobject/gobject.c:3581 #23 g_object_unref (_object=0x55f487af11f0) at ../gobject/gobject.c:3473 #24 0x00007fe6f7d7ab46 in g_task_finalize (object=0x7fe6900093d0 [GTask]) at ../gio/gtask.c:655 #25 0x00007fe6f7c795e2 in g_object_unref (_object=<optimized out>) at ../gobject/gobject.c:3581 #26 g_object_unref (_object=0x7fe6900093d0) at ../gobject/gobject.c:3473 #27 0x00007fe6f7b78583 in g_source_callback_unref (cb_data=0x55f487993750) at ../glib/gmain.c:1664 #28 g_source_callback_unref (cb_data=0x55f487993750) at ../glib/gmain.c:1657 #29 0x00007fe6f7b78ab9 in g_source_destroy_internal (source=0x55f4876cd600, context=0x55f4872dc190, have_lock=1) at ../glib/gmain.c:1329 #30 0x00007fe6f7b7c348 in g_main_dispatch (context=0x55f4872dc190) at ../glib/gmain.c:3374 #31 g_main_context_dispatch (context=0x55f4872dc190) at ../glib/gmain.c:4062 #32 0x00007fe6f7b7c668 in g_main_context_iterate (context=0x55f4872dc190, block=block@entry=1, dispatch=dispatch@entry=1, self=<optimized out>) at ../glib/gmain.c:4138 #33 0x00007fe6f7b7c983 in g_main_loop_run (loop=loop@entry=0x55f4872dd340) at ../glib/gmain.c:4336 #34 0x00007fe6f777edd0 in WTF::RunLoop::run() () at ../Source/WTF/wtf/glib/RunLoopGLib.cpp:108 #35 0x00007fe6f8abb662 in WebKit::AuxiliaryProcessMainBase<WebKit::NetworkProcess, false>::run(int, char**) (argc=3, argv=0x7ffc417f6fd8, this=0x7ffc417f6e60) at ../Source/WebKit/Shared/AuxiliaryProcessMain.h:57 #36 WebKit::AuxiliaryProcessMainBase<WebKit::NetworkProcess, false>::run(int, char**) (argv=0x7ffc417f6fd8, argc=3, this=0x7ffc417f6e60) at ../Source/WebKit/Shared/AuxiliaryProcessMain.h:57 #37 WebKit::AuxiliaryProcessMain<WebKit::NetworkProcessMainSoup>(int, char**) (argc=3, argv=0x7ffc417f6fd8) at ../Source/WebKit/Shared/AuxiliaryProcessMain.h:96 #38 0x00007fe6f7fe0062 in __libc_start_main (main= 0x55f485fc56c0 <main(int, char**)>, argc=3, argv=0x7ffc417f6fd8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc417f6fc8) at ../csu/libc-start.c:308 #39 0x000055f485fc56fe in _start () at ../sysdeps/x86_64/start.S:120 We have introduced some network process memory corruption either (a) in WebKit, sometime since 2.32, or (b) in libsoup 3. One or the other, I'm not sure which. Well, it could also be glib-networking, or anything really. Who knows. Since this is memory corruption, the backtrace is likely not useful. The actual problem could be anywhere. We probably won't be able to fix it unless we can catch it under valgrind or asan. Sadly, running the network process under either seems pretty difficult.... 1788501 1 mcatanzaro 2021-08-30 08:10:48 -0700 Haven't seen this in a while. My guess is this got quietly fixed in libsoup.