Bug 28148 – Requests through the SOUP layer leaks memory

• Status: RESOLVED
• Resolution: FIXED
• Component: WebKitGTK

[John Kjellberg]

Created attachment 34451 [details]
Code making one XMLHttpRequest every 0.1s

The attached test case makes one XMLHttpRequest() call each 0.1s. As times goes by it uses more and more memory. This will most likely also affect any type of HTTP request made.

I have found a code change that will stop the leak. However, that isn't really a solution, but it might be a hint to someone that knows the code better. I based the change on how it worked in earlier revisions of the code that didn't have this leak.
(Why the "m_isWebSecurityEnabled" change is needed I have no idea. Only found it by a lucky coincident)

[John Kjellberg]

Created attachment 34458 [details]
Code that seems to remove the leak

Hack that gives an example of code that fixes the leak in the wrong way. Only meant as a hint.

[John Kjellberg]

As an additional note. Revision 41128 did not have this specific leak(but it did have the https://bugs.webkit.org/show_bug.cgi?id=26716 leak).

[Priit Laes (IRC: plaes)]

(In reply to comment #0)
> Created an attachment (id=34451) [details]
> Code making one XMLHttpRequest every 0.1s
> 
> The attached test case makes one XMLHttpRequest() call each 0.1s. As times goes
> by it uses more and more memory. This will most likely also affect any type of
> HTTP request made.
> 
> I have found a code change that will stop the leak. However, that isn't really
> a solution, but it might be a hint to someone that knows the code better. I
> based the change on how it worked in earlier revisions of the code that didn't
> have this leak.
> (Why the "m_isWebSecurityEnabled" change is needed I have no idea. Only found
> it by a lucky coincident)

Hmm.. this is actually causing crash when closing the page after a few requests... :S

[John Kjellberg]

(In reply to comment #3)
> (In reply to comment #0)
> > Created an attachment (id=34451) [details] [details]
> > Code making one XMLHttpRequest every 0.1s
> > 
> > The attached test case makes one XMLHttpRequest() call each 0.1s. As times goes
> > by it uses more and more memory. This will most likely also affect any type of
> > HTTP request made.
> > 
> > I have found a code change that will stop the leak. However, that isn't really
> > a solution, but it might be a hint to someone that knows the code better. I
> > based the change on how it worked in earlier revisions of the code that didn't
> > have this leak.
> > (Why the "m_isWebSecurityEnabled" change is needed I have no idea. Only found
> > it by a lucky coincident)
> 
> Hmm.. this is actually causing crash when closing the page after a few
> requests... :S

That's interesting. I have been running it for quite long time and then closed the whole GtkLauncher window without any indication of a crash.

[Priit Laes (IRC: plaes)]

This happens in epiphany when I have at least two tabs open.

1) open the example code in another tab ie. by clicking the middle button.
2) close the tab
3) Wait for crash

PS. And sorry for hijacking your bug report ;)

Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f561a467760 (LWP 13232)]
0x00007f5618735d99 in WebCore::FrameLoader::loadResourceSynchronously (this=0x7f560115b850, request=<value optimized out>, 
    storedCredentials=WebCore::AllowStoredCredentials, error=@0x7fff47a2f930, response=@0x7fff47a2f870, data=@0x7fff47a2f960)
    at WebCore/loader/FrameLoader.cpp:3728
3728	            documentLoader()->applicationCacheHost()->maybeLoadFallbackSynchronously(newRequest, error, response, data);
(gdb) bt
#0  0x00007f5618735d99 in WebCore::FrameLoader::loadResourceSynchronously (this=0x7f560115b850, request=<value optimized out>, 
    storedCredentials=WebCore::AllowStoredCredentials, error=@0x7fff47a2f930, response=@0x7fff47a2f870, data=@0x7fff47a2f960)
    at WebCore/loader/FrameLoader.cpp:3728
#1  0x00007f5618719890 in WebCore::DocumentThreadableLoader::loadResourceSynchronously (document=0x7f560110f800, request=@0x7fff47a2f9f0, client=@0x7f5601087d90, 
    storedCredentials=WebCore::AllowStoredCredentials) at WebCore/loader/DocumentThreadableLoader.cpp:55
#2  0x00007f56188d19ea in WebCore::XMLHttpRequest::loadRequestSynchronously (this=0x7f5601087d80, request=@0x7fff47a2f9f0, ec=@0x7fff47a2fc2c)
    at WebCore/xml/XMLHttpRequest.cpp:663
#3  0x00007f56188d4e1d in WebCore::XMLHttpRequest::makeSameOriginRequest (this=0x7f5601087d80, ec=@0x7fff47a2fc2c) at WebCore/xml/XMLHttpRequest.cpp:510
#4  0x00007f56188d7c33 in WebCore::XMLHttpRequest::send (this=0x7f5601087d80, body=@0x7fff47a2fbb0, ec=@0x7fff47a2fc2c) at WebCore/xml/XMLHttpRequest.cpp:446
#5  0x00007f56188d7e0b in WebCore::XMLHttpRequest::send (this=0x7f560cfc2e60, ec=@0x0) at WebCore/xml/XMLHttpRequest.cpp:389
#6  0x00007f56184fbda6 in WebCore::JSXMLHttpRequest::send (this=0x7f5600ec7600, exec=0x7f56012900a0, args=@0x7fff47a2f930)
    at WebCore/bindings/js/JSXMLHttpRequestCustom.cpp:125
#7  0x00007f5618bab0d2 in WebCore::jsXMLHttpRequestPrototypeFunctionSend (exec=0x7f56012900a0, thisValue={m_ptr = 0x7f5600ec7600}, args=@0x7fff47a2fc80)
    at DerivedSources/JSXMLHttpRequest.cpp:373
#8  0x00007f561a5a21d4 in ?? ()
#9  0x00007f5601290058 in ?? ()
#10 0x0000000000000001 in ?? ()
#11 0x000000000188fac4 in ?? ()
#12 0x00007f5600ec9cc0 in ?? ()
#13 0x00007f560107cb20 in ?? ()
#14 0x0000000000000000 in ?? ()
(gdb) bt full
#0  0x00007f5618735d99 in WebCore::FrameLoader::loadResourceSynchronously (this=0x7f560115b850, request=<value optimized out>, 
    storedCredentials=WebCore::AllowStoredCredentials, error=@0x7fff47a2f930, response=@0x7fff47a2f870, data=@0x7fff47a2f960)
    at WebCore/loader/FrameLoader.cpp:3728
	referrer = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x7f56010fdea0}}
	initialRequest = {<WebCore::ResourceRequestBase> = {m_url = {m_string = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x7f5600f37380}}, 
      m_isValid = true, m_protocolInHTTPFamily = true, m_schemeEnd = 5, m_userStart = 8, m_userEnd = 8, m_passwordEnd = 8, m_hostEnd = 23, m_portEnd = 23, 
      m_pathAfterLastSlash = 24, m_pathEnd = 36, m_queryEnd = 36, m_fragmentEnd = 36}, m_cachePolicy = WebCore::UseProtocolCachePolicy, m_timeoutInterval = 10, 
    m_firstPartyForCookies = {m_string = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x7f5601206a28}}, m_isValid = true, 
      m_protocolInHTTPFamily = true, m_schemeEnd = 5, m_userStart = 8, m_userEnd = 8, m_passwordEnd = 8, m_hostEnd = 23, m_portEnd = 23, 
      m_pathAfterLastSlash = 24, m_pathEnd = 38, m_queryEnd = 47, m_fragmentEnd = 47}, m_httpMethod = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, 
        m_ptr = 0x7f5602740000}}, 
    m_httpHeaderFields = {<WTF::HashMap<WebCore::AtomicString, WebCore::String, WebCore::CaseFoldingHash, WTF::HashTraits<WebCore::AtomicString>, WTF::HashTraits<WebCore::String> >> = {<WTF::FastAllocBase> = {<No data fields>}, m_impl = {static m_minTableSize = 64, static m_maxLoad = <optimized out>, static m_minLoad = 6, 
          m_table = 0x7f56010c4400, m_tableSize = 64, m_tableSizeMask = 63, m_keyCount = 2, m_deletedCount = 0}}, <No data fields>}, 
    m_responseContentDispositionEncodingFallbackArray = {<WTF::FastAllocBase> = {<No data fields>}, m_size = 0, 
      m_buffer = {<WTF::VectorBufferBase<WebCore::String>> = {<WTFNoncopyable::Noncopyable> = {<WTF::FastAllocBase> = {<No data fields>}, <No data fields>}, 
          m_buffer = 0x7f560112c320, m_capacity = 0}, <No data fields>}}, m_httpBody = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x0}, 
    m_allowHTTPCookies = true, m_resourceRequestUpdated = true, m_platformRequestUpdated = false, m_reportUploadProgress = false}, <No data fields>}
	identifier = 11
	newRequest = {<WebCore::ResourceRequestBase> = {m_url = {m_string = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x7f5600f37380}}, 
      m_isValid = true, m_protocolInHTTPFamily = true, m_schemeEnd = 5, m_userStart = 8, m_userEnd = 8, m_passwordEnd = 8, m_hostEnd = 23, m_portEnd = 23, 
      m_pathAfterLastSlash = 24, m_pathEnd = 36, m_queryEnd = 36, m_fragmentEnd = 36}, m_cachePolicy = WebCore::UseProtocolCachePolicy, m_timeoutInterval = 10, 
    m_firstPartyForCookies = {m_string = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x7f5601206a28}}, m_isValid = true, 
      m_protocolInHTTPFamily = true, m_schemeEnd = 5, m_userStart = 8, m_userEnd = 8, m_passwordEnd = 8, m_hostEnd = 23, m_portEnd = 23, 
      m_pathAfterLastSlash = 24, m_pathEnd = 38, m_queryEnd = 47, m_fragmentEnd = 47}, m_httpMethod = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, 
        m_ptr = 0x7f5602740000}}, 
    m_httpHeaderFields = {<WTF::HashMap<WebCore::AtomicString, WebCore::String, WebCore::CaseFoldingHash, WTF::HashTraits<WebCore::AtomicString>, WTF::HashTraits<WebCore::String> >> = {<WTF::FastAllocBase> = {<No data fields>}, m_impl = {static m_minTableSize = 64, static m_maxLoad = <optimized out>, static m_minLoad = 6, 
          m_table = 0x7f56011c6800, m_tableSize = 64, m_tableSizeMask = 63, m_keyCount = 2, m_deletedCount = 0}}, <No data fields>}, 
    m_responseContentDispositionEncodingFallbackArray = {<WTF::FastAllocBase> = {<No data fields>}, m_size = 0, 
      m_buffer = {<WTF::VectorBufferBase<WebCore::String>> = {<WTFNoncopyable::Noncopyable> = {<WTF::FastAllocBase> = {<No data fields>}, <No data fields>}, 
          m_buffer = 0x7f560112c1c8, m_capacity = 0}, <No data fields>}}, m_httpBody = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x0}, 
    m_allowHTTPCookies = true, m_resourceRequestUpdated = true, m_platformRequestUpdated = false, m_reportUploadProgress = false}, <No data fields>}
#1  0x00007f5618719890 in WebCore::DocumentThreadableLoader::loadResourceSynchronously (document=0x7f560110f800, request=@0x7fff47a2f9f0, client=@0x7f5601087d90, 
    storedCredentials=WebCore::AllowStoredCredentials) at WebCore/loader/DocumentThreadableLoader.cpp:55
	sameOriginRequest = true
	data = {<WTF::FastAllocBase> = {<No data fields>}, m_size = 210, 
  m_buffer = {<WTF::VectorBufferBase<char>> = {<WTFNoncopyable::Noncopyable> = {<WTF::FastAllocBase> = {<No data fields>}, <No data fields>}, 
      m_buffer = 0x7f56010f3460 "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p>The requested URL /mem_test.htm was not found on this server.</p>\n</bod"..., m_capacity = 210}, <No data fields>}}
	error = {<WebCore::ResourceErrorBase> = {m_domain = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x0}}, m_errorCode = 0, m_failingURL = {
      m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x0}}, m_localizedDescription = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, 
        m_ptr = 0x0}}, m_isNull = true, m_isCancellation = false}, <No data fields>}
	response = {<WebCore::ResourceResponseBase> = {m_url = {m_string = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x7f5600f37460}}, 
      m_isValid = true, m_protocolInHTTPFamily = true, m_schemeEnd = 5, m_userStart = 8, m_userEnd = 8, m_passwordEnd = 8, m_hostEnd = 23, m_portEnd = 23, 
      m_pathAfterLastSlash = 24, m_pathEnd = 36, m_queryEnd = 36, m_fragmentEnd = 36}, m_mimeType = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, 
        m_ptr = 0x7f5601249cc0}}, m_expectedContentLength = 210, m_textEncodingName = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, 
        m_ptr = 0x7f5601250580}}, m_suggestedFilename = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x0}}, m_httpStatusCode = 404, 
    m_httpStatusText = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x7f5601249c80}}, 
    m_httpHeaderFields = {<WTF::HashMap<WebCore::AtomicString, WebCore::String, WebCore::CaseFoldingHash, WTF::HashTraits<WebCore::AtomicString>, WTF::HashTraits<WebCore::String> >> = {<WTF::FastAllocBase> = {<No data fields>}, m_impl = {static m_minTableSize = 64, static m_maxLoad = <optimized out>, static m_minLoad = 6, 
          m_table = 0x7f56010a9400, m_tableSize = 64, m_tableSizeMask = 63, m_keyCount = 6, m_deletedCount = 0}}, <No data fields>}, m_lastModifiedDate = 0, 
    m_isNull = false, m_haveParsedCacheControlHeader = false, m_haveParsedAgeHeader = false, m_haveParsedDateHeader = false, m_haveParsedExpiresHeader = false, 
    m_haveParsedLastModifiedHeader = false, m_cacheControlContainsNoCache = false, m_cacheControlContainsNoStore = false, 
    m_cacheControlContainsMustRevalidate = false, m_cacheControlMaxAge = 0, m_age = 0, m_date = 0, m_expires = 0, m_lastModified = 0}, <No data fields>}
	identifier = 18446744073709551615
#2  0x00007f56188d19ea in WebCore::XMLHttpRequest::loadRequestSynchronously (this=0x7f5601087d80, request=@0x7fff47a2f9f0, ec=@0x7fff47a2fc2c)
    at WebCore/xml/XMLHttpRequest.cpp:663
No locals.
#3  0x00007f56188d4e1d in WebCore::XMLHttpRequest::makeSameOriginRequest (this=0x7f5601087d80, ec=@0x7fff47a2fc2c) at WebCore/xml/XMLHttpRequest.cpp:510
	request = {<WebCore::ResourceRequestBase> = {m_url = {m_string = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x7f5600f37380}}, 
      m_isValid = true, m_protocolInHTTPFamily = true, m_schemeEnd = 5, m_userStart = 8, m_userEnd = 8, m_passwordEnd = 8, m_hostEnd = 23, m_portEnd = 23, 
      m_pathAfterLastSlash = 24, m_pathEnd = 36, m_queryEnd = 36, m_fragmentEnd = 36}, m_cachePolicy = WebCore::UseProtocolCachePolicy, 
    m_timeoutInterval = 2147483647, m_firstPartyForCookies = {m_string = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x0}}, m_isValid = false, 
      m_protocolInHTTPFamily = false, m_schemeEnd = 0, m_userStart = 0, m_userEnd = 0, m_passwordEnd = 0, m_hostEnd = 0, m_portEnd = 0, m_pathAfterLastSlash = 0, 
      m_pathEnd = 0, m_queryEnd = 0, m_fragmentEnd = 0}, m_httpMethod = {m_impl = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x7f5602740000}}, 
    m_httpHeaderFields = {<WTF::HashMap<WebCore::AtomicString, WebCore::String, WebCore::CaseFoldingHash, WTF::HashTraits<WebCore::AtomicString>, WTF::HashTraits<WebCore::String> >> = {<WTF::FastAllocBase> = {<No data fields>}, m_impl = {static m_minTableSize = 64, static m_maxLoad = <optimized out>, static m_minLoad = 6, 
          m_table = 0x0, m_tableSize = 0, m_tableSizeMask = 0, m_keyCount = 0, m_deletedCount = 0}}, <No data fields>}, 
    m_responseContentDispositionEncodingFallbackArray = {<WTF::FastAllocBase> = {<No data fields>}, m_size = 0, 
      m_buffer = {<WTF::VectorBufferBase<WebCore::String>> = {<WTFNoncopyable::Noncopyable> = {<WTF::FastAllocBase> = {<No data fields>}, <No data fields>}, 
          m_buffer = 0x0, m_capacity = 0}, <No data fields>}}, m_httpBody = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x0}, m_allowHTTPCookies = true, 
    m_resourceRequestUpdated = true, m_platformRequestUpdated = false, m_reportUploadProgress = false}, <No data fields>}
#4  0x00007f56188d7c33 in WebCore::XMLHttpRequest::send (this=0x7f5601087d80, body=@0x7fff47a2fbb0, ec=@0x7fff47a2fc2c) at WebCore/xml/XMLHttpRequest.cpp:446
No locals.
#5  0x00007f56188d7e0b in WebCore::XMLHttpRequest::send (this=0x7f560cfc2e60, ec=@0x0) at WebCore/xml/XMLHttpRequest.cpp:389
No locals.
#6  0x00007f56184fbda6 in WebCore::JSXMLHttpRequest::send (this=0x7f5600ec7600, exec=0x7f56012900a0, args=@0x7fff47a2f930)
    at WebCore/bindings/js/JSXMLHttpRequestCustom.cpp:125
	ec = 0
	signedLineNumber = <value optimized out>
	sourceID = 140734395251744
	sourceURL = {m_rep = {<WTF::FastAllocBase> = {<No data fields>}, m_ptr = 0x7f5600f6baa0}, static nullUString = 0x7f56027330f8}
	function = {m_ptr = 0x7f560113d7e0}
#7  0x00007f5618bab0d2 in WebCore::jsXMLHttpRequestPrototypeFunctionSend (exec=0x7f56012900a0, thisValue={m_ptr = 0x7f5600ec7600}, args=@0x7fff47a2fc80)
    at DerivedSources/JSXMLHttpRequest.cpp:373
No locals.
#8  0x00007f561a5a21d4 in ?? ()

[Xan Lopez]

I have opened bug #28250 for the crasher issue. About the leak, I'm not sure the part of the patch that deals with the soup session makes much sense; is that needed to make the leak go away? Or is it just the security preference?

[John Kjellberg]

(In reply to comment #6)
> I have opened bug #28250 for the crasher issue. About the leak, I'm not sure
> the part of the patch that deals with the soup session makes much sense; is
> that needed to make the leak go away? Or is it just the security preference?

The security preference part I have know idea, it's really strange that it is needed at all. In r41128 there where no leak and the security fix wasn't needed(I think). The change is that before(in r11428 and my patch) there was one global "SoupSession" instead of creating one new for each startHttp() call.

I would guess that "handle->defaultSession()" which creates a new SoupSession must be followed by a "free" of some kind. But I'm not familiar enough with the code and code flow to know where that should be.

[Xan Lopez]

(In reply to comment #7)
> I would guess that "handle->defaultSession()" which creates a new SoupSession
> must be followed by a "free" of some kind. But I'm not familiar enough with the
> code and code flow to know where that should be.

Well, but unless I'm mistaken we don't do that. The session we return is a static variable, so it's only created once.

[John Kjellberg]

(In reply to comment #8)
> (In reply to comment #7)
> > I would guess that "handle->defaultSession()" which creates a new SoupSession
> > must be followed by a "free" of some kind. But I'm not familiar enough with the
> > code and code flow to know where that should be.
> 
> Well, but unless I'm mistaken we don't do that. The session we return is a
> static variable, so it's only created once.

Ah sorry. A missed the little detail about the "static" keyword there. :/

I don't really see the reason for that code to fix anything either. But I have tested with and without it multiple times. Without the change to "ResourceHandleSoup.cpp" it leaks, with it it doesn't.

BTW I'm testing leak with "cat /proc/<pid>/smaps" and then diff the output. The heap is building quite fast.

[Xan Lopez]

(In reply to comment #9)
> Ah sorry. A missed the little detail about the "static" keyword there. :/
> 
> I don't really see the reason for that code to fix anything either. But I have
> tested with and without it multiple times. Without the change to
> "ResourceHandleSoup.cpp" it leaks, with it it doesn't.
> 
> BTW I'm testing leak with "cat /proc/<pid>/smaps" and then diff the output. The
> heap is building quite fast.

Well, I don't understant what could happen then. If the leak was in the security code it would make sense, but I don't see how the patch for the ResourceHandleSoup file could affect anything. Could you pass webkit through valgrind without any of the patches and see what it says?

[John Kjellberg]

(In reply to comment #10)
> (In reply to comment #9)
> > Ah sorry. A missed the little detail about the "static" keyword there. :/
> > 
> > I don't really see the reason for that code to fix anything either. But I have
> > tested with and without it multiple times. Without the change to
> > "ResourceHandleSoup.cpp" it leaks, with it it doesn't.
> > 
> > BTW I'm testing leak with "cat /proc/<pid>/smaps" and then diff the output. The
> > heap is building quite fast.
> 
> Well, I don't understant what could happen then. If the leak was in the
> security code it would make sense, but I don't see how the patch for the
> ResourceHandleSoup file could affect anything. Could you pass webkit through
> valgrind without any of the patches and see what it says?

I have now run valgrid for about 15h and the biggest leak it reports is related to libsoup. See attachment.

[John Kjellberg]

Created attachment 34816 [details]
Valgrind report after running about 15h with flag --leak-check=full

[Dan Winship]

(In reply to comment #12)
> Created an attachment (id=34816) [details]
> Valgrind report after running about 15h with flag --leak-check=full

Hm... any chance you could try that again after installing libsoup debugging symbols, and adding "--num-callers=40" to the valgrind command line?

I tried running just against the page you attached, but didn't see the leak; although maybe the leak doesn't show up in that case because the xmlhttprequest is getting a 404 since the page it's requesting doesn't exist on bugs.webkit.org?

[John Kjellberg]

(In reply to comment #13)
> (In reply to comment #12)
> > Created an attachment (id=34816) [details] [details]
> > Valgrind report after running about 15h with flag --leak-check=full
> 
> Hm... any chance you could try that again after installing libsoup debugging
> symbols, and adding "--num-callers=40" to the valgrind command line?
> 
> I tried running just against the page you attached, but didn't see the leak;
> although maybe the leak doesn't show up in that case because the xmlhttprequest
> is getting a 404 since the page it's requesting doesn't exist on
> bugs.webkit.org?

Sorry I haven't hade the time to come back to you sooner.

The script actually calls itself, but I guess Bugzilla renames the attachment so that won't work.

I have now run it with the parameter as you said. Not sure if it contained any more useful information though.

[John Kjellberg]

Created attachment 38913 [details]
Valgrind report with flags --leak-check=full --num-callers=40

[Dan Winship]

doh, sorry

==13071== 49,195,760 bytes in 198,162 blocks are possibly lost in loss record 168 of 169
==13071==    at 0x4024362: memalign (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==13071==    by 0x40243FF: posix_memalign (in /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
==13071==    by 0x518D823: (within /usr/lib/libglib-2.0.so.0.1800.4)
==13071==    by 0x518F290: g_slice_alloc (in /usr/lib/libglib-2.0.so.0.1800.4)

You need to set G_SLICE=always-malloc in the environment if you are going to use valgrind. Otherwise, valgrind thinks the memory in use by the glib slice allocator is "possibly lost", even though glib is actually still keeping track of it.

Pretty sure this is INVALID (but I don't have bits to close it).

[Xan Lopez]

(In reply to comment #16)
> doh, sorry
> 
> ==13071== 49,195,760 bytes in 198,162 blocks are possibly lost in loss record
> 168 of 169
> ==13071==    at 0x4024362: memalign (in
> /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
> ==13071==    by 0x40243FF: posix_memalign (in
> /usr/lib/valgrind/x86-linux/vgpreload_memcheck.so)
> ==13071==    by 0x518D823: (within /usr/lib/libglib-2.0.so.0.1800.4)
> ==13071==    by 0x518F290: g_slice_alloc (in /usr/lib/libglib-2.0.so.0.1800.4)
> 
> You need to set G_SLICE=always-malloc in the environment if you are going to
> use valgrind. Otherwise, valgrind thinks the memory in use by the glib slice
> allocator is "possibly lost", even though glib is actually still keeping track
> of it.
> 
> Pretty sure this is INVALID (but I don't have bits to close it).

Doh, forgot about that too. Closing for now, please reopen in case the leak turns out to be real.

[John Kjellberg]

I run valgrind with env-variable "G_SLICE=always-malloc" and it still reports leaked memory. And even if it didn't there would still be a leak since the memory usage increases over time and nothing indicates that it is a cache being filled.

[John Kjellberg]

Created attachment 38986 [details]
Valgrind report with flags --leak-check=full --num-callers=40 and G_SLICE="always-malloc"

[Dan Winship]

weird. I don't see that leak.

What exact versions of webkit and libsoup are you using? Any non-standard configure options or environment variables? What browser are you using? Any extensions? Are you using a proxy?

[John Kjellberg]

(In reply to comment #20)
> weird. I don't see that leak.
> 
> What exact versions of webkit and libsoup are you using? Any non-standard
> configure options or environment variables? What browser are you using? Any
> extensions? Are you using a proxy?

I'm using revision 47028 from the subversion server. Libsoup version 2.27.4. The test file is called "mem_test.htm" and is in the document of a root of a cherokee webserver version 0.99.9.
I'm using the GtkLauncher from the same revision as WebKit.
All running on a local computer.
I'm quite sure everything i standard stuff.

[John Kjellberg]

I now have more information about this bug. The reason that the hack patch removed the leak was that it would then create two different "SoupSession" objects. And the one used by the "startHttp()" didn't have the "sniffer" functionality!

What I did was to revert back to the un-patched code. Then in the file WebKit/gtk/webkit/webkitprivate.cpp I commented out the last lines in that source:
 //SoupSessionFeature* sniffer = static_cast<SoupSessionFeature*>(g_object_new(SOUP_TYPE_CONTENT_S    NIFFER, NULL));
//soup_session_add_feature(session, sniffer);
//g_object_unref(sniffer);

and the leak was gone!

Is this a bug or some kind or feature that I don't understand? Could it be fixed with some configuration? I will continue to research is, but I would love to get a comment from someone that knows about "sniffer".

Thanks

[Dan Winship]

(In reply to comment #22)
> What I did was to revert back to the un-patched code. Then in the file
> WebKit/gtk/webkit/webkitprivate.cpp I commented out the last lines in that
> source:
>  //SoupSessionFeature* sniffer =
> static_cast<SoupSessionFeature*>(g_object_new(SOUP_TYPE_CONTENT_S    NIFFER,
> NULL));
> //soup_session_add_feature(session, sniffer);
> //g_object_unref(sniffer);
> 
> and the leak was gone!

Aha. This is the new Content-Type-auto-figuring-out feature. And that explains why you saw the bug and we didn't; because WebKit was later fixed to not do Content-Type sniffing for XML-RPC requests, so with current WebKit, the leak wouldn't happen.

This is now fixed in libsoup master (abc8a146f4bc8936bb416de00fb4e57fca2a21bd).

[Dan Winship]

(In reply to comment #23)
> WebKit was later fixed to not do
> Content-Type sniffing for XML-RPC requests

I meant XMLHttpRequest of course

[Gustavo Noronha (kov)]

As noted by Dan Winship, this is now fixed in soup.