Bug 80587 - [Soup] Crash when loading www.seriespepito.com. "Too many open files"
Summary: [Soup] Crash when loading www.seriespepito.com. "Too many open files"
Status: RESOLVED DUPLICATE of bug 41630
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebKitGTK (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Linux
: P2 Normal
Assignee: Nobody
URL: http://www.seriespepito.com
Keywords:
Depends on:
Blocks:
 
Reported: 2012-03-08 04:16 PST by Sergio Villar Senin
Modified: 2012-03-13 13:19 PDT (History)
4 users (show)

See Also:


Attachments

Note You need to log in before you can comment on or make changes to this bug.
Description Sergio Villar Senin 2012-03-08 04:16:36 PST
WebKitGtk+ crashes when trying to load the above URL. The exact error I get is (cannot reproduce in debug builds nor under strace maybe because they run slower).

GLib-ERROR **: Creating pipes for GWakeup: Too many open files

Program received signal SIGTRAP, Trace/breakpoint trap.

I guess this is due to too many asynchronous operations, although the amount of resources it downloads (~55) is not that big. The SoupCache is not involved as it happens the same with GtkLauncher. Will try to get something from Debug builds.
Comment 1 Sergio Villar Senin 2012-03-09 01:30:02 PST
OK I've just verified that the problem in this case is the DNS prefetching. If I disable it then the crash does no longer happens. I guess something can be done at libsoup level...

BTW: this crash happens also with epiphany 3.2.1 in debian sid for example, it isn't not only a bug in master.
Comment 2 Dan Winship 2012-03-09 05:48:35 PST
see also bug 41630... although I'm not sure that bug exists, so maybe it was fixed at a higher level? (Or maybe I'm just not using epiphany enough to see it any more.)

if you run against debug glib/libsoup, where is it when it crashes?

and what does lsof show when it runs out of fds? (I'm guessing it's probably sockets to the nameserver...)
Comment 3 Dan Winship 2012-03-09 05:48:55 PST
(In reply to comment #2)
> see also bug 41630... although I'm not sure that bug exists

STILL exists, that is
Comment 4 Sergio Villar Senin 2012-03-12 02:40:08 PDT
(In reply to comment #1)
> OK I've just verified that the problem in this case is the DNS prefetching. If I disable it then the crash does no longer happens. I guess something can be done at libsoup level...
> 
> BTW: this crash happens also with epiphany 3.2.1 in debian sid for example, it isn't not only a bug in master.

(In reply to comment #2)
> see also bug 41630... although I'm not sure that bug exists, so maybe it was fixed at a higher level? (Or maybe I'm just not using epiphany enough to see it any more.)

I'm not sure it's the same bug, because I do not see DNS requests timing out but causing the SIGTRAP I mentioned on bug description.

> if you run against debug glib/libsoup, where is it when it crashes?

I've been trying, but it's difficult (no luck so far) to get the crash with debug builds as I said.

> and what does lsof show when it runs out of fds? (I'm guessing it's probably sockets to the nameserver...)

Good point, I'll get that info.
Comment 5 Sergio Villar Senin 2012-03-12 02:41:06 PDT
(In reply to comment #2)
> see also bug 41630... although I'm not sure that bug exists, so maybe it was fixed at a higher level? (Or maybe I'm just not using epiphany enough to see it any more.)

Although that's probably the reason why some images are not shown at the end of the load when it does not crash.
Comment 6 Dan Winship 2012-03-12 05:33:34 PDT
(In reply to comment #4)
> I'm not sure it's the same bug, because I do not see DNS requests timing out 

sure, but it could be the same underlying cause: if we send way way way too many DNS requests at once, then we could run out of sockets
Comment 7 Sergio Villar Senin 2012-03-12 06:50:01 PDT
(In reply to comment #6)
> (In reply to comment #4)
> > I'm not sure it's the same bug, because I do not see DNS requests timing out 
> 
> sure, but it could be the same underlying cause: if we send way way way too many DNS requests at once, then we could run out of sockets

Agree. Not sure what the fix should be, since the default gio resolver uses a threaded pool with an unlimited (-1) number of threads. I was also thinking, could it be that we end up calling soup_add_completion too much?

From my initial testing, it looks like WebKit tries to prefetch too many times the same resources over and over again. I've been taking a look at other ports and some of them like Windows use a prefetch queue that limits the maximim simultaneous requests to 8. I can try to do something similar and see what happens.
Comment 8 Zan Dobersek 2012-03-13 00:38:28 PDT
I'm no soup expert, but a similar problem was occurring loading a heavy page on github.

The problem was fixed in libsoup, GNOME bugzilla bug and commit:
https://bugzilla.gnome.org/show_bug.cgi?id=668508
http://git.gnome.org/browse/libsoup/commit/?id=856df33301221711789f0db744fce951eb70ba76

Is it the same problem?
Comment 9 Sergio Villar Senin 2012-03-13 08:00:22 PDT
(In reply to comment #8)
> I'm no soup expert, but a similar problem was occurring loading a heavy page on github.
> 
> The problem was fixed in libsoup, GNOME bugzilla bug and commit:
> https://bugzilla.gnome.org/show_bug.cgi?id=668508
> http://git.gnome.org/browse/libsoup/commit/?id=856df33301221711789f0db744fce951eb70ba76
> 
> Is it the same problem?

Kind of. The problem in this case is that we run out of sockets because webkit triggers an enormous amount of simultaneous DNS resolutions. I'm working on a fix for this, it basically involves using a queue to limit that number (something similar to what the other ports do BTW).
Comment 10 Sergio Villar Senin 2012-03-13 13:19:52 PDT
I think it's better to track this issue in bug 41630 as they're basically the same.

*** This bug has been marked as a duplicate of bug 41630 ***