Bug 25855

Summary: [GTK] Should support proper Content Encoding
Product: WebKit Reporter: Gustavo Noronha (kov) <gns@gnome.org>
Component: WebKit GtkAssignee: Nobody <webkit-unassigned@lists.webkit.org>
Status: RESOLVED FIXED    
Severity: Normal CC: baptiste.millemathias@gmail.com, christian@twotoasts.de, commit-queue@webkit.org, danw@gnome.org, eric225125@gmail.com, martin.sourada@gmail.com, sa@whiz.se, slomo@coaxion.net, tevaum@gmail.com, uzytkownik2@gmail.com, webkit.review.bot@gmail.com, zandobersek@gmail.com
Priority: P2 Keywords: Gtk, Soup
Version: 528+ (Nightly build)   
Hardware: PC   
OS: Mac OS X 10.5   
Attachments:
Description Flags
patch to use SoupContentDecoder none

Description From 2009-05-18 06:28:06 PST
Currently WebKitGTK+ is limited to using identity as content encoding. We are unable to take advantage of compression, for faster transfer rates, for instance. See https://bugs.webkit.org/show_bug.cgi?id=25854 and http://bugzilla.gnome.org/show_bug.cgi?id=522772 for more information.
------- Comment #1 From 2009-06-06 02:17:47 PST -------
*** Bug 26233 has been marked as a duplicate of this bug. ***
------- Comment #2 From 2009-07-07 04:04:37 PST -------
*** Bug 25481 has been marked as a duplicate of this bug. ***
------- Comment #3 From 2009-09-13 14:39:43 PST -------
*** Bug 29221 has been marked as a duplicate of this bug. ***
------- Comment #4 From 2009-10-06 06:07:55 PST -------
*** Bug 30066 has been marked as a duplicate of this bug. ***
------- Comment #5 From 2009-10-13 07:16:08 PST -------
OK, since real Content-Encoding support is probably not coming until GNOME 2.30, and we're continuing to trip over sites that use "Content-Encoding: gzip" even though we send "Accept-Encoding: identity" (notably archive.org), maybe a gross hack is in order... what we'd do is, check from gotHeadersCallback if the response includes "Content-Encoding: gzip"; if so, disconnect from content-sniffed and got-chunk and connect to got-body. Wait until the got-body handler is called, unzip the body data all at once, call soup_content_sniffer_sniff() if necessary, and then belatedly emit all the webkit "signals". This would break incremental rendering of the data, but it would only be getting used on broken sites anyway.
------- Comment #6 From 2009-10-21 10:33:49 PST -------
(In reply to comment #5)
> OK, since real Content-Encoding support is probably not coming until GNOME
> 2.30, and we're continuing to trip over sites that use "Content-Encoding: gzip"
> even though we send "Accept-Encoding: identity" (notably archive.org), maybe a
> gross hack is in order... what we'd do is, check from gotHeadersCallback if the
> response includes "Content-Encoding: gzip"; if so, disconnect from
> content-sniffed and got-chunk and connect to got-body. Wait until the got-body
> handler is called, unzip the body data all at once, call
> soup_content_sniffer_sniff() if necessary, and then belatedly emit all the
> webkit "signals". This would break incremental rendering of the data, but it
> would only be getting used on broken sites anyway.

I have been thinking the same. Are you planning to write such a patch, or should I try to secure some time to get it done?
------- Comment #7 From 2009-10-21 11:04:57 PST -------
i was not planning to write it myself
------- Comment #8 From 2009-10-23 05:17:54 PST -------
ok, i'm working on avb's update of my original patches. see the linked b.g.o bug
------- Comment #9 From 2009-11-07 05:42:10 PST -------
*** Bug 30934 has been marked as a duplicate of this bug. ***
------- Comment #10 From 2009-11-11 01:54:02 PST -------
I just wanted to let you know in case you didn't see it in the gnome bug posted in the first message that support for proper content encoding should be released soon with libsoup-2.28.2.  There is also a patch there that supposedly enables these features for webkit.
------- Comment #11 From 2009-11-30 06:20:46 PST -------
*** Bug 31982 has been marked as a duplicate of this bug. ***
------- Comment #12 From 2009-12-16 05:59:12 PST -------
Created an attachment (id=44968) [details]
patch to use SoupContentDecoder

(note that while this claims to require libsoup >= 2.28.2, i haven't actually updated the gnome-2-28 branch yet, so you'll want to test against master. that will be fixed before this gets committed of course.)
------- Comment #13 From 2009-12-16 06:01:35 PST -------
style-queue ran check-webkit-style on attachment 44968 [details] without any errors.
------- Comment #14 From 2009-12-16 10:29:23 PST -------
(In reply to comment #12)
> Created an attachment (id=44968) [details] [details]
> patch to use SoupContentDecoder
> 
> (note that while this claims to require libsoup >= 2.28.2, i haven't actually
> updated the gnome-2-28 branch yet, so you'll want to test against master. that
> will be fixed before this gets committed of course.)

The patch needs an #ifdef SOUP_TYPE_CONTENT_DECODER because otherwise it fails with current releases.
------- Comment #15 From 2009-12-16 10:52:54 PST -------
(From update of attachment 44968 [details])
Thanks!
------- Comment #16 From 2009-12-16 10:53:55 PST -------
(In reply to comment #14)
> The patch needs an #ifdef SOUP_TYPE_CONTENT_DECODER because otherwise it fails
> with current releases.

We decided to not handle this, and assume people mostly know what they're doing when using development releases. 2.29.4 will have it.
------- Comment #17 From 2009-12-16 11:14:44 PST -------
(From update of attachment 44968 [details])
Clearing flags on attachment: 44968

Committed r52208: <http://trac.webkit.org/changeset/52208>
------- Comment #18 From 2009-12-16 11:14:51 PST -------
All reviewed patches have been landed.  Closing bug.
------- Comment #19 From 2010-05-04 16:39:49 PST -------
*** Bug 29768 has been marked as a duplicate of this bug. ***