Bug 25855

Summary: [GTK] Should support proper Content Encoding
Product: WebKit Reporter: Gustavo Noronha (kov) <gns>
Component: WebKit GtkAssignee: Nobody <webkit-unassigned>
Status: RESOLVED FIXED    
Severity: Normal CC: baptiste.millemathias, christian, commit-queue, danw, eric225125, martin.sourada, sa, slomo, tevaum, uzytkownik2, webkit.review.bot, zan
Priority: P2 Keywords: Gtk, Soup
Version: 528+ (Nightly build)   
Hardware: PC   
OS: Mac OS X 10.5   
Attachments:
Description Flags
patch to use SoupContentDecoder none

Description Gustavo Noronha (kov) 2009-05-18 06:28:06 PDT
Currently WebKitGTK+ is limited to using identity as content encoding. We are unable to take advantage of compression, for faster transfer rates, for instance. See https://bugs.webkit.org/show_bug.cgi?id=25854 and http://bugzilla.gnome.org/show_bug.cgi?id=522772 for more information.
Comment 1 Xan Lopez 2009-06-06 02:17:47 PDT
*** Bug 26233 has been marked as a duplicate of this bug. ***
Comment 2 Jan Alonzo 2009-07-07 04:04:37 PDT
*** Bug 25481 has been marked as a duplicate of this bug. ***
Comment 3 Xan Lopez 2009-09-13 14:39:43 PDT
*** Bug 29221 has been marked as a duplicate of this bug. ***
Comment 4 Xan Lopez 2009-10-06 06:07:55 PDT
*** Bug 30066 has been marked as a duplicate of this bug. ***
Comment 5 Dan Winship 2009-10-13 07:16:08 PDT
OK, since real Content-Encoding support is probably not coming until GNOME 2.30, and we're continuing to trip over sites that use "Content-Encoding: gzip" even though we send "Accept-Encoding: identity" (notably archive.org), maybe a gross hack is in order... what we'd do is, check from gotHeadersCallback if the response includes "Content-Encoding: gzip"; if so, disconnect from content-sniffed and got-chunk and connect to got-body. Wait until the got-body handler is called, unzip the body data all at once, call soup_content_sniffer_sniff() if necessary, and then belatedly emit all the webkit "signals". This would break incremental rendering of the data, but it would only be getting used on broken sites anyway.
Comment 6 Gustavo Noronha (kov) 2009-10-21 10:33:49 PDT
(In reply to comment #5)
> OK, since real Content-Encoding support is probably not coming until GNOME
> 2.30, and we're continuing to trip over sites that use "Content-Encoding: gzip"
> even though we send "Accept-Encoding: identity" (notably archive.org), maybe a
> gross hack is in order... what we'd do is, check from gotHeadersCallback if the
> response includes "Content-Encoding: gzip"; if so, disconnect from
> content-sniffed and got-chunk and connect to got-body. Wait until the got-body
> handler is called, unzip the body data all at once, call
> soup_content_sniffer_sniff() if necessary, and then belatedly emit all the
> webkit "signals". This would break incremental rendering of the data, but it
> would only be getting used on broken sites anyway.

I have been thinking the same. Are you planning to write such a patch, or should I try to secure some time to get it done?
Comment 7 Dan Winship 2009-10-21 11:04:57 PDT
i was not planning to write it myself
Comment 8 Dan Winship 2009-10-23 05:17:54 PDT
ok, i'm working on avb's update of my original patches. see the linked b.g.o bug
Comment 9 Sven Arvidsson 2009-11-07 05:42:10 PST
*** Bug 30934 has been marked as a duplicate of this bug. ***
Comment 10 eric225125 2009-11-11 01:54:02 PST
I just wanted to let you know in case you didn't see it in the gnome bug posted in the first message that support for proper content encoding should be released soon with libsoup-2.28.2.  There is also a patch there that supposedly enables these features for webkit.
Comment 11 Gustavo Noronha (kov) 2009-11-30 06:20:46 PST
*** Bug 31982 has been marked as a duplicate of this bug. ***
Comment 12 Dan Winship 2009-12-16 05:59:12 PST
Created attachment 44968 [details]
patch to use SoupContentDecoder

(note that while this claims to require libsoup >= 2.28.2, i haven't actually updated the gnome-2-28 branch yet, so you'll want to test against master. that will be fixed before this gets committed of course.)
Comment 13 WebKit Review Bot 2009-12-16 06:01:35 PST
style-queue ran check-webkit-style on attachment 44968 [details] without any errors.
Comment 14 Christian Dywan 2009-12-16 10:29:23 PST
(In reply to comment #12)
> Created an attachment (id=44968) [details]
> patch to use SoupContentDecoder
> 
> (note that while this claims to require libsoup >= 2.28.2, i haven't actually
> updated the gnome-2-28 branch yet, so you'll want to test against master. that
> will be fixed before this gets committed of course.)

The patch needs an #ifdef SOUP_TYPE_CONTENT_DECODER because otherwise it fails with current releases.
Comment 15 Gustavo Noronha (kov) 2009-12-16 10:52:54 PST
Comment on attachment 44968 [details]
patch to use SoupContentDecoder

Thanks!
Comment 16 Gustavo Noronha (kov) 2009-12-16 10:53:55 PST
(In reply to comment #14)
> The patch needs an #ifdef SOUP_TYPE_CONTENT_DECODER because otherwise it fails
> with current releases.

We decided to not handle this, and assume people mostly know what they're doing when using development releases. 2.29.4 will have it.
Comment 17 WebKit Commit Bot 2009-12-16 11:14:44 PST
Comment on attachment 44968 [details]
patch to use SoupContentDecoder

Clearing flags on attachment: 44968

Committed r52208: <http://trac.webkit.org/changeset/52208>
Comment 18 WebKit Commit Bot 2009-12-16 11:14:51 PST
All reviewed patches have been landed.  Closing bug.
Comment 19 Martin Robinson 2010-05-04 16:39:49 PDT
*** Bug 29768 has been marked as a duplicate of this bug. ***