Bug 170597 – Web Inspector: Compression inaccurately reported when Content-Length header omitted

• Component: Web Inspector
• Status: RESOLVED DUPLICATE

[Harry Roberts]

Created attachment 306487 [details]
Highlighted issues in screenshot

**Issue:** Network panel detects compression (e.g. gzip) but inaccurately reports transferred size and compression delta if `Content-Length` header is not provided.

**Steps to reproduce:**

A)

1. Visit csswizardry.com
2. Open network panel with Size and Transferred columns visible
3. Hard reload page and inspect csswizardry.com document resource (i.e. the HTML page).
4. Notice that Size and Transferred values are similar (Transferred slightly larger to account for headers)
5. View csswizardry.com document resource (i.e. the HTML page) in Details sidebar.
6. Notice that Encoded/Decoded/Transferred size are also similar.
7. Compressed is marked Yes—Safari knew this resource was compressed.
8. Compression is set to 1×. Delta should actually be around 3.99× at time of writing.

B)

1. Follow the above steps for Twitter’s `widget.js` file.
2. Notice that Size and Transferred values are more like what we’d expect to see from a compressed (e.g. gzip) asset.
3. Compressed is also marked Yes—Safari knew this resource was compressed.
4. Notice that compression delta is 3.51×—much more like what we would expect to see.

C)

I know that my homepage *is* compressed, as does Safari, but it’s misreporting the Transferred size. By process of elimination, I noticed that the key difference between correctly reported and incorrectly reported values is that the correct assets have the Content-Length header present; incorrect assets have the Content-Length header missing. Ergo, I think the problem lies somewhere here.

**N.B.:** This is just a hunch, but I can’t draw any other lines between the cause and the problem. I may well be way off the mark here.

**Attachments:**

Attachment (2880×3600px, 647KB) shows two screenshots:

* Top shows annotated screenshot for **incorrectly** reported assets (2880×1800px, 511KB).
* Bottom shows annotated screenshot for **correctly** reported assets (2880×1800px, 511KB).

[Harry Roberts]

Please ignore text ‘(2880×1800px, 511KB)’: I initially intended to upload two screenshots before realising it wasn’t possible. Unable to edit original post to remove this text.

[Blaze Burg]

(In reply to Harry Roberts from comment #1)
> Please ignore text ‘(2880×1800px, 511KB)’: I initially intended to upload
> two screenshots before realising it wasn’t possible. Unable to edit original
> post to remove this text.

 You have to upload them one at a time, unfortunately. Click "Add an attachment" again.

[Blaze Burg]

(In reply to Harry Roberts from comment #0)
> Created attachment 306487 [details]
> Highlighted issues in screenshot
> 
> **Issue:** Network panel detects compression (e.g. gzip) but inaccurately
> reports transferred size and compression delta if `Content-Length` header is
> not provided.
> 
> **Steps to reproduce:**
> 
> A)
> 
> 1. Visit csswizardry.com
> 2. Open network panel with Size and Transferred columns visible
> 3. Hard reload page and inspect csswizardry.com document resource (i.e. the
> HTML page).
> 4. Notice that Size and Transferred values are similar (Transferred slightly
> larger to account for headers)
> 5. View csswizardry.com document resource (i.e. the HTML page) in Details
> sidebar.
> 6. Notice that Encoded/Decoded/Transferred size are also similar.
> 7. Compressed is marked Yes—Safari knew this resource was compressed.
> 8. Compression is set to 1×. Delta should actually be around 3.99× at time
> of writing.

I'm unable to verify this expected compression. When I save the file to disk it has a size of around 56KB and the reported transfer size is 55.09KB. This matches the compression of near 1x.

When I view this in Chrome, the transfer size is reported as 13.7KB, which matches your expected compression ratio.

So, either the displayed transfer size in Safari is wrong, or Safari didn't actually accept the gzip'd resource and actually transferred the uncompressed resource.

If you have server logs it should be possible to verify what's actually being sent. You could also try using Wireshark or something.

> 
> B)
> 
> 1. Follow the above steps for Twitter’s `widget.js` file.
> 2. Notice that Size and Transferred values are more like what we’d expect to
> see from a compressed (e.g. gzip) asset.
> 3. Compressed is also marked Yes—Safari knew this resource was compressed.
> 4. Notice that compression delta is 3.51×—much more like what we would
> expect to see.
> 
> C)
> 
> I know that my homepage *is* compressed, as does Safari, but it’s
> misreporting the Transferred size. By process of elimination, I noticed that
> the key difference between correctly reported and incorrectly reported
> values is that the correct assets have the Content-Length header present;
> incorrect assets have the Content-Length header missing. Ergo, I think the
> problem lies somewhere here.

This says to me that Safari might require Content-Length header to use compression.

[Harry Roberts]

Created attachment 306569 [details]
Charles Proxy screenshot showing Safari transfer

[Harry Roberts]

I’ve attached a Charles screenshot; it looks like Safari is getting sent gzipped assets as far as I can tell: https://bug-170597-attachments.webkit.org/attachment.cgi?id=306569

[Blaze Burg]

(In reply to Harry Roberts from comment #5)
> I’ve attached a Charles screenshot; it looks like Safari is getting sent
> gzipped assets as far as I can tell:
> https://bug-170597-attachments.webkit.org/attachment.cgi?id=306569

This is really helpful, thanks for your detailed report and data. I think Joe will take a look at this.

[Radar WebKit Bug Importer]

<rdar://problem/31535280>

[Joseph Pecoraro]

Thanks for filing this detailed report!

This will end up getting addressed by my changes in Bug 155112.
https://bugs.webkit.org/show_bug.cgi?id=155112

With my changes I see:

    Encoded         13.57 KB
    Decoded	    54.72 KB
    Transferred	    14.02 KB
    
    Compressed	    Yes
    Compression	    4.03×

With is in line with your expectations.

Unfortunately, because Safari gets this information from underlying networking frameworks (in this case CFNetwork) the changes won't immediately be available.

I'll dup and transfer the relevant information.

*** This bug has been marked as a duplicate of bug 155112 ***