Bug 234159 - CSP: Always use UTF-8 encoded content when checking hashes
Summary: CSP: Always use UTF-8 encoded content when checking hashes
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: WebCore Misc. (show other bugs)
Version: WebKit Nightly Build
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Patrick Griffis
URL:
Keywords: InRadar
Depends on:
Blocks:
 
Reported: 2021-12-10 11:18 PST by Patrick Griffis
Modified: 2022-02-08 15:50 PST (History)
5 users (show)

See Also:


Attachments
Patch (4.31 KB, patch)
2021-12-10 11:19 PST, Patrick Griffis
no flags Details | Formatted Diff | Diff
Patch (11.60 KB, patch)
2021-12-11 14:58 PST, Patrick Griffis
no flags Details | Formatted Diff | Diff
Patch (11.17 KB, patch)
2021-12-11 14:59 PST, Patrick Griffis
no flags Details | Formatted Diff | Diff
Patch (12.56 KB, patch)
2021-12-12 10:23 PST, Patrick Griffis
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Patrick Griffis 2021-12-10 11:18:50 PST
CSP: Always use UTF-8 encoded content when checking hashes
Comment 1 Patrick Griffis 2021-12-10 11:19:43 PST
Created attachment 446767 [details]
Patch
Comment 2 Patrick Griffis 2021-12-10 11:21:32 PST Comment hidden (obsolete)
Comment 3 Patrick Griffis 2021-12-11 14:58:07 PST
Created attachment 446903 [details]
Patch
Comment 4 Patrick Griffis 2021-12-11 14:59:07 PST
Created attachment 446904 [details]
Patch
Comment 5 Patrick Griffis 2021-12-11 15:00:01 PST
Still not sure why running the browser directly has different results, however this certainly is more in-line with the spec and passes more WPT tests.
Comment 6 Patrick Griffis 2021-12-12 10:23:11 PST
Created attachment 446935 [details]
Patch
Comment 7 Patrick Griffis 2021-12-12 11:49:16 PST
The test failure really only happens when running via WebKit's HTTP server.

For example: `hash-always-converted-to-utf-8/iso-8859-1.html`

- `wpt serve` succeeds.
- https://wpt.live succeeds.
- `run-webkit-httpd` fails.

So its something specific to how the tests are run. I would assume its not specific to the HTTP server as macOS and GTK have different servers that both fail.
Comment 8 Patrick Griffis 2021-12-12 11:53:31 PST
> So its something specific to how the tests are run. I would assume its not
> specific to the HTTP server as macOS and GTK have different servers that
> both fail.

Sorry forgot to mention that the issue is in the raw data received at the HTTP layer.

libsoup logs this as incoming HTTP data:

`run-webkit-httpd`:
>     <!-- ? (micro sign) has the value of 0xB5 in latin-1 and of 0xC2B5 in utf-8 but the hash value should be the same as the utf-8 computed one -->

https://wpt.live:
>     <!-- \xb5 (micro sign) has the value of 0xB5 in latin-1 and of 0xC2B5 in utf-8 but the hash value should be the same as the utf-8 computed one -->

So the browser gets already invalid data from the HTTP server.
Comment 9 Radar WebKit Bug Importer 2021-12-17 11:19:17 PST
<rdar://problem/86643075>
Comment 10 Kate Cheney 2021-12-20 10:44:49 PST
Comment on attachment 446935 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=446935&action=review

r=me. Could you file a bug about the unexpected http server results?

> Source/WebCore/page/csp/ContentSecurityPolicy.cpp:-371
> -    // FIXME: Compute the digest with respect to the raw bytes received from the page.

We should remember to mark this bug resolved once this patch lands.
Comment 11 EWS 2021-12-20 12:14:07 PST
Committed r287270 (245426@main): <https://commits.webkit.org/245426@main>

All reviewed patches have been landed. Closing bug and clearing flags on attachment 446935 [details].
Comment 12 Brent Fulgham 2022-02-08 15:50:28 PST
This change should be present in STP 139, iOS 15.4 Beta, and macOS 12.3 Beta.