Bug 72178 - Fix mixed content handling for AssociatedURLLoader.
Summary: Fix mixed content handling for AssociatedURLLoader.
Status: RESOLVED FIXED
Alias: None
Product: WebKit
Classification: Unclassified
Component: Media (show other bugs)
Version: 528+ (Nightly build)
Hardware: Unspecified Unspecified
: P2 Normal
Assignee: Aaron Colwell
URL:
Keywords:
Depends on:
Blocks:
 
Reported: 2011-11-11 13:52 PST by Aaron Colwell
Modified: 2011-12-07 18:45 PST (History)
12 users (show)

See Also:


Attachments
Patch (3.45 KB, patch)
2011-11-11 13:54 PST, Aaron Colwell
no flags Details | Formatted Diff | Diff
Patch (9.00 KB, patch)
2011-11-28 14:41 PST, Aaron Colwell
no flags Details | Formatted Diff | Diff
Added Layout Tests (16.00 KB, patch)
2011-11-29 10:47 PST, Aaron Colwell
no flags Details | Formatted Diff | Diff
Only consult targetType() when determining whether it is ok to request the resource. (15.63 KB, patch)
2011-11-30 14:59 PST, Aaron Colwell
no flags Details | Formatted Diff | Diff
Fix redirects and added layout tests to verify the fix. (26.01 KB, patch)
2011-12-01 13:43 PST, Aaron Colwell
no flags Details | Formatted Diff | Diff
Updated ChangeLog text. (26.12 KB, patch)
2011-12-02 09:45 PST, Aaron Colwell
no flags Details | Formatted Diff | Diff
Addressing CR comments (26.23 KB, patch)
2011-12-02 11:43 PST, Aaron Colwell
no flags Details | Formatted Diff | Diff
Partial revert to restore old behavior for workers. (1.79 KB, patch)
2011-12-07 09:53 PST, Aaron Colwell
no flags Details | Formatted Diff | Diff
Revert mixed content handling for video fix and follow-up rebases and Skipped updates. (34.45 KB, patch)
2011-12-07 13:43 PST, Aaron Colwell
no flags Details | Formatted Diff | Diff

Note You need to log in before you can comment on or make changes to this bug.
Description Aaron Colwell 2011-11-11 13:52:31 PST
Currently the AssociatedURLLoader does not check for mixed content so Chromium doesn't put the mixed content indicator in the omnibox when https:// content contains a <video> with an http: URL. (http://code.google.com/p/chromium/issues/detail?id=97166) 

The attached patch adds the necessary checks to properly handle mixed content.
Comment 1 Aaron Colwell 2011-11-11 13:54:32 PST
Created attachment 114766 [details]
Patch
Comment 2 Aaron Colwell 2011-11-14 09:30:39 PST
Can you review this please?
Comment 3 Adam Barth 2011-11-15 14:07:18 PST
Comment on attachment 114766 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=114766&action=review

We'll also need a test.

> Source/WebKit/chromium/src/AssociatedURLLoader.cpp:148
> +    if (!m_loader->checkIfDisplayInsecureContent(newRequest.url())) {
> +        m_loader->cancel();
> +        return;
> +    }

Don't we need all the checks from CachedResourceLoader::canRequest?  For example, what if this load is forbidden for other reasons?
Comment 4 Adam Barth 2011-11-15 14:08:40 PST
http://trac.webkit.org/browser/trunk/Source/WebCore/loader/cache/CachedResourceLoader.cpp#L266

Notice that we do a bunch of other checks at the same time as we check for mixed content.  If we're missing the mixed content checks, we're probably missing those as well.
Comment 5 Aaron Colwell 2011-11-15 14:17:33 PST
(In reply to comment #4)
> http://trac.webkit.org/browser/trunk/Source/WebCore/loader/cache/CachedResourceLoader.cpp#L266
> 
> Notice that we do a bunch of other checks at the same time as we check for mixed content.  If we're missing the mixed content checks, we're probably missing those as well.


Thanks for the pointers. I figured there might be more to this, but I didn't know where to look. :)
Comment 6 jochen 2011-11-16 00:31:55 PST
I wonder why the checks aren't applied in the first place. Doesn't the loader use a threadable loader which in turn uses a cached resource loader that should apply the correct checks?
Comment 7 Aaron Colwell 2011-11-16 14:03:29 PST
(In reply to comment #6)
> I wonder why the checks aren't applied in the first place. Doesn't the loader use a threadable loader which in turn uses a cached resource loader that should apply the correct checks?

So it appears that you are correct that AssociatedURLLoader creates a DocumentThreadableLoader which creates a CachedResourceLoader via CachedResourceLoader::requestRawResource(). Since the CachedResourceLoader has a type of RawResource the display checks are skipped. 

How about this for a fix:
1. Add a MediaResource to CachedResource::Type
2. Update switch() statements in CachedResourceLoader
3. Create the inverse of cachedResourceTypeToTargetType() that maps ResourceRequest::TargetType to CachedResource::Type
4. Update DocumentThreadableLoader::loadRequest() to use the new mapping function to determine what type of CachedResourceLoader() to create instead of always creating a RawResource CachedResource.

I believe this should fix the problem and possibly fix any other TargetTypes that happen to be getting loaded through DocumentThreadableLoader.
Comment 8 jochen 2011-11-16 14:10:07 PST
Adding Nate who added that code path
Comment 9 Adam Barth 2011-11-16 16:55:26 PST
Not all loads that go through AssociatedURLLoader are for media.  Perhaps we need some way to disambiguate what kind of load we're doing.
Comment 10 Aaron Colwell 2011-11-17 08:50:59 PST
(In reply to comment #9)
> Not all loads that go through AssociatedURLLoader are for media.  Perhaps we need some way to disambiguate what kind of load we're doing.

I agree. That is why I was suggesting using TargetType to CachedResource::Type translation to differentiate the different types of requests. Only when TargetType is TargetIsMedia will we create a CachedResource with Type MediaResource.
Comment 11 jochen 2011-11-17 16:02:48 PST
(In reply to comment #10)
> (In reply to comment #9)
> > Not all loads that go through AssociatedURLLoader are for media.  Perhaps we need some way to disambiguate what kind of load we're doing.
> 
> I agree. That is why I was suggesting using TargetType to CachedResource::Type translation to differentiate the different types of requests. Only when TargetType is TargetIsMedia will we create a CachedResource with Type MediaResource.

Note that currently TargetType is only available on Chromium, so we can't rely on it for a general solution.
Comment 12 Adam Barth 2011-11-17 16:05:40 PST
> Note that currently TargetType is only available on Chromium, so we can't rely on it for a general solution.

AssociatedURLLoader only exists in Chromium as well.  (It's a Chromium API concept.)
Comment 13 jochen 2011-11-17 16:08:35 PST
(In reply to comment #12)
> > Note that currently TargetType is only available on Chromium, so we can't rely on it for a general solution.
> 
> AssociatedURLLoader only exists in Chromium as well.  (It's a Chromium API concept.)

What about workers? If a worker requests a script, it'll be loaded as CachedRawResource instead of CachedScript, right?
Comment 14 Aaron Colwell 2011-11-18 15:14:34 PST
(In reply to comment #13)
> (In reply to comment #12)
> > > Note that currently TargetType is only available on Chromium, so we can't rely on it for a general solution.
> > 
> > AssociatedURLLoader only exists in Chromium as well.  (It's a Chromium API concept.)
> 
> What about workers? If a worker requests a script, it'll be loaded as CachedRawResource instead of CachedScript, right?

I believe workers would start using whatever TargetType was specified in CrossThreadResourceRequestData::m_targetType. Right now I believe they are just using RawResource as well. If someone could create me a quick worker example, I'd be happy to investigate more.
Comment 15 Aaron Colwell 2011-11-28 14:41:47 PST
Created attachment 116822 [details]
Patch
Comment 16 Aaron Colwell 2011-11-28 14:45:07 PST
Uploaded a new patch to give people a better idea of what I had in mind. 

I'm a little nervous about this change since I'm not at all familiar with this code or how wide an impact such a change would be. Expert opinions definitely wanted.
Comment 17 jochen 2011-11-28 14:46:12 PST
Comment on attachment 116822 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=116822&action=review

> Source/WebCore/ChangeLog:8
> +        No new tests. (OOPS!)

Was it possible to test this? E.g. try to modify one of the tests for mixed content?
Comment 18 Aaron Colwell 2011-11-28 14:50:36 PST
(In reply to comment #17)
> (From update of attachment 116822 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=116822&action=review
> 
> > Source/WebCore/ChangeLog:8
> > +        No new tests. (OOPS!)
> 
> Was it possible to test this? E.g. try to modify one of the tests for mixed content?

Haven't gotten to that yet. I wanted to make sure I was on the right track before investing too much effort in polishing the patch. I'll look around the LayoutTests to see if I can find something.
Comment 19 Early Warning System Bot 2011-11-28 14:51:16 PST
Comment on attachment 116822 [details]
Patch

Attachment 116822 [details] did not pass qt-ews (qt):
Output: http://queues.webkit.org/results/10676465
Comment 20 Aaron Colwell 2011-11-29 10:47:40 PST
Created attachment 116997 [details]
Added Layout Tests
Comment 21 jochen 2011-11-30 11:45:53 PST
Comment on attachment 116997 [details]
Added Layout Tests

View in context: https://bugs.webkit.org/attachment.cgi?id=116997&action=review

> Source/WebCore/loader/cache/CachedResourceLoader.cpp:291
> +    return static_cast<CachedRawResource*>(requestResource(type, request, String(), options, ResourceLoadPriorityUnresolved, false));

this won't work. If the type is e.g. ImageResource, requestResource will return a CachedResource

Maybe CachedRawResource can be smarter about its own type (e.g. move the targetTypeToCachedResourceType to CachedRawResource), and have the cached resource loader query the CachedRawResource for its type before doing its type-based checks?
Comment 22 Aaron Colwell 2011-11-30 14:59:30 PST
Created attachment 117282 [details]
Only consult targetType() when determining whether it is ok to request the resource.
Comment 23 Aaron Colwell 2011-11-30 17:39:16 PST
Comment on attachment 117282 [details]
Only consult targetType() when determining whether it is ok to request the resource.

Just discovered that redirects to insecure URLs aren't handled by this patch
Comment 24 Aaron Colwell 2011-12-01 13:43:22 PST
Created attachment 117479 [details]
Fix redirects and added layout tests to verify the fix.
Comment 25 Aaron Colwell 2011-12-01 13:50:44 PST
Please take another look. The patch intentionally focuses on just getting canRequest() to do checks with the proper type. I tried various ways of changing type() to return the appropriate type, but various code makes assumptions about (type() == RawResource) => CachedRawResource() and I'd get crashes because of bad casts and various other problems. 

I believe this is the lowest risk patch to get the desired behavior.
Comment 26 jochen 2011-12-01 13:54:47 PST
Comment on attachment 117479 [details]
Fix redirects and added layout tests to verify the fix.

View in context: https://bugs.webkit.org/attachment.cgi?id=117479&action=review

> Source/WebCore/ChangeLog:7
> +

You could add some text what this patch exactly does
Comment 27 jochen 2011-12-01 13:55:11 PST
otherwise, looks good
Comment 28 Aaron Colwell 2011-12-02 09:45:21 PST
Created attachment 117642 [details]
Updated ChangeLog text.
Comment 29 Aaron Colwell 2011-12-02 10:58:08 PST
Can I get an r+ & c+ on the latest patch pretty please. :)
Comment 30 Adam Barth 2011-12-02 11:03:09 PST
Comment on attachment 117642 [details]
Updated ChangeLog text.

View in context: https://bugs.webkit.org/attachment.cgi?id=117642&action=review

Assuming we're not digging ourselves a bigger hole w.r.t. targetType, this looks fine.  @jochen?

> Source/WebCore/loader/SubresourceLoader.cpp:130
> +    CachedResource::Type canRequestType = m_resource->type();

This variable should be a noun.  Maybe requestTypeForCanRequest ?

> Source/WebCore/loader/SubresourceLoader.cpp:135
> +#if PLATFORM(CHROMIUM)
> +    if (canRequestType == CachedResource::RawResource)
> +        canRequestType = CachedResource::targetTypeToCachedResourceType(request().targetType());
> +#endif

Is targetType() staying around?  I thought there were bug threads about deleting it...  Maybe those threads were just about moving it to Chromium specific code?

> Source/WebCore/loader/cache/CachedRawResource.h:33
>      CachedRawResource(ResourceRequest&);

This first constructor should have the explicit keyword.
Comment 31 Aaron Colwell 2011-12-02 11:43:32 PST
Comment on attachment 117642 [details]
Updated ChangeLog text.

View in context: https://bugs.webkit.org/attachment.cgi?id=117642&action=review

>> Source/WebCore/loader/SubresourceLoader.cpp:130
>> +    CachedResource::Type canRequestType = m_resource->type();
> 
> This variable should be a noun.  Maybe requestTypeForCanRequest ?

Done.

>> Source/WebCore/loader/SubresourceLoader.cpp:135
>> +#endif
> 
> Is targetType() staying around?  I thought there were bug threads about deleting it...  Maybe those threads were just about moving it to Chromium specific code?

I honestly don't know. I'm not familiar with the history of this code. I just know that TargetType is how the video tag indicates that it is making a request for media. I think trying to change that is much higher risk and will likely trigger several yak shaves. This change at least exposes a use case that needs to be handled when TargetType is eventually removed.

>> Source/WebCore/loader/cache/CachedRawResource.h:33
>>      CachedRawResource(ResourceRequest&);
> 
> This first constructor should have the explicit keyword.

Done
Comment 32 Aaron Colwell 2011-12-02 11:43:58 PST
Created attachment 117662 [details]
Addressing CR comments
Comment 33 Adam Barth 2011-12-02 12:58:45 PST
(In reply to comment #31)
> (From update of attachment 117642 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=117642&action=review
> 
> >> Source/WebCore/loader/SubresourceLoader.cpp:135
> >> +#endif
> > 
> > Is targetType() staying around?  I thought there were bug threads about deleting it...  Maybe those threads were just about moving it to Chromium specific code?
> 
> I honestly don't know. I'm not familiar with the history of this code. I just know that TargetType is how the video tag indicates that it is making a request for media. I think trying to change that is much higher risk and will likely trigger several yak shaves. This change at least exposes a use case that needs to be handled when TargetType is eventually removed.

Yeah, I was mostly asking jochen, who was involved in the discussions about the future of targetType().
Comment 34 jochen 2011-12-02 13:07:52 PST
(In reply to comment #33)
> (In reply to comment #31)
> > (From update of attachment 117642 [details] [details])
> > View in context: https://bugs.webkit.org/attachment.cgi?id=117642&action=review
> > 
> > >> Source/WebCore/loader/SubresourceLoader.cpp:135
> > >> +#endif
> > > 
> > > Is targetType() staying around?  I thought there were bug threads about deleting it...  Maybe those threads were just about moving it to Chromium specific code?
> > 
> > I honestly don't know. I'm not familiar with the history of this code. I just know that TargetType is how the video tag indicates that it is making a request for media. I think trying to change that is much higher risk and will likely trigger several yak shaves. This change at least exposes a use case that needs to be handled when TargetType is eventually removed.
> 
> Yeah, I was mostly asking jochen, who was involved in the discussions about the future of targetType().

It's chromium only currently. I still hope that I can make it a general part of loader
Comment 35 Adam Barth 2011-12-02 13:31:19 PST
> It's chromium only currently. I still hope that I can make it a general part of loader

So, Alexey agreed that we don't need to remove it?
Comment 36 Adam Barth 2011-12-02 13:43:18 PST
Comment on attachment 117662 [details]
Addressing CR comments

I chatted with jochen.  It sounds like this is the right approach for this patch.
Comment 37 Aaron Colwell 2011-12-02 13:46:53 PST
(In reply to comment #36)
> (From update of attachment 117662 [details])
> I chatted with jochen.  It sounds like this is the right approach for this patch.

Thanks for the review. I appreciate it. :)
Comment 38 WebKit Review Bot 2011-12-02 17:38:44 PST
Comment on attachment 117662 [details]
Addressing CR comments

Clearing flags on attachment: 117662

Committed r101883: <http://trac.webkit.org/changeset/101883>
Comment 39 WebKit Review Bot 2011-12-02 17:38:50 PST
All reviewed patches have been landed.  Closing bug.
Comment 40 Csaba Osztrogonác 2011-12-03 01:22:28 PST
New tests introduced in this bug fail on SL, GTK and Qt too:

SL fails:
- http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r101914%20(35151)/http/tests/security/mixedContent/redirect-http-to-https-video-in-main-frame-pretty-diff.html
- http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r101914%20(35151)/http/tests/security/mixedContent/insecure-video-in-main-frame-pretty-diff.html
- http://build.webkit.org/results/SnowLeopard%20Intel%20Release%20(Tests)/r101914%20(35151)/http/tests/security/mixedContent/redirect-https-to-http-video-in-main-frame-pretty-diff.html

GTK fails:
- http://build.webkit.org/results/GTK%20Linux%2032-bit%20Release/r101898%20(19680)/http/tests/security/mixedContent/redirect-http-to-https-video-in-main-frame-pretty-diff.html
- http://build.webkit.org/results/GTK%20Linux%2032-bit%20Release/r101898%20(19680)/http/tests/security/mixedContent/insecure-video-in-main-frame-pretty-diff.html
- http://build.webkit.org/results/GTK%20Linux%2032-bit%20Release/r101898%20(19680)/http/tests/security/mixedContent/redirect-https-to-http-video-in-main-frame-pretty-diff.html

Qt fails:
- http://build.webkit.org/results/Qt%20Linux%20Release/r101902%20(40538)/http/tests/security/mixedContent/redirect-http-to-https-video-in-main-frame-pretty-diff.html
- http://build.webkit.org/results/Qt%20Linux%20Release/r101902%20(40538)/http/tests/security/mixedContent/insecure-video-in-main-frame-pretty-diff.html
- http://build.webkit.org/results/Qt%20Linux%20Release/r101902%20(40538)/http/tests/security/mixedContent/redirect-https-to-http-video-in-main-frame-pretty-diff.html

Could you check what happened?
Comment 41 Csaba Osztrogonác 2011-12-03 01:40:31 PST
I skipped them on Qt: http://trac.webkit.org/changeset/101918/trunk/LayoutTests/platform/qt/Skipped

Please unskip them if the bug is fixed once.
Comment 42 Philippe Normand 2011-12-03 04:01:29 PST
I'm checking the issue on GTK. I think it doesn't work because we don't use the SubResourceLoader for videos.
Comment 43 Vsevolod Vlasov 2011-12-05 03:02:36 PST
This caused http://code.google.com/p/chromium/issues/detail?id=106383 (chromium ui_tests WorkerTest.SharedWorkerHttpAuth fails consistently.)
Comment 44 Vsevolod Vlasov 2011-12-05 03:07:57 PST
Rebaselined tests introduced by this patch here: http://trac.webkit.org/changeset/101981
Comment 45 Vsevolod Vlasov 2011-12-05 03:39:17 PST
Committed r101986: <http://trac.webkit.org/changeset/101986>

One more rebaseline since leopard and snowleopard had different results.
Comment 46 Vsevolod Vlasov 2011-12-05 09:14:53 PST
Committed r101997: <http://trac.webkit.org/changeset/101997>

And one more.
Now it has custom expectations on mac-snowleopard and mac-cg-snowleopard only.
Comment 47 jochen 2011-12-05 13:29:58 PST
Note that the diffs basically say that insecure content is displayed on these ports without a warning :-/
Comment 48 Vsevolod Vlasov 2011-12-06 05:52:49 PST
No, for me it looks like these warnings are displayed twice on snowleopard.
Comment 49 Aaron Colwell 2011-12-07 09:53:10 PST
Created attachment 118222 [details]
Partial revert to restore old behavior for workers.
Comment 50 Aaron Colwell 2011-12-07 09:57:47 PST
It appears this change has caused a regression in SharedWorkers (http://crbug.com/106383) and I've been unable to figure out how to fix them over the last to days. 

Could someone please review the partial revert so that I can at least temporarily restore workers to their old behavior that does less security checks.

Thanks.
Comment 51 Adam Barth 2011-12-07 10:36:54 PST
Comment on attachment 118222 [details]
Partial revert to restore old behavior for workers.

View in context: https://bugs.webkit.org/attachment.cgi?id=118222&action=review

> Source/WebCore/ChangeLog:11
> +        No new tests. (OOPS!)

Can we not test this problem here?

> Source/WebCore/loader/cache/CachedResource.cpp:139
>      case ResourceRequest::TargetIsScript:
> -        return CachedResource::Script;
> +        // TODO: Change back to CachedResource::Script once http://crbug.com/106383 has been fixed.
> +        return CachedResource::RawResource;

TODO -> FIXME

This patch looks wrong on it's face.  I'd rather revert the original patch entirely than land this "obviously wrong" code.  Can we not just fix the bug directly?
Comment 52 Aaron Colwell 2011-12-07 13:43:11 PST
Created attachment 118268 [details]
Revert mixed content handling for video fix and follow-up rebases and Skipped updates.
Comment 53 Aaron Colwell 2011-12-07 13:49:09 PST
Talked with levin@ and he said he'd help me figure out the issues w/ SharedWorkers. In the mean time I'm just going to revert everything.
Comment 54 Adam Barth 2011-12-07 13:52:38 PST
Comment on attachment 118268 [details]
Revert mixed content handling for video fix and follow-up rebases and Skipped updates.

Thanks.
Comment 55 WebKit Review Bot 2011-12-07 18:45:17 PST
Comment on attachment 118268 [details]
Revert mixed content handling for video fix and follow-up rebases and Skipped updates.

Clearing flags on attachment: 118268

Committed r102300: <http://trac.webkit.org/changeset/102300>
Comment 56 WebKit Review Bot 2011-12-07 18:45:24 PST
All reviewed patches have been landed.  Closing bug.