Bug 293611 – Unable to validate proxy certificate nor provide client auth credential for proxy

• Status: NEW
• Component: Page Loading

[Mateusz Szymański]

This is a follow up on [this thread](https://developer.apple.com/forums/thread/751750).

I'm running a HTTPS CONNECT proxy that uses self-signed TLS certificate with known Root CA. The proxy expects the client to provide a valid certificate for authentication with mTLS.

I want my WebKit based client application to verify the issuer of proxy's certificate and submit the mTLS credential upon request. I don't want to neither add the Root CA to global trust store nor to store the mTLS certificate in any location that's accessible outside of my client application.

The following code that uses `NSURLRequest` works as expected:

```NSURLSessionConfiguration *config = [NSURLSessionConfiguration ephemeralSessionConfiguration];
 
    NSDictionary *proxyDict = @{
        (NSString *)kCFNetworkProxiesHTTPEnable: @YES,
        (NSString *)kCFNetworkProxiesHTTPProxy : GetHost(),
        (NSString *)kCFNetworkProxiesHTTPPort : GetPort(),
       
        @"HTTPSEnable" : @YES,
        @"HTTPSProxy": GetHost(),
        @"HTTPSPort": GetPort(),
    };
 
    config.connectionProxyDictionary = proxyDict;
    
    NSURLSession *session = [NSURLSession sessionWithConfiguration:config delegate:self delegateQueue:[NSOperationQueue mainQueue]];
    NSURLRequest *request1 = [NSURLRequest requestWithURL:[NSURL URLWithString:@"https://api.ipify.org"]];
 
    NSURLSessionDataTask *task = [session dataTaskWithRequest:request1 completionHandler:
                                  ^(NSData *data, NSURLResponse *response, NSError *error) {
                                   
                                      if (error) {
                                      NSLog(@"Failed === Response:%@ %@\n", response, error);
                                      }
                                      NSString* newStr = [[NSString alloc] initWithData:data encoding:NSUTF8StringEncoding];
                                      NSLog(@"%@",newStr);
                                  }];
    [task resume];

// delegates

- (void)URLSession:(NSURLSession *)session didReceiveChallenge:(NSURLAuthenticationChallenge *)challenge completionHandler:(void (^)(NSURLSessionAuthChallengeDisposition, NSURLCredential * _Nullable))completionHandler
{
    NSLog(@"%s",__func__);
    completionHandler(NSURLSessionAuthChallengePerformDefaultHandling, nil);
}
 
- (void)URLSession:(NSURLSession *)session task:(NSURLSessionTask *)task
                            didReceiveChallenge:(NSURLAuthenticationChallenge *)challenge
 completionHandler:(void (NS_SWIFT_SENDABLE ^)(NSURLSessionAuthChallengeDisposition disposition, NSURLCredential * _Nullable credential))completionHandler {
    NSLog(@"%s",__func__);
    completionHandler(NSURLSessionAuthChallengePerformDefaultHandling, nil);
}```

properly entering both `didReceiveChallenge` blocks, allowing me to both validate proxy's cert as well as to submit my credential in response.

However, if I produce similar WebKit based configuration I don't receive the callbacks (sample code for iOS app):

```#import "ViewController.h"
#import <WebKit/WebKit.h>
 
@import Foundation;
@import WebKit;
 
@interface ViewController () <WKNavigationDelegate>
@property (nonatomic,strong) WKWebView* webView;
@property (nonatomic, strong) WKWebViewConfiguration * webConfig;
@end
 
@implementation ViewController
 
- (void)loadView {
    [super loadView];
    
    nw_protocol_options_t tls_options = nw_tls_create_options();
    sec_protocol_options_t sec_options = nw_tls_copy_sec_protocol_options(tls_options);
 
    sec_protocol_options_set_challenge_block(
        sec_options,
        ^(sec_protocol_metadata_t metadata, sec_protocol_challenge_complete_t challenge_complete) {
            NSLog(@"Inside of challenge block");
            challenge_complete(nil);
        },
        dispatch_get_main_queue());
    
    nw_endpoint_t proxy_endpoint =
    nw_endpoint_create_host(GetHost(), GetPort());
    
    nw_relay_hop_t relay =
    nw_relay_hop_create(nil, proxy_endpoint, tls_options);
    nw_proxy_config_t proxy_config =
    nw_proxy_config_create_relay(relay, nil);
    
    nw_proxy_config_add_match_domain(proxy_config, "api.ipify.org");
    
    self.webConfig = [[WKWebViewConfiguration alloc] init];
    self.webConfig.websiteDataStore = [WKWebsiteDataStore nonPersistentDataStore];
    self.webConfig.websiteDataStore.proxyConfigurations = @[ proxy_config ];
    
    self.webView = [[WKWebView alloc] initWithFrame:self.view.frame configuration:self.webConfig];
    
    self.webView.navigationDelegate = self;
    
    [self.view addSubview:self.webView];
}
 
- (void)viewDidLoad {
    [super viewDidLoad];
    NSLog(@"%s",__func__);
    NSURL* url = [[NSURL alloc] initWithString:@"https://api.ipify.org"];
    NSURLRequest* request = [[NSURLRequest alloc] initWithURL:url];
    [self.webView loadRequest:request];
}
 
- (void)webView:(WKWebView *)webView didStartProvisionalNavigation:(WKNavigation *)navigation {
    NSLog(@"%s",__func__);
}
 
- (void)webView:(WKWebView *)webView didFailProvisionalNavigation:(WKNavigation *)navigation withError:(NSError *)error {
    NSLog(@"%s. Error %@",__func__,error);
}
 
- (void)webView:(WKWebView *)webView didReceiveAuthenticationChallenge:(NSURLAuthenticationChallenge *)challenge completionHandler:(void (^)(NSURLSessionAuthChallengeDisposition, NSURLCredential *))completionHandler {
    NSLog(@"%s",__func__);
    NSLog(@"protection space: %@", challenge.protectionSpace.authenticationMethod);
    completionHandler(NSURLSessionAuthChallengePerformDefaultHandling, nil);
}
 
@end```

The logs state:

```	-[ViewController viewDidLoad]
	-[ViewController webView:didStartProvisionalNavigation:]
	-[ViewController webView:didFailProvisionalNavigation:withError:]. Error Error Domain=NSURLErrorDomain Code=-1206 "The server “api.ipify.org” requires a client certificate."```

So no attempt to get client credential was made.

When running similar code based on MiniBrowser bundled with WebKit I can observe the following output in the logs:

```0x139201800 - [pageProxyID=5, webPageID=6, frameID=4294967297, resourceID=185, isMainResource=1, destination=3, isSynchronous=0] NetworkResourceLoader::startNetworkLoad: Going to the network (description=LocalDataTask <383186CF-0D26-4BAA-A2EB-3B3ACC4F9EE8>.<9>)
boringssl_context_handle_fatal_alert(2294) [C4.1.1.1.1:2][0x13f60d5c0] write alert, level: fatal, description: bad certificate
boringssl_context_error_print(2284) [C4.1.1.1.1:2][0x13f60d5c0] Error: 4393584304:error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED:/AppleInternal/Library/BuildRoots/1c8f7852-1ca9-11f0-b28b-226177e5bb69/Library/Caches/com.apple.xbs/Sources/boringssl/ssl/handshake.cc:397:
__nw_protocol_tcp_disconnect [C4.1.1.1.1:3] Sandbox does not allow __nexus_set_opt
nw_http2_connection_close [C4.1.1.1.1:1] <i1> id table is NULL, cannot close streams
boringssl_session_handshake_incomplete(244) [C4.1.1.1.1:2][0x13f60d5c0] SSL library error
boringssl_session_handshake_error_print(47) [C4.1.1.1.1:2][0x13f60d5c0] Error: 4393584304:error:1000007d:SSL routines:OPENSSL_internal:CERTIFICATE_VERIFY_FAILED:/AppleInternal/Library/BuildRoots/1c8f7852-1ca9-11f0-b28b-226177e5bb69/Library/Caches/com.apple.xbs/Sources/boringssl/ssl/handshake.cc:397:
nw_protocol_boringssl_handshake_negotiate_proceed(787) [C4.1.1.1.1:2][0x13f60d5c0] handshake failed at state 12288: not completed
nw_endpoint_flow_failed_with_error [C4.1.1.1.1 <host>:<port> in_progress channel-flow (satisfied (Path is satisfied), viable, interface: en0[802.11], ipv4, dns, uses wifi)] already failing, returning
nw_endpoint_flow_failed_with_error [C4.1.1.1.1 <host>:<port> cancelled channel-flow ((null))] already failing, returning
nw_endpoint_flow_failed_with_error [C5.1.1.1.1 <host>:<port> in_progress channel-flow (satisfied (Path is satisfied), viable, interface: en0[802.11], ipv4, dns, uses wifi)] already failing, returning
nw_endpoint_flow_failed_with_error [C5.1.1.1.1 <host>:<port> cancelled channel-flow ((null))] already failing, returning
Connection 4: received failure notification
Connection 4: failed to connect 3:-9808, reason -1
Connection 4: encountered error(3:-9808)
Task <989D82B0-BD04-4AEE-A581-3552DACD095C>.<8> HTTP load failed, 0/0 bytes (error code: -1202 [3:-9808])
Connection 5: received failure notification
Connection 5: failed to connect 3:-9808, reason -1
Connection 5: encountered error(3:-9808)
Task <383186CF-0D26-4BAA-A2EB-3B3ACC4F9EE8>.<9> HTTP load failed, 0/0 bytes (error code: -1202 [3:-9808])
Task <989D82B0-BD04-4AEE-A581-3552DACD095C>.<8> finished with error [-1202] Error Domain=NSURLErrorDomain Code=-1202 "The certificate for this server is invalid. You might be connecting to a server that is pretending to be “api.ipify.org” which could put your confidential information at risk." UserInfo={NSErrorFailingURLStringKey=http://api.ipify.org/, NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, _kCFStreamErrorDomainKey=3, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <989D82B0-BD04-4AEE-A581-3552DACD095C>.<8>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <989D82B0-BD04-4AEE-A581-3552DACD095C>.<8>"
), NSLocalizedDescription=The certificate for this server is invalid. You might be connecting to a server that is pretending to be “api.ipify.org” which could put your confidential information at risk., NSErrorFailingURLKey=http://api.ipify.org/, NSUnderlyingError=0x600000ddcd50 {Error Domain=kCFErrorDomainCFNetwork Code=-1202 "(null)" UserInfo={_kCFStreamPropertySSLClientCertificateState=0, _kCFNetworkCFStreamSSLErrorOriginalValue=-9808, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9808, _NSURLErrorNWPathKey=satisfied (Path is satisfied), viable, interface: en0[802.11], ipv4, dns, uses wifi}}, _kCFStreamErrorCodeKey=-9808}
Task <383186CF-0D26-4BAA-A2EB-3B3ACC4F9EE8>.<9> finished with error [-1202] Error Domain=NSURLErrorDomain Code=-1202 "The certificate for this server is invalid. You might be connecting to a server that is pretending to be “api.ipify.org” which could put your confidential information at risk." UserInfo={NSErrorFailingURLStringKey=http://api.ipify.org/, NSLocalizedRecoverySuggestion=Would you like to connect to the server anyway?, _kCFStreamErrorDomainKey=3, _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <383186CF-0D26-4BAA-A2EB-3B3ACC4F9EE8>.<9>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalDataTask <383186CF-0D26-4BAA-A2EB-3B3ACC4F9EE8>.<9>"
), NSLocalizedDescription=The certificate for this server is invalid. You might be connecting to a server that is pretending to be “api.ipify.org” which could put your confidential information at risk., NSErrorFailingURLKey=http://api.ipify.org/, NSUnderlyingError=0x600000dddda0 {Error Domain=kCFErrorDomainCFNetwork Code=-1202 "(null)" UserInfo={_kCFStreamPropertySSLClientCertificateState=0, _kCFNetworkCFStreamSSLErrorOriginalValue=-9808, _kCFStreamErrorDomainKey=3, _kCFStreamErrorCodeKey=-9808, _NSURLErrorNWPathKey=satisfied (Path is satisfied), interface: en0[802.11], ipv4, dns, uses wifi}}, _kCFStreamErrorCodeKey=-9808}```

Again, no challenge block is invoked UI-side.

[Alexey Proskuryakov]

This could be rather tricky to support given that networking happens in a different process, but probably not impossible.

[Mateusz Szymański]

(In reply to Alexey Proskuryakov from comment #1)
> This could be rather tricky to support given that networking happens in a
> different process, but probably not impossible.

Any callback that I can process client-side would be appreciated here honestly, not necessarily the block-based approach presented.

[Alex Christensen]

This is not public API, but would _WKWebsiteDataStoreConfiguration.preventsSystemHTTPProxyAuthentication meet your needs here?

[Radar WebKit Bug Importer]

<rdar://problem/152488345>