We have identified an issue with our WebKit that prevents it from accessing websites on a fresh Windows installation. This issue can be reproduced as follows: 1. Create a new Windows Server 2022 VM or Windows 11 2. Run MiniBrowser.exe 3. Navigate to https://playwright.dev Expected Result: It works Actual Result: Server Trust Evaluation Request: Unable to get local issuer certificate Interestingly, if you run `curl https://playwright.dev` between steps 2 and 3, it works. This indicates that running cURL modifies the system in a way that allows our libcurl-based WebKit to function correctly. Investigation revealed that our WebKit is using the Windows certificate manager for SSL (schannel). However, normal cURL on Windows (non-windows-shipped build) uses OpenSSL. If you use this build, with CURL_SSL_BACKEND=schannel env var, it behaves the same as windows-shipped cURL build. -> Based on that we know that this is something in cURL (and not special Microsoft build magic) or how they talk to the schannel api. The root certificates are not installed when the machine is fresh. However, when curl is invoked, it seems to add the certificate lazily, thus making it available for subsequent connections (like MiniBrowser.exe). To see the Root certificates on Windows, open 'certlm.exe' -> 'Trusted Root Certification Authorities' -> 'Certificates' and find 'ISRG Root X1' there, delete it to reproduce the issue over and over again. Run MiniBrowser.exe to see the error, and curl so that the certificate gets added. Questions for further investigation: - What specific steps does cURL take that enable it to function properly on a fresh Windows installation? - Why does curl.exe behave differently to our libcurl instruction? I found https://github.com/curl/curl/blob/d135d040df5b276df5736688eba88d150b0d8a57/lib/vtls/schannel_verify.c#L182C15-L182C47 which looks related. Downstream issue: https://github.com/microsoft/playwright/issues/24512
Interesting. But, the schannel backend behavior seems weird to me. If it trusts and imports any root certs, it's too dangerous. How does it determine 'ISRG Root X1' is trustworthy?
Maybe its related to https://github.com/WebKitForWindows/WebKitRequirements/blob/2a339560f9413b0b667b5c4902189f960a883d9c/ports/curl/portfile.cmake#L81C23-L82, will try to compile my own WebKitRequirements tomorrow and play with the options. > If it trusts and imports any root certs, it's too dangerous. How does it determine 'ISRG Root X1' is trustworthy? Absolutely, there needs to be much more related to this.
Update: As of today we do in WebKitRequirements: vcpkg.exe install curl[libressl,http3,ipv6] --triplet x64-windows-webkit which does not work, but I can confirm that the following works: vcpkg.exe install curl[ipv6] --triplet x64-windows-webkit -> libressl breaks SSL handling here.
@Fujii What is the SSL strategy for the Windows builds in terms of who takes care? Sounds like since we stick with Libressl in curl, we should / could ship a CA bundle? Alternatively we could switch to schannel and let Windows handle it which works too and is the default in curl (Microsoft windows builds) and cURL for Windows.
> To see the Root certificates on Windows, open 'certlm.exe' -> 'Trusted Root Certification Authorities' -> 'Certificates' and find 'ISRG Root X1' there, delete it to reproduce the issue over and over again. Run MiniBrowser.exe to see the error, and curl so that the certificate gets added. > How does it determine 'ISRG Root X1' is trustworthy? The Microsoft Root Certificate Program was automatically updating the root certificates. When I disabled this feature, the certificate was not added. And I got the error "Unable to get local issuer's certificate". https://learn.microsoft.com/en-us/windows-server/identity/ad-cs/certificate-trust
This is just my guess, but I think that if you use schannel, you will not get a root certificate error because communication is done via the feature. On the other hand, curl + libressl will not go through the feature, so an error may occur.
Great discovery, for further note: you can disable/enable this feature via: 1. Type gpedit.msc, and click OK. 2. Computer Configuration > Administrative Templates > System > Internet Communication Management, and then click Internet Communication Settings. On the Internet Communication Settings page, double-click Turn off Automatic Root Certificates Update. 3. Set to enabled -> then curl (aka. schannel) won't work either anymore. The question still stands if we want to switch our backend to schannel or ship a own CA bundle.
WebKit's curl port is built with openssl (libressl) as the SSL backend. For example, there is code using those APIs in CurllSSLVerifier.cpp and OpenSSLHelper.cpp. So switching to schannel doesn't sound like a good idea.
Looks like a workaround running this file, thats why we didn't experience it on GitHub Actions: https://github.com/actions/runner-images/blob/4ec9fdae13222a0b2b278ae1ddcdc9e8b44901ed/images/win/scripts/Installers/Install-RootCA.ps1
We want to use the OpenSSL backend because it's pretty cross-platform, while the schannel backend is Windows specific, to reduce the maintenance cost of the curl integration code. Bundling root CA certs seems to make little sense comparing to manually updating the Windows certs store, using a script like Install-RootCA.ps1 or invoking schannel-backed `curl` command. But, WinCairo is a OpenSource project, we have to accept your patch if you want to maintain.
<rdar://problem/113674881>