227118 2021-06-17 04:53:02 -0700 [Curl] Add curl option CURLOPT_NOSIGNAL to omit numerous sigaction calls 2021-06-20 18:18:51 -0700 1 1 1 Unclassified WebKit WebCore Misc. WebKit Nightly Build Other Other RESOLVED FIXED InRadar P2 Minor --- 1 sojulibra webkit-unassigned basuke chris.reid don.olmstead ews-watchlist fujii galpeter stephan.szabo takashi.komori webkit-bug-importer oldest_to_newest 1770591 0 sojulibra 2021-06-17 04:53:02 -0700 When large number of files are added to download queue at once, the download speed gets significantly slower on a platform we are working for. We found that the most part of time are consumed for sigaction system call in curl. - https://github.com/curl/curl/blob/84d2839740ca78041ac7419d9aaeac55c1e1c729/lib/sigpipe.h#L52 - https://github.com/curl/curl/blob/84d2839740ca78041ac7419d9aaeac55c1e1c729/lib/sigpipe.h#L56 - https://github.com/curl/curl/blob/84d2839740ca78041ac7419d9aaeac55c1e1c729/lib/sigpipe.h#L69 The number of times of sigaction call becomes enormously as we increase file handles registered by curl_multi_add_handle(). Here is the test results that show the total count of sigaction calls with different number of file handles. Each file size is 64KB. * 512 files (32MB in total): 348144 times (of sigaction call) * 1024 files (64MB in total): 1344192 times * 2048 files (128MB in total): 5300697 times * 4096 files (256MB in total): 21088311 times It seems the times of sigaction-call grow along O(N^2) where N is the number of file handles. The sigaction is there to ignore SIGPIPE signals that is triggered when the other end of socket are closed. However for the platform we are targeting, we have little concern about the unexpected signals since the socket are configured with SO_NOSIGPIPE option with setsockopt(), and use send() with MSG_NOSIGNAL as its forth argument. - https://github.com/curl/curl/blob/ee97f176970c9667bfbdaab89bc637e77099d1df/lib/connect.c#L1085-L1086 - https://github.com/curl/curl/blob/ee97f176970c9667bfbdaab89bc637e77099d1df/lib/curl_setup_once.h#L116 Also curl with c-ares or threaded async DNS resolver doesn't rely on SIGALRM signal to detect timeouts. The cost of syscall is relatively high because of user-kernel context switching. These sigaction can be avoided by a curl option, CURLOPT_NOSIGNAL (https://curl.se/libcurl/c/CURLOPT_NOSIGNAL.html). With the option enabled, curl omits to install signal handlers by assuming no signals triggered for the process. 1770941 1 431756 sojulibra 2021-06-17 23:12:25 -0700 Created attachment 431756 Patch 1771058 2 fujii 2021-06-18 13:17:52 -0700 Why does curl block SIGPIPE even on platforms supporting MSG_NOSIGNAL and SO_NOSIGPIPE? Increasing sigaction calls as O(N^2) seems like a curl bug. 1771062 3 ews-feeder 2021-06-18 13:41:44 -0700 Committed r279046 (238966@main): <https://commits.webkit.org/238966@main> All reviewed patches have been landed. Closing bug and clearing flags on attachment 431756. 1771063 4 webkit-bug-importer 2021-06-18 13:42:31 -0700 <rdar://problem/79512069> 1771204 5 sojulibra 2021-06-20 18:18:51 -0700 Not all systems provide enough way to avoid SIGPIPE. SO_NOSIGPIPE socket option is available in BSD based OS. MSG_NOSIGNAL flag has been brought in since Linux 2.2. Thank you for your review. 431756 2021-06-17 23:12:25 -0700 2021-06-18 13:41:45 -0700 Patch bug-227118-20210618074803.patch text/plain 1637 sojulibra SW5kZXg6IFNvdXJjZS9XZWJDb3JlL0NoYW5nZUxvZwo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBTb3VyY2UvV2Vi Q29yZS9DaGFuZ2VMb2cJKHJldmlzaW9uIDI3OTAzMCkKKysrIFNvdXJjZS9XZWJDb3JlL0NoYW5n ZUxvZwkod29ya2luZyBjb3B5KQpAQCAtMSwzICsxLDIzIEBACisyMDIxLTA2LTE4ICBTb3VqeXUg VEFOQUtBICA8c291anl1LnRhbmFrYUBhY2Nlc3MtY29tcGFueS5jb20+CisKKyAgICAgICAgW0N1 cmxdIEFkZCBjdXJsIG9wdGlvbiBDVVJMT1BUX05PU0lHTkFMIHRvIG9taXQgbnVtZXJvdXMgc2ln YWN0aW9uIGNhbGxzCisgICAgICAgIGh0dHBzOi8vYnVncy53ZWJraXQub3JnL3Nob3dfYnVnLmNn aT9pZD0yMjcxMTgKKworICAgICAgICBSZXZpZXdlZCBieSBOT0JPRFkgKE9PUFMhKS4KKworICAg ICAgICBBcyB3ZSBpbmNyZWFzZSBmaWxlIGhhbmRsZXMgcmVnaXN0ZXJlZCBieSBjdXJsX211bHRp X2FkZF9oYW5kbGUoKSwKKyAgICAgICAgbnVtYmVyIG9mIHRpbWVzIG9mIHNpZ2FjdGlvbiBleGVj dXRpb25zIGluIEN1cmwgZ3Jvd3MgcmFwaWRseS4gVGhhdCBjYW4KKyAgICAgICAgY2F1c2UgdW5p Z25vcmFibGUgZG93bmxvYWQgc3BlZWQgZHJvcC4KKworICAgICAgICBXaXRoIGEgY3VybCBvcHRp b24gQ1VSTE9QVF9OT1NJR05BTCBlbmFibGVkLCBDdXJsIGRvZXNuJ3QgdXNlIHRoZQorICAgICAg ICBzeXNjYWxsIHRvIGlnbm9yZSBTSUdQSVBFIHNpZ25hbHMuIEl0J3Mgc2FmZSBhcyBsb25nIGFz IHdlIGNvbmZpZ3VyZSB0bworICAgICAgICBwcmV2ZW50IFNJR1BJUEUgZnJvbSBiZWluZyB0cmln Z2VyZWQuCisKKyAgICAgICAgTm8gbmV3IHRlc3RzLgorCisgICAgICAgICogcGxhdGZvcm0vbmV0 d29yay9jdXJsL0N1cmxDb250ZXh0LmNwcDoKKyAgICAgICAgKFdlYkNvcmU6OkN1cmxIYW5kbGU6 OkN1cmxIYW5kbGUpOgorCiAyMDIxLTA2LTE3ICBTYWlkIEFib3UtSGFsbGF3YSAgPHNhaWRAYXBw bGUuY29tPgogCiAgICAgICAgIFtDb2NvYV0gRGlzYWJsZSBoYXJkd2FyZSBkZWNvZGluZyBpbiB0 aGUgV2ViUHJvY2VzcwpJbmRleDogU291cmNlL1dlYkNvcmUvcGxhdGZvcm0vbmV0d29yay9jdXJs L0N1cmxDb250ZXh0LmNwcAo9PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBTb3VyY2UvV2ViQ29yZS9wbGF0Zm9ybS9u ZXR3b3JrL2N1cmwvQ3VybENvbnRleHQuY3BwCShyZXZpc2lvbiAyNzg4MjMpCisrKyBTb3VyY2Uv V2ViQ29yZS9wbGF0Zm9ybS9uZXR3b3JrL2N1cmwvQ3VybENvbnRleHQuY3BwCSh3b3JraW5nIGNv cHkpCkBAIC0yODIsNiArMjgyLDcgQEAgQ3VybEhhbmRsZTo6Q3VybEhhbmRsZSgpCiB7CiAgICAg bV9oYW5kbGUgPSBjdXJsX2Vhc3lfaW5pdCgpOwogICAgIGN1cmxfZWFzeV9zZXRvcHQobV9oYW5k bGUsIENVUkxPUFRfRVJST1JCVUZGRVIsIG1fZXJyb3JCdWZmZXIpOworICAgIGN1cmxfZWFzeV9z ZXRvcHQobV9oYW5kbGUsIENVUkxPUFRfTk9TSUdOQUwsIDFMKTsKIAogICAgIGVuYWJsZVNoYXJl SGFuZGxlKCk7CiAgICAgZW5hYmxlQWxsb3dlZFByb3RvY29scygpOwo=