223269 2021-03-16 12:09:21 -0700 webKit-patch fails to upload patch with 20000 files changed due to shell limitation: Argument list too long 2021-03-23 12:10:12 -0700 1 1 1 Unclassified WebKit Tools / Tests Other Unspecified Unspecified NEW https://bugs.webkit.org/show_bug.cgi?id=222964 https://bugs.webkit.org/show_bug.cgi?id=214485 InRadar P2 Normal --- 1 kkinnunen webkit-unassigned ddkilzer jbedard webkit-bug-importer oldest_to_newest 1740102 0 kkinnunen 2021-03-16 12:09:21 -0700 WebKit patch fails to upload patch with 20000 files git show --summary |wc -l 22512 While large amount, it's not inconceivable. The patch in question tried to introduce new WebGL conformance test suites, e.g. replace LayoutTests/{1.0.3,2.0.0} -> {1.0.4,2.0.1}. The WebGL conformance test suite has ~2500 testcases. WebKit generates one test driver .html and one -expected.txt for each test. kkinnunen@Kimmos-iMac-Pro OpenSource % Tools/Scripts/webkit-patch upload -g HEAD Traceback (most recent call last): File "Tools/Scripts/webkit-patch", line 80, in <module> main() File "Tools/Scripts/webkit-patch", line 75, in main WebKitPatch(os.path.abspath(__file__)).main() File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 305, in main result = command.check_arguments_and_execute(options, args, self) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 123, in check_arguments_and_execute return self.execute(options, args, tool) or 0 File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/commands/abstractsequencedcommand.py", line 55, in execute self._sequence.run_and_handle_errors(tool, options, state) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 73, in run_and_handle_errors self._run(tool, options, state) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 67, in _run step(tool, options).run(state) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/steps/checkstyle.py", line 66, in run self._tool.executive.run_and_throw_if_fail(self._tool.deprecated_port().check_webkit_style_command() + args, cwd=self._tool.scm().checkout_root) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/common/system/executive.py", line 148, in run_and_throw_if_fail exit_code = self._run_command_with_teed_output(args, child_stdout, **kwargs) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/common/system/executive.py", line 123, in _run_command_with_teed_output **kwargs) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/common/system/executive.py", line 532, in popen result = subprocess.Popen(string_args, env=env, **kwargs) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 394, in __init__ errread, errwrite) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1047, in _execute_child raise child_exception OSError: [Errno 7] Argument list too long kkinnunen@Kimmos-iMac-Pro OpenSource % tools/Scripts/webkit-patch upload -g HEAD --ignore-style Traceback (most recent call last): File "tools/Scripts/webkit-patch", line 80, in <module> main() File "tools/Scripts/webkit-patch", line 75, in main WebKitPatch(os.path.abspath(__file__)).main() File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 305, in main result = command.check_arguments_and_execute(options, args, self) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 123, in check_arguments_and_execute return self.execute(options, args, tool) or 0 File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/commands/abstractsequencedcommand.py", line 55, in execute self._sequence.run_and_handle_errors(tool, options, state) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 73, in run_and_handle_errors self._run(tool, options, state) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 67, in _run step(tool, options).run(state) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/steps/confirmdiff.py", line 73, in run diff = self.cached_lookup(state, "diff") File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/steps/abstractstep.py", line 62, in cached_lookup state[key] = promise(self, state) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/tool/steps/abstractstep.py", line 52, in <lambda> "diff": lambda self, state: self._tool.scm().create_patch(self._options.git_commit, changed_files=self._changed_files(state)), File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/common/checkout/scm/git.py", line 375, in create_patch return self.prepend_svn_revision(self.run(command, decode_output=False, cwd=self.checkout_root)) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/common/checkout/scm/scm.py", line 80, in run decode_output=decode_output) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/common/system/executive.py", line 406, in run_command close_fds=self._should_close_fds()) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/common/system/executive.py", line 532, in popen result = subprocess.Popen(string_args, env=env, **kwargs) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 394, in __init__ errread, errwrite) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 1047, in _execute_child raise child_exception OSError: [Errno 7] Argument list too long kkinnunen@Kimmos-iMac-Pro OpenSource % tools/Scripts/check-webkit-style -g HEAD Traceback (most recent call last): File "tools/Scripts/check-webkit-style", line 46, in <module> sys.exit(CheckWebKitStyle().main()) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/style/main.py", line 159, in main patch_checker.check(patch) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/webkitpy/style/patchreader.py", line 60, in check patch_string = string_utils.decode(patch_string, target_type=str) File "/Users/kkinnunen/WebKit/OpenSource/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/string_utils.py", line 44, in decode return data.decode(encoding, errors=errors) UnicodeDecodeError: 'utf-8' codec can't decode byte 0x82 in position 3450534: invalid start byte 1740110 1 ddkilzer 2021-03-16 12:17:08 -0700 We should probably break the check-webkit-style issue out into its own bug, since it has a different cause (unless we use this as an umbrella bug). CC Jonathan just FYI. 1740163 2 jbedard 2021-03-16 13:40:56 -0700 Not sure how worth fixing this issue is, we'll be moving to pull-requests in the near future, which is the architectural fix for this sort of problem. 1740542 3 kkinnunen 2021-03-17 01:31:32 -0700 Filed bug 223367 about the check-webkit-style, which probably is not going away. 1742937 4 webkit-bug-importer 2021-03-23 12:10:12 -0700 <rdar://problem/75748788>