apply-watchlist fails for patches with non-unicode characters

• Component: Tools / Tests
• Status: ASSIGNED ---

[Aakash Jain]

apply-watchlist fails for patches with non-unicode characters, both with python2 and python3. e.g.:

e.g.:
https://ews-build.webkit-uat.org/#/builders/2/builds/115 (python 2)
https://ews-build.webkit-uat.org/#/builders/2/builds/114 (python 3)

Traceback (most recent call last):
  File "Tools/Scripts/webkit-patch", line 80, in <module>
    main()
  File "Tools/Scripts/webkit-patch", line 75, in main
    WebKitPatch(os.path.abspath(__file__)).main()
  File "/Volumes/Data/worker/Apply-WatchList-EWS/build/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 305, in main
    result = command.check_arguments_and_execute(options, args, self)
  File "/Volumes/Data/worker/Apply-WatchList-EWS/build/Tools/Scripts/webkitpy/tool/multicommandtool.py", line 123, in check_arguments_and_execute
    return self.execute(options, args, tool) or 0
  File "/Volumes/Data/worker/Apply-WatchList-EWS/build/Tools/Scripts/webkitpy/tool/commands/abstractsequencedcommand.py", line 55, in execute
    self._sequence.run_and_handle_errors(tool, options, state)
  File "/Volumes/Data/worker/Apply-WatchList-EWS/build/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 73, in run_and_handle_errors
    self._run(tool, options, state)
  File "/Volumes/Data/worker/Apply-WatchList-EWS/build/Tools/Scripts/webkitpy/tool/commands/stepsequence.py", line 67, in _run
    step(tool, options).run(state)
  File "/Volumes/Data/worker/Apply-WatchList-EWS/build/Tools/Scripts/webkitpy/tool/steps/applywatchlist.py", line 48, in run
    cc_and_messages = self._tool.watch_list().determine_cc_and_messages(diff)
  File "/Volumes/Data/worker/Apply-WatchList-EWS/build/Tools/Scripts/webkitpy/common/watchlist/watchlist.py", line 71, in determine_cc_and_messages
    definitions = self.find_matching_definitions(diff)
  File "/Volumes/Data/worker/Apply-WatchList-EWS/build/Tools/Scripts/webkitpy/common/watchlist/watchlist.py", line 40, in find_matching_definitions
    patch_files = DiffParser(diff.splitlines()).files
  File "/Volumes/Data/worker/Apply-WatchList-EWS/build/Tools/Scripts/webkitpy/common/checkout/diff_parser.py", line 148, in __init__
    self.files = self._parse_into_diff_files(diff_input)
  File "/Volumes/Data/worker/Apply-WatchList-EWS/build/Tools/Scripts/webkitpy/common/checkout/diff_parser.py", line 159, in _parse_into_diff_files
    line = string_utils.decode(line).rstrip("\n")
  File "/Volumes/Data/worker/Apply-WatchList-EWS/build/Tools/Scripts/libraries/webkitcorepy/webkitcorepy/string_utils.py", line 46, in decode
    return data.decode(encoding, errors=errors)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xcf in position 18: invalid continuation byte

[Jonathan Bedard]

Seems like we might need to parse this as bytes, then, which is what I was trying to avoid.

[Radar WebKit Bug Importer]

<rdar://problem/82975130>