The error is here: https://bugs.webkit.org/show_bug.cgi?id=37981#c6 I have non-utf8 data in a layout test result (0xa0, a whitespace character). I think this is quite common. I'm not sure what the right fix is here. You could also imagine layout tests that are not utf8.
The right fix is to treat patch files a binary data, since I now realize they have no encoding. I'll fix the callers and post a patch.
This seems wrong. I agree with you that patch files are binary. But in this patch, it appears the -expected.txt file is not utf-8? All output from DumpRenderTree is utf-8 as far as I know. So how did you generate this expected.txt file and why wouldn't it be utf-8? https://bug-37981-attachments.webkit.org/attachment.cgi?id=54123
I have a patch to fix our handling of patch files. Finishing testing now.
Created attachment 54210 [details] Patch
Comment on attachment 54210 [details] Patch WebKitTools/Scripts/webkitpy/common/checkout/scm.py:503 + # as we're passing --name-status which does not output any data. This comment seems gratuitously different than the others. Also, technically, file names (on unix) might not be UTF8 because they are just bytes. I don't think we should worry about that here though. Just more background for the trail of tears. Nice!
Comment on attachment 54210 [details] Patch Rejecting patch 54210 from commit-queue. Failed to run "['WebKitTools/Scripts/test-webkitpy']" exit_code: 1 Last 500 characters of output: cripts/webkitpy/common/prettypatch.py", line 42, in pretty_diff_file pretty_diff = self.pretty_diff(diff) File "/Users/eseidel/Projects/CommitQueue/WebKitTools/Scripts/webkitpy/common/prettypatch.py", line 60, in pretty_diff return self._executive.run_command(args, input=diff, decode_output=False) TypeError: run_command() got an unexpected keyword argument 'decode_output' ---------------------------------------------------------------------- Ran 385 tests in 5.007s FAILED (errors=3) Full output: http://webkit-commit-queue.appspot.com/results/1856073
Committed r58210: <http://trac.webkit.org/changeset/58210>