Bug 63419 – Add encoding parameter to filesystem.py functions

• Component: Tools / Tests
• Status: RESOLVED WONTFIX

[Roland Steiner]

Currently, functions to open/read files largely assume 'utf8' as the encoding to use. This should be override-able by the caller.

[Roland Steiner]

Created attachment 98668 [details]
Patch

[Dominic Cooney]

Comment on attachment 98668 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=98668&action=review

> Tools/Scripts/webkitpy/common/system/filesystem.py:-199
> -    def open_text_file_for_writing(self, path, append=False):

There were no callsites hitherto using append?

> Tools/Scripts/webkitpy/common/system/filesystem_unittest.py:178
> +            fs.write_text_file(text_path, unicode_text_string, encoding='Windows-1252')

Python 2.7 documentation seems to indicate that it is little-w windows… how do you know your encoding is working?

[Tony Chang]

Out of curiosity, what will you be using this for?  I just wonder if it would be simpler (less code) for you to manually do the encoding conversion and use read_binary_file/write_binary_file.

[Dirk Pranke]

Comment on attachment 98668 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=98668&action=review

These interfaces were intentionally designed so that the caller did not have to worry about the encoding, and that if if a file was being passed to the text versions of these routines, we could be fairly certain it was stored on disk as utf-8. I do not want to lose those principles lightly.

Until there's multiple callers that need this functionality, I would prefer it if you just used the read/write_binary_file routines and did the encoding/decoding in the calling routine instead (like Tony suggests). Is that okay?

>> Tools/Scripts/webkitpy/common/system/filesystem.py:-199
>> -    def open_text_file_for_writing(self, path, append=False):
> 
> There were no callsites hitherto using append?

I think that that is correct. I think I added it at one point and ended up not needing it.

[Roland Steiner]

(In reply to comment #2)
> > Tools/Scripts/webkitpy/common/system/filesystem_unittest.py:178
> > +            fs.write_text_file(text_path, unicode_text_string, encoding='Windows-1252')
> 
> Python 2.7 documentation seems to indicate that it is little-w windows… how do you know your encoding is working?

Comparison to the expected binary representation seems to work (after the issues from https://bugs.webkit.org/show_bug.cgi?id=63514 have been corrected).

(In reply to comment #3)
> Out of curiosity, what will you be using this for?  I just wonder if it would be simpler (less code) for you to manually do the encoding conversion and use read_binary_file/write_binary_file.

This is meant to read/write Visual Studion project files for https://bugs.webkit.org/show_bug.cgi?id=61772 which are encoded in "Windows-1252" (using an upper-case "W" ^_-)

(In reply to comment #4)
> (From update of attachment 98668 [details])
> View in context: https://bugs.webkit.org/attachment.cgi?id=98668&action=review
> 
> These interfaces were intentionally designed so that the caller did not have to worry about the encoding, and that if if a file was being passed to the text versions of these routines, we could be fairly certain it was stored on disk as utf-8. I do not want to lose those principles lightly.
> 
> Until there's multiple callers that need this functionality, I would prefer it if you just used the read/write_binary_file routines and did the encoding/decoding in the calling routine instead (like Tony suggests). Is that okay?

Sure can do, although I don't see the drawback here.

> >> Tools/Scripts/webkitpy/common/system/filesystem.py:-199
> >> -    def open_text_file_for_writing(self, path, append=False):
> > 
> > There were no callsites hitherto using append?
> 
> I think that that is correct. I think I added it at one point and ended up not needing it.

Yes, I checked all call-sites to make sure it wasn't used, but forgot to document this part of the change.