Bug 63234 – line ending strip suggestion in KeywordLookupGenerator.py

• Component: Tools / Tests
• Status: RESOLVED FIXED

[Cameron Wong]

Hi, I am trying to build Webkit in Cygwin. The source is checkout with TortoiseSVN. When I run build-webkit, KeywordLookupGenerator.py throws error of "expected description ending with @end". I think it's a line ending issue. I suggest making a small improvement in this script:

In about line 68:
    if not terminator == "@end"

Change it to:
    if not terminator.rstrip() == "@end"

It works for me currently although I am trying to fix another compiling issues...

Thanks a lot!

Cameron Wong

[Alexey Proskuryakov]

Does making this change alone fix the build for you? <http://www.webkit.org/building/checkout.html> says that one should use cygwin version of svn, and I thought that a lot would break if TortoiseSVN (or another client that changes line endings) were used.

[Adam Roben (:aroben)]

I think it is highly likely that there are lots of line-ending-related issue to be found if you use a non-Cygwin version of Subversion. But it would be nice to make it work! Peter Kasting did some work in this area a while back; he may know more.

[Peter Kasting]

In general I think it's a good things to make scripts able to handle both CR and CRLF.  I fixed enough issues long ago that I think there shouldn't be all that many remaining, but I would imagine they also creep in over time.

I'm not going to post a patch for this particular case because I don't know python, though.

[Cameron Wong]

> Does making this change alone fix the build for you?

No. Besides adding rstrip() in KeywordLookupGenerator.py, I also added following setting in Cygwin.bat before launching bash. This can fix the issue of failing to run shell scripts before of bad line ending.
    set SHELLOPTS=igncr

Then, I got error of non-ASCII character in:
    Source\WebCore\platform\DefaultLocalizationStrategy.cpp

I changed the non-ASCII string into English. There is another file (forget the filename) that is non-ASCII character, but I could't find with my eye. So I re-saved that file in ASCII-only format.

Now `build-webkit --debug` is successfully finish running although I am trying to figuring out how to `debug-safari`.

[Peter Kasting]

(In reply to comment #4)
> Then, I got error of non-ASCII character in:
>     Source\WebCore\platform\DefaultLocalizationStrategy.cpp
> 
> I changed the non-ASCII string into English. There is another file (forget the filename) that is non-ASCII character, but I could't find with my eye. So I re-saved that file in ASCII-only format.

Can you file a separate bug for these cases?  We should fix these regardless; embedded Unicode (or other encodings) can cause various problems for particular build environments.

[Cameron Wong]

> Then, I got error of non-ASCII character in:
>     Source\WebCore\platform\DefaultLocalizationStrategy.cpp
>Can you file a separate bug for these cases?

I've open a ticket for it
https://bugs.webkit.org/show_bug.cgi?id=63375

[Justin Haygood]

Created attachment 196798 [details]
Patch

[Justin Haygood]

Comment on attachment 196798 [details]
Patch

Requesting commit

[WebKit Commit Bot]

Comment on attachment 196798 [details]
Patch

Clearing flags on attachment: 196798

Committed r147934: <http://trac.webkit.org/changeset/147934>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[Siddhu]

I was facing the same bug in Ubuntu, even after the patch for cygwin.

In my case the CR-LF line endings were added to all the files in the repo.

So I faced this problem in shebang for the build script file too.

Temporarily resolved the issue by commenting the check for cygwin.

Will converting the CR-LF to LF work on windows?

In that case, there won't be any need for the platform check, i.e. regardless of the platform, the CR-LF, if present, will be converted to LF.