Bug 104124 – [Qt] Fix the build if the path contains "~" character

• Status: RESOLVED
• Resolution: INVALID
• Component: Tools / Tests

[Csaba Osztrogonác]

Debian/Ubuntu package names usually contain "~" character and now the build 
is broken if the path contains any tilde with the following error message:

make[3]: execvp: /bin/sh: Argument list too long
make[3]: *** [../../lib/libWebCore.so.1.0.0] Error 127
make[3]: Leaving directory `/home/oszi/WebKit~tilde/WebKitBuild/Release/Source/WebCore'
make[2]: *** [sub-Target-pri-make_first-ordered] Error 2
make[2]: Leaving directory `/home/oszi/WebKit~tilde/WebKitBuild/Release/Source/WebCore'
make[1]: *** [sub-Source-WebCore-WebCore-pro-make_first-ordered] Error 2
make[1]: Leaving directory `/home/oszi/WebKit~tilde/WebKitBuild/Release'
make: *** [incremental] Error 2

[Csaba Osztrogonác]

Created attachment 177750 [details]
failing linking command

[Csaba Osztrogonác]

The size of the linking command is huge - 132095 bytes, 
but it isn't enough reason for this beakage, because

- $/bin/sh ../../../../linking-command.txt -- it works fine, so 
  it can't be command problem because of command length limit
- simple "make" after escaped "~" characters works fine

Maybe make tries to resolve tilde characters inside "WebKit~tilde" strings ...

[Csaba Osztrogonác]

But the root of the problem isn't only the tilde character ...

There is a bug somewhere in the qmake build system, because the objects 
created from generated sources are placed in this strange path:
obj/release/home/oszi/WebKit~tilde/WebKitBuild/Release/Source/WebCore/generated/JSSVGFEMergeElement.o

I think it should be obj/release/generated/JSSVGFEMergeElement.o
without /home/oszi/WebKit~tilde/WebKitBuild/Release/Source/WebCore/

This additional long path is unrelated to tilde character, it is always present.

I have a workaround for this bug: removing "CONFIG += object_parallel_to_source"
line from Tools/qmake/mkspecs/features/unix/default_pre.prf. In this case the
full path isn't added for generated sources. I know it isn't the best fix
because clashing filenames can kill the build. Fortunately there isn't any
clashing filenames in QtWebKit trunk now. But everything is possible in
the future. :)

[Csaba Osztrogonác]

I think the bug isn't is WebKit, but in qmake somewhere in
qtbase/qmake/generators/makefile.cpp near "object_parallel_to_source" 
processing part. I'm going the debug it a little bit.

[Csaba Osztrogonác]

object_parallel_to_source config isn't tested and used at all in qmake,
only WebKit uses this buggy feature now :-/

[Tor Arne Vestbø]

(In reply to comment #3)
> I have a workaround for this bug: removing "CONFIG += object_parallel_to_source"

That's replacing one bug with another, the config is there for a reason. We should find the root cause and fix it.

[Csaba Osztrogonác]

(In reply to comment #6)
> (In reply to comment #3)
> > I have a workaround for this bug: removing "CONFIG += object_parallel_to_source"
> 
> That's replacing one bug with another, the config is there for a reason. We should find the root cause and fix it.

To make it clear: the reason for introducing this object_parallel_to_source
config was to make "GNUmake gcc_MD_depends" configs resistent against moving sources - http://trac.webkit.org/changeset/102452 (and avoiding possible name
clashes inside WebCore/JavascriptCore/WebKit/WebKit2)

I disagree if the workaround is replacing one bug with another one. The build
is totally broken now if the directory contains a tilde character. Removing
this config fixes the build for non-incremental builds. I didn't propose to
remove it, as I mentioned it is only a workaround for this bug. I agree that the object_parallel_to_source config can be useful for developers to avoid
incremental build issues and to reduce the likelihood of different problems
because of clashing filenames.

Of course we should fix the bugs instead of working around them. 
I think we need to following fixes:
- make qmake not to use buggy deep paths for "object_parallel_to_source" config
- make qmake escape all ~ in object/source name not to confuse make/sh/bash/...

[Simon Hausmann]

(In reply to comment #7)
> (In reply to comment #6)
> > (In reply to comment #3)
> > > I have a workaround for this bug: removing "CONFIG += object_parallel_to_source"
> > 
> > That's replacing one bug with another, the config is there for a reason. We should find the root cause and fix it.
> 
> To make it clear: the reason for introducing this object_parallel_to_source
> config was to make "GNUmake gcc_MD_depends" configs resistent against moving sources - http://trac.webkit.org/changeset/102452 (and avoiding possible name
> clashes inside WebCore/JavascriptCore/WebKit/WebKit2)
> 
> I disagree if the workaround is replacing one bug with another one. The build
> is totally broken now if the directory contains a tilde character. Removing
> this config fixes the build for non-incremental builds. I didn't propose to
> remove it, as I mentioned it is only a workaround for this bug. I agree that the object_parallel_to_source config can be useful for developers to avoid
> incremental build issues and to reduce the likelihood of different problems
> because of clashing filenames.

It's the same on the bot, isn't it? AFAICS without object_parallel_to_source even on the bots we're likely to get false positive builds, i.e. a build that seemingly succeeds but actually failed to recompile a file or re-link a library and therefore produces a binary that causes test failures.

Those kind of failures we definitely want to avoid at all cost.

[Csaba Osztrogonác]

It can't be problem on the bots now, because "GNUmake gcc_MD_depends" is
disabled by the configure test because of the icecream bug. Nowadays all stable Debian/Ubuntu distribution uses this buggy icecc. As far as I see, Ubuntu 13.04 will use the fixed version. (but I haven't tried) (Additionally GNUmake config is still disabled explicitly on the bots, because I had to disable it before we had configure test.)

But let me check first if the real bug is fixable in qmake easily. ;)

[Csaba Osztrogonác]

I made some digging around this bug - part1 (very deep paths). The bug is in
qmake obviously and related to shadow build - for example build-webkit builds into WebKitBuild instead of root of the source tree. So it isn't so
important now, because it won't cause problems for production builds. But
I'm going to fix qmake soon.

[Jocelyn Turcotte]

=== Bulk closing of Qt bugs ===

If you believe that this bug report is still relevant for a non-Qt port of webkit.org, please re-open it and remove [Qt] from the summary.

If you believe that this is still an important QtWebKit bug, please fill a new report at https://bugreports.qt-project.org and add a link to this issue. See http://qt-project.org/wiki/ReportingBugsInQt for additional guidelines.