Bug 79498 – [GTK] Webkit-gtk-1.7.90 tarball fails to build with MAKEOPTS=-jN

• Component: WebKit EFL
• Status: RESOLVED FIXED

[Priit Laes (IRC: plaes)]

Trying to build webkit-1.7.90 tarball with MAKEOPTS=-jN (N=3 in my case) fails with various errors...

[Priit Laes (IRC: plaes)]

Created attachment 128744 [details]
build.log

  GEN    Programs/jsc
cp: cannot stat `Programs/jsc-3': No such file or directory
make[1]: *** [Programs/jsc] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory `/home/tmp/portage/net-libs/webkit-gtk-1.7.90-r300/work/webkit-1.7.90'


I've also seen a different error complaining about WebKit-3.0.gir.

[Sergio Villar Senin]

(In reply to comment #1)
> Created an attachment (id=128744) [details]
> build.log
> 
>   GEN    Programs/jsc
> cp: cannot stat `Programs/jsc-3': No such file or directory

I can confirm that I always get this one with current trunk in both Debug and Release builds.

[Martin Robinson]

(In reply to comment #2)
> (In reply to comment #1)
> > Created an attachment (id=128744) [details] [details]
> > build.log
> > 
> >   GEN    Programs/jsc
> > cp: cannot stat `Programs/jsc-3': No such file or directory
> 
> I can confirm that I always get this one with current trunk in both Debug and Release builds.

This is very unusual. The rule for this target is:


Programs/jsc$(EXEEXT): Programs/jsc-@WEBKITGTK_API_MAJOR_VERSION@$(EXEEXT)
    $(AM_V_GEN)cp -f Programs/jsc-@WEBKITGTK_API_MAJOR_VERSION@$(EXEEXT) Programs/jsc$(EXEEXT)
Programs_jsc_LDADD =
Programs_jsc_SOURCES =

[Martin Robinson]

Does this bug still exist when you do not pass "--disable-dependency-tracking" ?

[Priit Laes (IRC: plaes)]

(In reply to comment #4)
> Does this bug still exist when you do not pass "--disable-dependency-tracking" ?

I actually have dependency tracking enabled (the disable variant is added automatically by portage, so I just enable it in again):
[snip]
./configure --disable-dependency-tracking ... --enable-dependency-tracking
[/snip]

With '--disable-dependency-tracking' it would fail at first few targets...

[Priit Laes (IRC: plaes)]

Just an idea (sorry, bedtime):

$(AM_V_GEN)cp -f $(builddir)/Programs/jsc-@WEBKITGTK_API_MAJOR_VERSION@$(EXEEXT) $(builddir)/Programs/jsc$(EXEEXT)

[Martin Robinson]

(In reply to comment #6)
> Just an idea (sorry, bedtime):
> 
> $(AM_V_GEN)cp -f $(builddir)/Programs/jsc-@WEBKITGTK_API_MAJOR_VERSION@$(EXEEXT) $(builddir)/Programs/jsc$(EXEEXT)

Would be great if one of you who have this problem could confirm whether or not this fixes the issue.

[Priit Laes (IRC: plaes)]

(In reply to comment #7)
> (In reply to comment #6)
> > Just an idea (sorry, bedtime):
> > 
> > $(AM_V_GEN)cp -f $(builddir)/Programs/jsc-@WEBKITGTK_API_MAJOR_VERSION@$(EXEEXT) $(builddir)/Programs/jsc$(EXEEXT)
> 
> Would be great if one of you who have this problem could confirm whether or not this fixes the issue.

Didn't work, apparently jsc hasn't yet been built :

sol webkit-1.7.90 # find . |grep jsc |egrep -v '.(Po|Plo|cpp|h)$'
./Source/WebCore/bridge/jni/jsc
./Source/WebCore/bridge/jni/jsc/.dirstamp
./Source/WebCore/bridge/jni/jsc/.deps
./Source/WebCore/bridge/jni/jsc/.deps/.dirstamp
./Source/WebCore/bridge/jsc
./Source/WebCore/bridge/jsc/.dirstamp
./Source/WebCore/bridge/jsc/.deps
./Source/WebCore/bridge/jsc/.deps/.dirstamp

[Alexandre Rostovtsev]

Created attachment 128879 [details]
build log showing WebKit-1.0.typelib parallel make failure

Here is a build log with MAKEOPTS="-j5 V=1 --debug" showing a parallel make failure when generating WebKit-1.0.typelib for a gtk2 build of webkit-gtk-1.7.90.

  File `WebKit-1.0.gir' does not exist.
   File `JSCore-1.0.gir' does not exist.
  Must remake target `JSCore-1.0.gir'.
Invoking recipe from GNUmakefile:70085 to update target `JSCore-1.0.gir'.
cp ./Source/WebKit/gtk/JSCore-1.0.gir ./
  File `Programs/resources/inspector/inspector.html' does not exist.
 Must remake target `Programs/resources/inspector/inspector.html'.
Invoking recipe from GNUmakefile:70077 to update target `Programs/resources/inspector/inspector.html'.
mkdir -p ./Programs/resources/inspector/UglifyJS
  File `JSCore-1.0.typelib' does not exist.
 Must remake target `JSCore-1.0.typelib'.
Invoking recipe from GNUmakefile:70120 to update target `JSCore-1.0.typelib'.
/usr/bin/g-ir-compiler --includedir ./Source/WebKit/gtk --includedir . JSCore-1.0.gir -o JSCore-1.0.typelib
  File `WebKit-1.0.typelib' does not exist.
 Must remake target `WebKit-1.0.typelib'.
mv -f DerivedSources/WebCore/.deps/libWebCoreInternals_la-JSInternals.Tpo DerivedSources/WebCore/.deps/libWebCoreInternals_la-JSInternals.Plo
mkdir -p ./Programs/resources/inspector/images
Invoking recipe from GNUmakefile:70120 to update target `WebKit-1.0.typelib'.
/usr/bin/g-ir-compiler --includedir ./Source/WebKit/gtk --includedir . WebKit-1.0.gir -o WebKit-1.0.typelib
[...]
error parsing file WebKit-1.0.gir: Failed to open file 'WebKit-1.0.gir': No such file or directory
make[1]: *** [WebKit-1.0.typelib] Error 1

[Philippe Normand]

Also happens in ToT, quite often for me :( This is annoying I have to stop the EWS.

[Philippe Normand]

(In reply to comment #10)
> Also happens in ToT, quite often for me :( This is annoying I have to stop the EWS.

If I disable WebKit2 build, jsc-3 is created but the build fails later on:

  CXXLD  Programs/jsc-3
  CXXLD  libjavascriptcoregtk-3.0.la
  CXXLD  Libraries/libTestRunnerInjectedBundle.la
libtool: link: cannot find the library `libjavascriptcoregtk-3.0.la' or unhandled argument `libjavascriptcoregtk-3.0.la'
make[1]: *** [Programs/minidom] Error 1
make[1]: *** Waiting for unfinished jobs....
libtool: link: cannot find the library `libjavascriptcoregtk-3.0.la' or unhandled argument `libjavascriptcoregtk-3.0.la'
make[1]: *** [Programs/jsc-3] Error 1
make[1]: Leaving directory `/home/phil/gst/jhbuild/build/WebKit/WebKitBuild/tmp/Release'
make: *** [all] Error 2

Could this whole issue be related with r108523 ?

[Philippe Normand]

(In reply to comment #11)
> (In reply to comment #10)
> > Also happens in ToT, quite often for me :( This is annoying I have to stop the EWS.
> 
> If I disable WebKit2 build, jsc-3 is created but the build fails later on:
> 
>   CXXLD  Programs/jsc-3
>   CXXLD  libjavascriptcoregtk-3.0.la
>   CXXLD  Libraries/libTestRunnerInjectedBundle.la
> libtool: link: cannot find the library `libjavascriptcoregtk-3.0.la' or unhandled argument `libjavascriptcoregtk-3.0.la'
> make[1]: *** [Programs/minidom] Error 1
> make[1]: *** Waiting for unfinished jobs....
> libtool: link: cannot find the library `libjavascriptcoregtk-3.0.la' or unhandled argument `libjavascriptcoregtk-3.0.la'
> make[1]: *** [Programs/jsc-3] Error 1
> make[1]: Leaving directory `/home/phil/gst/jhbuild/build/WebKit/WebKitBuild/tmp/Release'
> make: *** [all] Error 2
> 

Even more strangely, libjavascriptcoregtk-3.0.la does exist:
WebKitBuild/tmp/Release/libjavascriptcoregtk-3.0.la
WebKitBuild/tmp/Release/.libs/libjavascriptcoregtk-3.0.la

[Gustavo Noronha (kov)]

(In reply to comment #12)
> > If I disable WebKit2 build, jsc-3 is created but the build fails later on:
> > 
> >   CXXLD  Programs/jsc-3
> >   CXXLD  libjavascriptcoregtk-3.0.la
> >   CXXLD  Libraries/libTestRunnerInjectedBundle.la
> > libtool: link: cannot find the library `libjavascriptcoregtk-3.0.la' or unhandled argument `libjavascriptcoregtk-3.0.la'
[...]
> 
> Even more strangely, libjavascriptcoregtk-3.0.la does exist:
> WebKitBuild/tmp/Release/libjavascriptcoregtk-3.0.la
> WebKitBuild/tmp/Release/.libs/libjavascriptcoregtk-3.0.la

It makes sense. libjavascriptcoregtk-3.0.la was being built in parallel with whatever failed - and that thing that failed did not find it. If you try running make a second time it should work. Looks like we need to enforce a few more dependencies =(

[Philippe Normand]

(In reply to comment #13)
> (In reply to comment #12)
> > > If I disable WebKit2 build, jsc-3 is created but the build fails later on:
> > > 
> > >   CXXLD  Programs/jsc-3
> > >   CXXLD  libjavascriptcoregtk-3.0.la
> > >   CXXLD  Libraries/libTestRunnerInjectedBundle.la
> > > libtool: link: cannot find the library `libjavascriptcoregtk-3.0.la' or unhandled argument `libjavascriptcoregtk-3.0.la'
> [...]
> > 
> > Even more strangely, libjavascriptcoregtk-3.0.la does exist:
> > WebKitBuild/tmp/Release/libjavascriptcoregtk-3.0.la
> > WebKitBuild/tmp/Release/.libs/libjavascriptcoregtk-3.0.la
> 
> It makes sense. libjavascriptcoregtk-3.0.la was being built in parallel with whatever failed - and that thing that failed did not find it. If you try running make a second time it should work. Looks like we need to enforce a few more dependencies =(

Yes, I started a patch. Almost. There.

[Philippe Normand]

Created attachment 129240 [details]
Patch

[Philippe Normand]

I'm not sure I got this right but it builds, at least.

[WebKit Review Bot]

Thanks for the patch. If this patch contains new public API please make sure it follows the guidelines for new WebKit2 GTK+ API. See http://trac.webkit.org/wiki/WebKitGTK/AddingNewWebKit2API

[Sergio Villar Senin]

(In reply to comment #16)
> I'm not sure I got this right but it builds, at least.

didn't work for me :(

[Alexandre Rostovtsev]

For what it's worth, here is the ugly workaround that I have been using.
I add the following to the root GNUmakefile.am:

all-built-sources-local: $(BUILT_SOURCES) autotoolsconfig.h

all-ltlibraries-local: GNUmakefile $(LTLIBRARIES)

all-programs-local: GNUmakefile $(PROGRAMS)

all-data-local: GNUmakefile $(DATA)


and then do

make -j5 all-built-sources-local
make -j5 all-ltlibraries-local
make -j5 all-programs-local
make -j5 WebKit-3.0.gir # or WebKit-1.0.gir for gtk2 builds
make -j5 all-data-local
make -j5

This hasn't failed yet in numerous build runs, although of course it's not suitable as a permanent solution.

[Martin Robinson]

Comment on attachment 129240 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=129240&action=review

There's a lot of stuff going on in this patch... Are these all required to fix this particular issue or is it just extra left over from experimentation?

> Source/WebKit2/GNUmakefile.am:1517
> +Programs/WebKitPluginProcess: libWebCoreGtk2.la

This needs to include the EXE suffix.

> Source/WebKit2/UIProcess/API/gtk/tests/GNUmakefile.am:144
> +Programs/WebKit2APITests/TestPrinting: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestWebKitWebContext: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestWebKitWebView: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestLoaderClient: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestWebKitSettings: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestBackForwardList: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestDownloads: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestWebKitPolicyClient: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestWebViewEditor: Libraries/libWebKit2APITestCore.la

I'm really suspicious of this. Libraries/libWebKit2APITestCore.la is already in webkit2_tests_ldadd, so there should already be an explicit dependency.

> Tools/GNUmakefile.am:41
> +webcore_internal_sources = \

Better call this libwebcoreinternals_sources or rename libwebcoreinternals_built_sources to webcoreinternals_built_sources and call this webcoreinternals_sources.

> Tools/GNUmakefile.am:60
> +webcore_internal_built_sources = \
>  	DerivedSources/WebCore/JSInternals.cpp \
>  	DerivedSources/WebCore/JSInternals.h   \
>  	DerivedSources/WebCore/JSInternalSettings.cpp \
>  	DerivedSources/WebCore/JSInternalSettings.h
>  
> +libwebcoreinternals_built_sources = $(webcore_internal_built_sources)

Why add webcore_internal_built_sources, when libwebcoreinternals_built_sources already exists?

[Philippe Normand]

(In reply to comment #20)

> 
> > Source/WebKit2/UIProcess/API/gtk/tests/GNUmakefile.am:144
> > +Programs/WebKit2APITests/TestPrinting: Libraries/libWebKit2APITestCore.la
> > +Programs/WebKit2APITests/TestWebKitWebContext: Libraries/libWebKit2APITestCore.la
> > +Programs/WebKit2APITests/TestWebKitWebView: Libraries/libWebKit2APITestCore.la
> > +Programs/WebKit2APITests/TestLoaderClient: Libraries/libWebKit2APITestCore.la
> > +Programs/WebKit2APITests/TestWebKitSettings: Libraries/libWebKit2APITestCore.la
> > +Programs/WebKit2APITests/TestBackForwardList: Libraries/libWebKit2APITestCore.la
> > +Programs/WebKit2APITests/TestDownloads: Libraries/libWebKit2APITestCore.la
> > +Programs/WebKit2APITests/TestWebKitPolicyClient: Libraries/libWebKit2APITestCore.la
> > +Programs/WebKit2APITests/TestWebViewEditor: Libraries/libWebKit2APITestCore.la
> 
> I'm really suspicious of this. Libraries/libWebKit2APITestCore.la is already in webkit2_tests_ldadd, so there should already be an explicit dependency.
> 

Without these I get:

  CXXLD  Programs/WebKit2APITests/TestBackForwardList
  CXXLD  Programs/WebKit2APITests/TestDownloads
  CXXLD  Programs/WebKit2APITests/TestLoaderClient
  CXXLD  Programs/WebKit2APITests/TestPrinting
libtool: link: cannot find the library `Libraries/libWebKit2APITestCore.la' or unhandled argument `Libraries/libWebKit2APITestCore.la'
make[1]: *** [Programs/WebKit2APITests/TestBackForwardList] Error 1
make[1]: *** Waiting for unfinished jobs....
libtool: link: cannot find the library `Libraries/libWebKit2APITestCore.la' or unhandled argument `Libraries/libWebKit2APITestCore.la'
make[1]: *** [Programs/WebKit2APITests/TestDownloads] Error 1
libtool: link: cannot find the library `Libraries/libWebKit2APITestCore.la' or unhandled argument `Libraries/libWebKit2APITestCore.la'
make[1]: *** [Programs/WebKit2APITests/TestLoaderClient] Error 1
libtool: link: cannot find the library `Libraries/libWebKit2APITestCore.la' or unhandled argument `Libraries/libWebKit2APITestCore.la'
make[1]: *** [Programs/WebKit2APITests/TestPrinting] Error 1
make[1]: Leaving directory `/home/phil/gst/jhbuild/build/WebKit/WebKitBuild/Release'

[Martin Robinson]

(In reply to comment #21)
> > I'm really suspicious of this. Libraries/libWebKit2APITestCore.la is already in webkit2_tests_ldadd, so there should already be an explicit dependency.
> > 
> 
> Without these I get:
> 
>   CXXLD  Programs/WebKit2APITests/TestBackForwardList
>   CXXLD  Programs/WebKit2APITests/TestDownloads
>   CXXLD  Programs/WebKit2APITests/TestLoaderClient
>   CXXLD  Programs/WebKit2APITests/TestPrinting
> libtool: link: cannot find the library `Libraries/libWebKit2APITestCore.la' or unhandled argument `Libraries/libWebKit2APITestCore.la'
> make[1]: *** [Programs/WebKit2APITests/TestBackForwardList] Error 1
> make[1]: *** Waiting for unfinished jobs....

Perhaps something is wrong with the way we've defined the convenience library. There is definitely a dependency already set up between the programs and library. Do you mind doing a verbose make dump for one of these failures?

[Philippe Normand]

Created attachment 131681 [details]
Patch

[Martin Robinson]

Comment on attachment 131681 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=131681&action=review

Manually making the binaries depend on a convenience library that they explicitly link against seems very weird. Ifpossible, split out those parts of your patch into a different patch. I'd like to get the other fixes (that look great, by the way) committed and focus on the stuff that looks kind of hacky later.

> Source/WebKit2/GNUmakefile.am:1530
> +Programs/WebKitPluginProcess$(EXEEXT): libWebCoreGtk2.la

I'm still astounded that this should be necessary. What build failure occurs when you omit them?

> Source/WebKit2/UIProcess/API/gtk/tests/GNUmakefile.am:116
> +Programs/WebKit2APITests/AccessibilityTestServer: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestWebKitAccessibility: Libraries/libWebKit2APITestCore.la

Ditto and you are missing the EXE suffix variable as well.

> Source/WebKit2/UIProcess/API/gtk/tests/GNUmakefile.am:153
> +Programs/WebKit2APITests/TestPrinting: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestWebKitWebContext: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestWebKitWebView: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestLoaderClient: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestWebKitSettings: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestBackForwardList: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestDownloads: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestWebKitPolicyClient: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestWebViewEditor: Libraries/libWebKit2APITestCore.la
> +Programs/WebKit2APITests/TestWebKitFindController: Libraries/libWebKit2APITestCore.la

Ditto.

[Thierry Reding]

I've been looking at this bug from another perspective before I searched
the WebKit bugzilla. Strangely the error doesn't seem to exist with
GNU make 3.81. Investigating further I came across the following:

  http://savannah.gnu.org/bugs/?30653

Applying either of the patches in that report on top of GNU make 3.82
and using the new version fixes the parallel build for me.

Maybe this really is a GNU make bug?

Thierry

[Sergio Villar Senin]

(In reply to comment #25)
> I've been looking at this bug from another perspective before I searched
> the WebKit bugzilla. Strangely the error doesn't seem to exist with
> GNU make 3.81. Investigating further I came across the following:
> 
>   http://savannah.gnu.org/bugs/?30653
> 
> Applying either of the patches in that report on top of GNU make 3.82
> and using the new version fixes the parallel build for me.
> 
> Maybe this really is a GNU make bug?

I can confirm that they fix this for me. Actually I run "make check" on GNU make CVS and the parallel build unit tests are failing, so there is indeed something wrong in GNU make (BTW neither of those patches fixes the GNU make unit tests so I guess something more has to be made)

[Ionut Biru]

(In reply to comment #25)
> I've been looking at this bug from another perspective before I searched
> the WebKit bugzilla. Strangely the error doesn't seem to exist with
> GNU make 3.81. Investigating further I came across the following:
> 
>   http://savannah.gnu.org/bugs/?30653
> 
> Applying either of the patches in that report on top of GNU make 3.82
> and using the new version fixes the parallel build for me.
> 
> Maybe this really is a GNU make bug?
> 
> Thierry

thanks for this patch. I can confirm that now this bug is history

[Philippe Normand]

Ok let's close this then. Thanks a lot Thierry for digging out that 2 years old GNU Make patch. And let's hope it gets committed in their CVS one day :)

[Alexandre Rostovtsev]

(In reply to comment #25)
> I've been looking at this bug from another perspective before I searched
> the WebKit bugzilla. Strangely the error doesn't seem to exist with
> GNU make 3.81. Investigating further I came across the following:
> 
>   http://savannah.gnu.org/bugs/?30653
> 
> Applying either of the patches in that report on top of GNU make 3.82
> and using the new version fixes the parallel build for me.

Maybe it is a gnu make bug, but those patches don't fix it. I am still getting a parallel make failure (with MAKEOPTS=-j9) after applying either of the two patches attached to the savannah.gnu.org bug report to gentoo's make-3.82-r3.

I therefore believe this bug should be reopened.

[Grzegorz Czajkowski]

Just one question. Is there any reason why this bug is added to WebKit-EFL component? I think it should be added to WebKit-Gtk :)

[Rion]

GNU Make patch make-intermediate-parallel-bug.patch also solved the problem for me.

tried to compile webkit-gtk-1.8.0

PS sorry for noise in closed report

[Alberto Garcia]

*** Bug 85800 has been marked as a duplicate of this bug. ***