Bug 94488 – --disable-dependency-tracking causes build failure due to missing directories

• Status: RESOLVED
• Resolution: FIXED
• Component: WebKitGTK

[Jeremy Huddleston Sequoia]

webkit-1.8.2 fails to build on some configurations with:

webkit-1.8.2 $ sudo make V=1
/opt/local/bin/python ./Source/WebKit2/Scripts/generate-message-receiver.py ./Source/WebKit2/WebProcess/Authentication/AuthenticationManager.messages.in > DerivedSources/WebKit2/AuthenticationManagerMessageReceiver.cpp
/bin/sh: DerivedSources/WebKit2/AuthenticationManagerMessageReceiver.cpp: No such file or directory
make: *** [DerivedSources/WebKit2/AuthenticationManagerMessageReceiver.cpp] Error 1

The problem is that the DerivedSources/WebKit2 path does not exist.

...

If I create that path, the build continues until:


/opt/local/bin/perl Source/WebKit2/Scripts/generate-forwarding-headers.pl ./Source/WebKit2 ./DerivedSources/WebKit2/include gtk
/opt/local/bin/perl Source/WebKit2/Scripts/generate-forwarding-headers.pl ./Source/WebKit2 ./DerivedSources/WebKit2/include soup
/opt/local/bin/perl Source/WebKit2/Scripts/generate-forwarding-headers.pl ./Tools/WebKitTestRunner ./DerivedSources/WebKit2/include gtk
/opt/local/bin/perl Source/WebKit2/Scripts/generate-forwarding-headers.pl ./Tools/WebKitTestRunner ./DerivedSources/WebKit2/include soup
/opt/local/bin/perl \
		-I ./Source/WebCore/bindings/scripts \
		-I ./Tools/WebKitTestRunner/InjectedBundle/Bindings \
		./Source/WebCore/bindings/scripts/generate-bindings.pl \
		--defines \"\" --generator TestRunner \
		--include ./Tools/InjectedBundle/Bindings \
		--outputDir ./DerivedSources/InjectedBundle \
		Tools/WebKitTestRunner/InjectedBundle/Bindings/AccessibilityController.idl
Failed to open JSAccessibilityController.h for writing: No such file or directory at Tools/WebKitTestRunner/InjectedBundle/Bindings/CodeGeneratorTestRunner.pm line 52.

This time because DerivedSources/InjectedBundle does not exist.

...

This was reported earlier as bug #79762, but that bug was closed as invalid.  It does not look like an invalid bug to me.  Whatever is responsible for creating those paths seems to be failing.

[Jeremy Huddleston Sequoia]

The problem is that the paths under DerivedSources are not created at all by the build system.  There are no dependencies setup in the Makefiles to create them.  They are currently created by the depfiles phase of autoconf, but that is not done if configured using --disable-dependency-tracking (which is frequently done on OS X to build fat binaries).

The fix for this is to setup dependencies in the Makefiles for these paths and create them if not present.

[Tobias Mueller]

Created attachment 193474 [details]
Patch making --disable-dependency-tracking work

Aye. Same here.

Some more people suffering from this: http://lists.linuxtogo.org/pipermail/openembedded-core/2011-October/010590.html
https://bugs.gentoo.org/show_bug.cgi?id=406117

For completeness:
$ ./autogen.sh --disable-dependency-tracking
...
WebKit was configured with the following options:

Build configuration:
 Enable debugging (slow)                                  : no
 Compile with debug symbols (slow)                        : no
 Enable GCC build optimization                            : yes
 Code coverage support                                    : no
 Optimized memory allocator                               : yes
 Accelerated rendering backend                            : opengl(gl, egl, glx)

Features:
=======
 WebKit1 support                                          : yes
 WebKit2 support                                          : yes
 Accelerated Compositing                                  : yes
 Gamepad support                                          : no
 Geolocation support                                      : no
 HTML5 video element support                              : no
 JIT compilation                                          : autodetect
 Opcode stats                                             : no
 SVG fonts support                                        : yes
 SVG support                                              : yes
 Spellcheck support                                       : no
 Web Audio support                                        : no
 WebGL                                                    : yes
 XSLT support                                             : yes


GTK+ configuration:
 GTK+ version                                             : 3.0
 GDK target                                               : x11
 Introspection support                                    : no
 Generate documentation                                   : no

$ time make -j 40
  GEN    DerivedSources/WebCore/InternalSettingsGenerated.idl
  GEN    DerivedSources/webkitdom/WebKitDOMEventTarget.h
  GEN    DerivedSources/webkitdom/WebKitDOMEventTargetPrivate.h
  GEN    DerivedSources/webkitdom/webkitdomdefines.h
/bin/sh: DerivedSources/webkitdom/webkitdomdefines.h: No such file or directory
  GEN    DerivedSources/webkitdom/WebKitDOMObject.h
  GEN    DerivedSources/webkitdom/WebKitDOMCustom.h
  GEN    DerivedSources/webkitdom/webkitdom.h
/bin/sh: DerivedSources/webkitdom/webkitdom.h: No such file or directory
cp: cannot create regular file `DerivedSources/webkitdom/WebKitDOMEventTargetPrivate.h': No such file or directory
make: *** [DerivedSources/webkitdom/webkitdom.h] Error 1
make: *** Waiting for unfinished jobs....
make: *** [DerivedSources/webkitdom/webkitdomdefines.h] Error 1
make: *** [DerivedSources/webkitdom/WebKitDOMEventTargetPrivate.h] Error 1
cp: cannot create regular file `DerivedSources/webkitdom/WebKitDOMEventTarget.h': No such file or directory
make: *** [DerivedSources/webkitdom/WebKitDOMEventTarget.h] Error 1
cp: cannot create regular file `DerivedSources/webkitdom/WebKitDOMObject.h': No such file or directory
make: *** [DerivedSources/webkitdom/WebKitDOMObject.h] Error 1
cp: cannot create regular file `DerivedSources/webkitdom/WebKitDOMCustom.h': No such file or directory
make: *** [DerivedSources/webkitdom/WebKitDOMCustom.h] Error 1
$ make V=1 -j1 DerivedSources/InjectedBundle/JSAccessibilityController.cpp
/usr/bin/perl \
        -I ./Source/WebCore/bindings/scripts \
        -I ./Tools/WebKitTestRunner/InjectedBundle/Bindings \
        ./Source/WebCore/bindings/scripts/generate-bindings.pl \
        --defines \"\" --generator TestRunner \
        --include ./Tools/InjectedBundle/Bindings \
        --outputDir ./DerivedSources/InjectedBundle \
        Tools/WebKitTestRunner/InjectedBundle/Bindings/AccessibilityController.idl
Failed to open JSAccessibilityController.h for writing: No such file or directory at Tools/WebKitTestRunner/InjectedBundle/Bindings/CodeGeneratorTestRunner.pm line 53.
make: *** [DerivedSources/InjectedBundle/JSAccessibilityController.cpp] Error 2
$ 

If we create the directory before trying to put the file file there, it works:
$ git diff Tools/WebKitTestRunner/GNUmakefile.am
diff --git a/Tools/WebKitTestRunner/GNUmakefile.am b/Tools/WebKitTestRunner/GNUmakefile.am
index 1812569..c97743f 100644
--- a/Tools/WebKitTestRunner/GNUmakefile.am
+++ b/Tools/WebKitTestRunner/GNUmakefile.am
@@ -156,6 +156,7 @@ code_generation_dependencies = \
        Source/WebCore/bindings/scripts/generate-bindings.pl
 
 DerivedSources/InjectedBundle/JS%.cpp: Tools/WebKitTestRunner/InjectedBundle/Bindings/%.idl $(code_generation_dependencies)
+       mkdir -p $(@D)
        $(AM_V_GEN)$(PERL) \
                -I $(srcdir)/Source/WebCore/bindings/scripts \
                -I $(srcdir)/Tools/WebKitTestRunner/InjectedBundle/Bindings \
$ make --debug V=1 -j1 DerivedSources/InjectedBundle/JSAccessibilityController.cpp
GNU Make 3.82
Built for x86_64-unknown-linux-gnu
Copyright (C) 2010  Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Reading makefiles...
Updating goal targets....
 File `DerivedSources/InjectedBundle/JSAccessibilityController.cpp' does not exist.
Must remake target `DerivedSources/InjectedBundle/JSAccessibilityController.cpp'.
Invoking recipe from GNUmakefile:89360 to update target `DerivedSources/InjectedBundle/JSAccessibilityController.cpp'.
mkdir -p DerivedSources/InjectedBundle
/usr/bin/perl \
        -I ./Source/WebCore/bindings/scripts \
        -I ./Tools/WebKitTestRunner/InjectedBundle/Bindings \
        ./Source/WebCore/bindings/scripts/generate-bindings.pl \
        --defines \"\" --generator TestRunner \
        --include ./Tools/InjectedBundle/Bindings \
        --outputDir ./DerivedSources/InjectedBundle \
        Tools/WebKitTestRunner/InjectedBundle/Bindings/AccessibilityController.idl
Successfully remade target file `DerivedSources/InjectedBundle/JSAccessibilityController.cpp'.


I chased down every error I got and subsequently put mkdirs in the Makefile.ams. Now I don't get any error anymore, but if the order of the made targets changes for whatever reason, it may break again. Then, more mkdirs need to be added.

[Gustavo Noronha (kov)]

Comment on attachment 193474 [details]
Patch making --disable-dependency-tracking work

View in context: https://bugs.webkit.org/attachment.cgi?id=193474&action=review

Thanks for the patch! I think the changes are OK, just a few changelog style issues that need fixing.

> Source/WebCore/ChangeLog:6
> +        Autotools build fix: Ensure output directory existing
> +        before generating DerivedSources. This allows for
> +        --disable-dependency-tracking to be run.
> +        https://bugs.webkit.org/show_bug.cgi?id=94488

This should be the bug's title, not the change description, change description should go bellow the Reviewed by line.

> Source/WebCore/ChangeLog:10
> +        No new tests (OOPS!).

You should remove this line.

> Source/WebCore/ChangeLog:14
> +        * GNUmakefile.am:
> +            Added new target DerivedSources/ANGLE which is a directory to be
> +            created

You should start the change descrtiption detail right after the colon, lines do not need to be split at 80 columns, these could be a single line.

[Tobias Mueller]

Created attachment 220519 [details]
Changed changelog

I hope I incorporated the suggested changes to the changelog.

[Tobias Mueller]

Created attachment 220527 [details]
rebased against master

[Gustavo Noronha (kov)]

Comment on attachment 220527 [details]
rebased against master

Thanks!

[WebKit Commit Bot]

Comment on attachment 220527 [details]
rebased against master

Clearing flags on attachment: 220527

Committed r161808: <http://trac.webkit.org/changeset/161808>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.

[WebKit Commit Bot]

Re-opened since this is blocked by bug 126874

[Carlos Garcia Campos]

After this patch, every time you run make, even if nothing has changed, causes that some generated files are re-generated, causing a new build:

  GEN      DerivedSources/webkitdom/WebKitDOMEventTarget.h
  GEN      DerivedSources/webkitdom/WebKitDOMCustom.h
  GEN      DerivedSources/webkitdom/WebKitDOMObject.h
  GEN      DerivedSources/webkitdom/webkitdom.h
  GEN      DerivedSources/webkitdom/webkitdomdefines.h
  GEN      DerivedSources/webkitdom/WebKitDOMEventTargetPrivate.h
  GEN      DerivedSources/webkit/webkitenumtypes.cpp
  GEN      stamp-webkitmarshal.cpp
  GEN      stamp-webkitmarshal.h
  GEN      DerivedSources/InjectedBundle/JSAccessibilityTextMarker.cpp
  GEN      DerivedSources/InjectedBundle/JSAccessibilityController.cpp
  GEN      DerivedSources/InjectedBundle/JSAccessibilityTextMarkerRange.cpp
  GEN      DerivedSources/InjectedBundle/JSAccessibilityUIElement.cpp
  GEN      DerivedSources/InjectedBundle/JSEventSendingController.cpp
  GEN      DerivedSources/InjectedBundle/JSGCController.cpp
  GEN      DerivedSources/InjectedBundle/JSTestRunner.cpp
  GEN      DerivedSources/InjectedBundle/JSTextInputController.cpp

[Jeremy Huddleston Sequoia]

(In reply to comment #10)
> After this patch, every time you run make, even if nothing has changed, causes that some generated files are re-generated, causing a new build:

Is that really a regression?  Currently, we're seeing that 'make install' remakes everything even after a successful 'make'.  I haven't gotten around to investigating it yet.  If you're seeing good behavior, that's a data point of interest to me, and perhaps related.

[Carlos Garcia Campos]

(In reply to comment #11)
> (In reply to comment #10)
> > After this patch, every time you run make, even if nothing has changed, causes that some generated files are re-generated, causing a new build:
> 
> Is that really a regression?  Currently, we're seeing that 'make install' remakes everything even after a successful 'make'.  I haven't gotten around to investigating it yet.  If you're seeing good behavior, that's a data point of interest to me, and perhaps related.

make install needs to relink binaries because of the disabled fast install. But make after make should do nothing. It's quite annoying that some files are always rebuilt. For example the WebKitDOM header files regenerated makes that all dom bindings are compiled for every build.