Bug 192081 – Update scripts for generating .xcfilelist files

• Status: RESOLVED
• Resolution: FIXED
• Component: Tools / Tests

[Keith Rollin]

The initial pass at generate-xcfilelists (a script for creating/updating the .xcfilelist files needed for various Generate Foo Source build phases in Xcode) only generated the .xcfilelist files that held the output files; it did not generate the list of input files. As well, for the sources generated by DerivedSources.make makefiles, the script accomplished this via the implementation of a convention in the makefile that allowed the printing of these output files when invoked with the 'print_all_generated_files' target. Use of this convention is inconvenient and error-prone.

The script is now updated to address both of these issues. First, it generates for the input and output sets of files. Second, it does away with the convention in the DerivedSources.make makefiles and instead works from the dependency output information printed when `make` is invoked with --debug. This second part is implemented in the new script extract-dependencies-from-makefile.

[Radar WebKit Bug Importer]

<rdar://problem/46312290>

[Keith Rollin]

Created attachment 355880 [details]
Patch

[Alex Christensen]

Comment on attachment 355880 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=355880&action=review

> Tools/ChangeLog:37
> +        * Scripts/generate-xcfilelists:

When is this script intended to be used?  Is it part of the build system, or just something that people run when they add new files or change dependencies?

[Keith Rollin]

I'm working on the story for when it can be used. Right now, it's just something I ran by hand. I'm considering adding a build step that will run it in order to check the existing .xcfilelists so that people can then regenerate them and restart the build. Unfortunately, I can't run it during the build and create up-to-date .xcfilelists on-the-fly because XCBuild needs them before the build starts.

[Alex Christensen]

Comment on attachment 355880 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=355880&action=review

> Tools/Scripts/extract-dependencies-from-makefile:5
> +# Copyright (C) 2011 Apple Inc. All rights reserved.

2011?

> Tools/Scripts/extract-dependencies-from-makefile:16
> +# 3.  Neither the name of Apple Inc. ("Apple") nor the names of

We use a 2-clause copyright now.

> Tools/Scripts/generate-xcfilelists:204
> +        ${BUILT_PRODUCTS_DIR}/usr/local/include/WebKitAdditions
> +        ${SDKROOT}/usr/local/include/WebKitAdditions

These lists look fragile.  What are they needed for?
Also, I'm pretty sure it's wrong to look in the sdk and build directory for WebKitAdditions.

[Keith Rollin]

(In reply to Alex Christensen from comment #5)
> Comment on attachment 355880 [details]
> Patch
> 
> View in context:
> https://bugs.webkit.org/attachment.cgi?id=355880&action=review
> 
> > Tools/Scripts/extract-dependencies-from-makefile:5
> > +# Copyright (C) 2011 Apple Inc. All rights reserved.
> 
> 2011?

Bah.

> > Tools/Scripts/extract-dependencies-from-makefile:16
> > +# 3.  Neither the name of Apple Inc. ("Apple") nor the names of
> 
> We use a 2-clause copyright now.

What's a 2-clause copyright? What's a better source from which I should copy?

> > Tools/Scripts/generate-xcfilelists:204
> > +        ${BUILT_PRODUCTS_DIR}/usr/local/include/WebKitAdditions
> > +        ${SDKROOT}/usr/local/include/WebKitAdditions
> 
> These lists look fragile.  What are they needed for?
> Also, I'm pretty sure it's wrong to look in the sdk and build directory for
> WebKitAdditions.

The values for the various FOO_SEARCH_PATHS variables were taken from build output. I then replaced the host-specific absolute paths with the appropriate environment variables. In the case of the above, the values come from BaseTarget.xcconfig:

WEBKITADDITIONS_HEADER_SEARCH_PATHS = $(BUILT_PRODUCTS_DIR)/usr/local/include/WebKitAdditions $(SDKROOT)/usr/local/include/WebKitAdditions;
...
HEADER_SEARCH_PATHS = $(BUILT_PRODUCTS_DIR)/usr/local/include "$(WEBCORE_PRIVATE_HEADERS_DIR)/ForwardingHeaders" $(BUILT_PRODUCTS_DIR)/DerivedSources/WebKit2 $(WEBKITADDITIONS_HEADER_SEARCH_PATHS) $(LIBWEBRTC_HEADER_SEARCH_PATHS) $(SRCROOT) $(HEADER_SEARCH_PATHS);

I copy those values into generate-xcfilelists because I invoke DerivedSources.make, which references those variables. When I do that, it's important for the environments of that invocation and that of xcodebuild are the same. If the definitions of the variables change between the time I invoke DerivedSources.make and the time xcodebuild invokes DerivedSources.make, then that makefile may provide differing output.

Yes, this is fragile, since it means that the generate-xcfilelists script needs to be in-sync with the xcconfig file. I'm not, however, sure of an alternative. Do you have a suggestion?

[Brent Fulgham]

Comment on attachment 355880 [details]
Patch

View in context: https://bugs.webkit.org/attachment.cgi?id=355880&action=review

>> Tools/ChangeLog:37
>> +        * Scripts/generate-xcfilelists:
> 
> When is this script intended to be used?  Is it part of the build system, or just something that people run when they add new files or change dependencies?

Good question. What say you, Keith?

>>> Tools/Scripts/extract-dependencies-from-makefile:16
>>> +# 3.  Neither the name of Apple Inc. ("Apple") nor the names of
>> 
>> We use a 2-clause copyright now.
> 
> What's a 2-clause copyright? What's a better source from which I should copy?

See the copyright text in 'HTMLMediaElement.cpp" for example. It's just a more recent version of our preferred license header.

[Keith Rollin]

(In reply to Brent Fulgham from comment #7)
> Comment on attachment 355880 [details]
> Patch
> > When is this script intended to be used?  Is it part of the build system, or just something that people run when they add new files or change dependencies?
> 
> Good question. What say you, Keith?

I say I addressed it in comment #4. I need the patch in this bug to land before I can submit that work, since that work depends on the generate-xcfilelists in this bug.

I'll get on the bit about the copyright and clauses Tuesday or Wednesday (I'm out sick from work right now).

[Keith Rollin]

Created attachment 356458 [details]
Updated copyright header and date.

[Brent Fulgham]

Comment on attachment 356458 [details]
Updated copyright header and date.

lgtm. r=me.

[WebKit Commit Bot]

Comment on attachment 356458 [details]
Updated copyright header and date.

Clearing flags on attachment: 356458

Committed r238854: <https://trac.webkit.org/changeset/238854>

[WebKit Commit Bot]

All reviewed patches have been landed.  Closing bug.