Bug 73100 – Add CMake based build system for Wx port

• Component: Tools / Tests
• Status: RESOLVED WONTFIX

[Patrick R. Gansterer]

Created attachment 116552 [details]
WIP: First code drop of CMake build system for Wx

Add CMake base build system for Wx port

[Kevin Ollivier]

Comment on attachment 116552 [details]
WIP: First code drop of CMake build system for Wx

I don't think the wx port needs two build systems in the tree, and as the wx port maintainer, I am only going to use and maintain the waf build system, as I really need to keep the maintenance work to a minimum and waf does a better job of that than CMake would.

[Patrick R. Gansterer]

(In reply to comment #1)
> (From update of attachment 116552 [details])
> I don't think the wx port needs two build systems in the tree.
I want to unify our build systems. There is also a bug for a replacement of the Visual Studio files at bug 72816.

> I really need to keep the maintenance work to a minimum
Maintaining one build system is less work than 5! For _all_ webkit devs.

> waf does a better job of that than CMake would.
Can you explain that to me? What is a killer feature of waf vs. CMake? When did you used CMake the last time?

[Kevin Ollivier]

(In reply to comment #2)
> (In reply to comment #1)
> > (From update of attachment 116552 [details] [details])
> > I don't think the wx port needs two build systems in the tree.
> I want to unify our build systems. There is also a bug for a replacement of the Visual Studio files at bug 72816.
> 
> > I really need to keep the maintenance work to a minimum
> Maintaining one build system is less work than 5! For _all_ webkit devs.
> 
> > waf does a better job of that than CMake would.
> Can you explain that to me? What is a killer feature of waf vs. CMake? When did you used CMake the last time?

The killer feature is that when a new file is added to the build system, 90% of the time no one has to do anything. The build system automatically adds the file to the build for me so long as it's in one of the directories the port is set to build. Since the WebKit port is fairly clean, this is almost always the correct action to take, and I just add an exception on the rare occasions it doesn't work.

So basically, with almost no build system maintenance at all, the build will continue to work sometimes for weeks since my last update, whereas with the other ports someone needs to be updating the build system on more or less a daily basis. Sure, that effort can be shared, but on the wx port, it simply doesn't need to be done most of the time, which means more time coding features or debugging rather than doing tedious build system work. As an example, I actually checked build system commits one time, and over a period of time in which I updated the waf build system 26 times, the Gyp build files needed around 140 something updates. So that's about 6 times more build system commits to maintain a port than what wx has. I personally consider that easier to maintain.

Also, since waf uses Python, I have a full scripting language at my disposal whenever I want to solve a problem (i.e. I can write a script to determine files to build, rather than do everything by hand), so I'm not boxed into some build system's macro language nor am I limited to what native project files might support.

So from my perspective, CMake, Bakefile, and Gyp all basically work the same, there are differences but they're mostly in the project format and level of native project support. None of them do what I want them to do, which is let the computer do most of the maintenance work automatically based on the project structure and rules.

[Patrick R. Gansterer]

(In reply to comment #3)
> (In reply to comment #2)
> > (In reply to comment #1)
> > > (From update of attachment 116552 [details] [details] [details])
> > > I don't think the wx port needs two build systems in the tree.
> > I want to unify our build systems. There is also a bug for a replacement of the Visual Studio files at bug 72816.
> > 
> > > I really need to keep the maintenance work to a minimum
> > Maintaining one build system is less work than 5! For _all_ webkit devs.
> > 
> > > waf does a better job of that than CMake would.
> > Can you explain that to me? What is a killer feature of waf vs. CMake? When did you used CMake the last time?
> 
> The killer feature is that when a new file is added to the build system, 90% of the time no one has to do anything. The build system automatically adds the file to the build for me so long as it's in one of the directories the port is set to build. Since the WebKit port is fairly clean, this is almost always the correct action to take, and I just add an exception on the rare occasions it doesn't work.

The "implicit vs. explicit" way is in CMake possible too, but I prefer to list all files by hand.

> So basically, with almost no build system maintenance at all, the build will continue to work sometimes for weeks since my last update, whereas with the other ports someone needs to be updating the build system on more or less a daily basis. Sure, that effort can be shared, but on the wx port, it simply doesn't need to be done most of the time, which means more time coding features or debugging rather than doing tedious build system work. As an example, I actually checked build system commits one time, and over a period of time in which I updated the waf build system 26 times, the Gyp build files needed around 140 something updates. So that's about 6 times more build system commits to maintain a port than what wx has. I personally consider that easier to maintain.

If you want to see what's the port specific maintenance is, you need to look at the PlatformXXX.cmake files. If I don't change any port specific stuff, I don't have to touch them at all. The generic stuff is all handled in the shared CMakeLists.txt. If we get more ports to use CMake it will make sense to share stuff like the networkbackend too.

> Also, since waf uses Python, I have a full scripting language at my disposal whenever I want to solve a problem (i.e. I can write a script to determine files to build, rather than do everything by hand), so I'm not boxed into some build system's macro language nor am I limited to what native project files might support.

I don't see much need for this, but AFAIK CMake can put all targets into the native project files.
Please correct me if I'm wrong, but many stuff is done in Makefiles and not in waf.

> So from my perspective, CMake, Bakefile, and Gyp all basically work the same, there are differences but they're mostly in the project format and level of native project support. None of them do what I want them to do, which is let the computer do most of the maintenance work automatically based on the project structure and rules.

IMHO switching to CMake will result in less port specific maintenance.

Some of my arguments for CMake:
*) Reduce the count of build systems. It's true that if a generic file is added it works nearly without any work, but if you add any more specific stuff like code generators devs need to add it in _all_ build systems.
*) No dependency on Makefiles, CMake contains everything needed to builds.
*) Complete native windows support. There is no need for cygwin. Win32 perl and python are enough.
*) Reuse features of "other ports". E.g. there is work to get V8 as alternative JS engine into CMake. If we share the build system, you only need to set a define for switching between the JS engines. For all ports.

[Kevin Ollivier]

(In reply to comment #4)
> (In reply to comment #3)
> > (In reply to comment #2)
> > > (In reply to comment #1)
> > > > (From update of attachment 116552 [details] [details] [details] [details])
> > > > I don't think the wx port needs two build systems in the tree.
> > > I want to unify our build systems. There is also a bug for a replacement of the Visual Studio files at bug 72816.
> > > 
> > > > I really need to keep the maintenance work to a minimum
> > > Maintaining one build system is less work than 5! For _all_ webkit devs.
> > > 
> > > > waf does a better job of that than CMake would.
> > > Can you explain that to me? What is a killer feature of waf vs. CMake? When did you used CMake the last time?
> > 
> > The killer feature is that when a new file is added to the build system, 90% of the time no one has to do anything. The build system automatically adds the file to the build for me so long as it's in one of the directories the port is set to build. Since the WebKit port is fairly clean, this is almost always the correct action to take, and I just add an exception on the rare occasions it doesn't work.
> 
> The "implicit vs. explicit" way is in CMake possible too, but I prefer to list all files by hand.

Well, your preference is to do more maintenance work, whereas for me, that 'preference' reduces the time I can spend actually improving the port. That alone is a big problem for me switching.

> > So basically, with almost no build system maintenance at all, the build will continue to work sometimes for weeks since my last update, whereas with the other ports someone needs to be updating the build system on more or less a daily basis. Sure, that effort can be shared, but on the wx port, it simply doesn't need to be done most of the time, which means more time coding features or debugging rather than doing tedious build system work. As an example, I actually checked build system commits one time, and over a period of time in which I updated the waf build system 26 times, the Gyp build files needed around 140 something updates. So that's about 6 times more build system commits to maintain a port than what wx has. I personally consider that easier to maintain.
> 
> If you want to see what's the port specific maintenance is, you need to look at the PlatformXXX.cmake files. If I don't change any port specific stuff, I don't have to touch them at all. The generic stuff is all handled in the shared CMakeLists.txt. If we get more ports to use CMake it will make sense to share stuff like the networkbackend too.

Right, shared maintenance, which means that until the main ports are using it, sometimes when I'm pulling code I will have to find out what files haven't yet been added and add them myself.

> > Also, since waf uses Python, I have a full scripting language at my disposal whenever I want to solve a problem (i.e. I can write a script to determine files to build, rather than do everything by hand), so I'm not boxed into some build system's macro language nor am I limited to what native project files might support.
> 
> I don't see much need for this, but AFAIK CMake can put all targets into the native project files.
> Please correct me if I'm wrong, but many stuff is done in Makefiles and not in waf.

I don't have much of a need for native project files either, I think they're a waste of time, although waf can generate MSVC projects. The problem is that you have to either (1) add support for native project files, or (2) convince all those who DO like them to abandon them if you want them to all use CMake, or waf, or whatever.

As for the Makefiles, if by "many stuff", you mean running DerivedSources.make, yes, we do that, but it could easily be done in waf, and be rule-based as well, I just have not had the interest in doing it yet because it works "well enough" right now. (Though it is certainly not ideal.) If I had the time to do things like move other people's build systems to waf instead of adding more features to wxWebKit, it would most certainly have been done a long time ago. ;-) A nice side-effect is that thanks to Python, it would make it really easy to prune the DerivedSources files and remove old ones that no longer get generated and keep that dir clean.

OTOH, waf is already capable of sharing the network backend code, so it's not a simple issue of waf is limited and CMake is not. In both cases, we're simply talking about things that could be done, but that haven't so far been needed or seemed big enough of a problem to be done. Though I might rewrite the DerivedSources stuff over Christmas break or something regardless. :)

> > So from my perspective, CMake, Bakefile, and Gyp all basically work the same, there are differences but they're mostly in the project format and level of native project support. None of them do what I want them to do, which is let the computer do most of the maintenance work automatically based on the project structure and rules.
> 
> IMHO switching to CMake will result in less port specific maintenance.

Less than other build systems, but not less than waf. In an ideal state, from everything I've read and heard so far, CMake is "as good as" waf, until of course you need to do something CMake's project formats don't support, and then you have to start calling external tools and doing things like pulling in data from them in a format CMake can manipulate.

> Some of my arguments for CMake:
> *) Reduce the count of build systems. It's true that if a generic file is added it works nearly without any work, but if you add any more specific stuff like code generators devs need to add it in _all_ build systems.
> *) No dependency on Makefiles, CMake contains everything needed to builds.
> *) Complete native windows support. There is no need for cygwin. Win32 perl and python are enough.
> *) Reuse features of "other ports". E.g. there is work to get V8 as alternative JS engine into CMake. If we share the build system, you only need to set a define for switching between the JS engines. For all ports.

All of these advantages apply equally to waf. waf also has:

- built-in multi-core support, and will actually  by default compile using all of your cores (even on Windows/nmake);
- is in its entirety a ~90KB file, so we could actually put the build system in the tree and not require people to install it, unlike CMake; and 
- has the power of the entire Python standard library at its disposal (and Py3 compat if we want to go that route), which is a pretty massive plus IMHO.

And thanks to its rules based compilation, adding new ports is pretty easy. You would add your port name to the list (and what core features it uses, like CF, etc.), and it would automatically adjust the rules to include your port subdirectories, and WhateverPN.cpp (where PN is the port / feature name, like WhateverQt.cpp or WhateverCF.cpp) files to the build, and exclude those of other ports / features. (Because of the exclusion rule, I actually have most ports listed already.) Then it's mostly just a matter of passing --port=XYZ to waf, seeing how it builds, and adding/adjusting whatever port-specific compilation and linking flags you need. Heck, thanks to Python, we could even have waf output CMake or Gyp project files rather than build, if we wanted to.

I personally think waf has a lot going for it and stands up very well against any build system I've been presented with thus far. However, I don't think many people really take it seriously or give it a fair shake. This goes into the whole preferences discussion, which is the real sticking point. Each port developer has preferences, like your preference to explicitly list build files or some peoples' strong preference for project files, and these preferences come into conflict with each other. So the real issue that needs dealt with is not "which build system is best", it is, "which build system can best accommodate conflicting developer preferences", to which the answer is simply "none". Everyone does at some point have to agree on a best method and commit to it if we are to ever consolidate things. This is actually the main reason I don't really try to evangelize waf, and why I have suggested in the past having scripts that can generate file lists for various project formats as ways to reduce maintenance.

[Patrick R. Gansterer]

Wx has been removed from the tree.