Developers are bad for build systems

It has often been observed that developers are the enemy of build systems. It is a rare person who actually knows how they work. (The variety of OSS build system available now fall into two main categories: (1) make, cook, and jam (et al) work backwards from the outputs; and (2) automake, scons, waf (et al) that work forward from the sources. I tend to blend them, myself, as circumstances require.

One method I use to keep developers away from my build systems is to

  1. Ask the version control system for a definitive list of primary source files.
  2. Using heuristics, generate a Makefile or a Makefile.am file (etc).  Non-recursive, of course.
  3. Make sure the (generated) Makefile knows how to update the Makefile.

The great thing with this method is that when users add a new file to a change set (“svn add”, or similar) the next build will automatically notice it, and rewrite the Makefile (or whatever) and then build your new file.  You can approximate this technique using find(1), and filtering out all build products (heuristically).

Now, this isn’t simple, because users also structure source trees in weird and wonderful ways. But you can depend on some things, like a directory called “lib” almost certainly contains source files to be compiled and linked into a library.

Another pattern is to treat every .c or .cpp file in the “src” directory as a command to be compiled and linked, probably to including that common library of functions. (Except when a some of files under “src” should be a common library, and then you have to look in the source files, looking for the main() function, to find programs to build.)  Don’t bike shed the concept, just hack something that works, and then you can improve it later, in light of experience, and possibly some yak shaving much later for entertainment.

My projects tend to be larger than that, and I use a heuristic that every directory that contains main.c or main.cpp (etc) should have all of it source files compiled and linked into the one program, and also to that common library of functions, probably.  If a program has a name starting with “test_” is is built (for use by automated tests) but not installed.

The point is to manipulate the shape of your source directory tree so that a relatively simple program can generate the build system from the definitive list of primary source files.  For example: you can easily see the mapping from the list of primary source files to the Makfile.am contents, it’s almost 1:1.

Now that users don’t need to edit the Makefile, they don’t need to fsck it up, either.

Also, I always build executables into a single directory called “bin”. It makes it very easy to set the PATH when running automated tests.