Using inotify to trigger builds

Having switched from an Eclipse (which nicely takes care of building your project for you) to working in gVim (which does nothing of the sort), it’s a bit tedious to have to keep switching from the editor’s window to a terminal in the right directory to whack Up then Enter to run make again.

I know about inotify, a capability in the Linux kernel to watch files for changes, but I hadn’t realized there was a way to use it from the command line. Turns out there is! Erik de Castro Lopo pointed me to a program called inotifywatch that he was using in a little shell script to build his Haskell code. Erik had it set up to run make if one of the files he’d listed on the script’s command line changed.

Saving isn’t what you think it is

I wanted to see if I could expand the scope in a few ways. For one thing, if you had inotifywatch running on a defined list of files and you created a new source file, it wouldn’t trigger a build because it wasn’t being watched by inotify. So I had a poke at Erik’s script.

Testing showed that the script was working, but not quite for the reason that we thought. It was watching for the ‘modify’ event, but actually catching a non-zero exit code. That’s strange; I was expecting a normal 0 not error 1. Turns out 1 is the exit code in situations when the file you were watching is deleted. Huh? All that I did was save a file!

Of course, that’s not what many programs do when saving. To avoid the risk of destroying your original file in the event of having an I/O error when overwriting it, most editors do NOT modify your file in place; they write a new copy and then atomically rename it over the original. Other programs move your original to a backup name and write a new file. Either way, you’ve usually got a new inode. Most of the time.

And that makes the exact event(s) to listen for tricky to pin down. At first glance the ‘modify’ one seemed a reasonable choice, but as we’ve just seen that turns out to not be much use, and meanwhile you end up triggering due to changes made to transient garbage like Vim’s swap files — certainly what you don’t want to trigger a build. Given that a new file is being made, I then tried watching for the ‘create’ event, but it’s overblown with noise too. Finally, you want touching a file to result in a rebuild and that doesn’t involve a ‘create’ event.

It turns out that saving (however done) and touching a file have in common that at some point in the sequence your file (or its backup) will be opened and then closed for writing. inotify has a ‘close_write’ event (complimenting ‘close_nowrite’) so that’s the one to watch for.

If you want to experiment with figuring all this yourself, try doing:

$ inotifywait -m .

and then use your editor and build tools as usual. It’s pretty interesting. The inotifywait(1) program is part of the 'inotify-tools' package on Debian-based Linux systems.

Resurrection, Tron style

Triggering a build automatically is brilliant, but only half the equation; inevitably you want to run the program after building it. It gets harder; if the thing you’re hacking on is a service then, having run it, you’ve got to kill it off and restart it to find out if your code change fixed the problem. How many times have you been frustrated that your bugfix hasn’t taken only to realize you’ve forgotten to restart the thing you’re testing? Running the server manually in yet another terminal window and then killing it and restarting it — over and over — is quite a pain. So why not have that triggered as a result of the inotify driven build as well?

Managing concurrent tasks is harder than it should be. Bash has “job control“, of course, and we’re well used to using it in interactive terminals:

$ ./program
^Z
$

$ bg
[1] 13796
$ jobs
[1] Running    ./program
$

$ kill %1
[1] Terminated ./program
$

It’s one thing to run something and then abort it, but it’s another thing entirely to have a script that runs it and then kills it off in reaction to a subsequent event. Job control is lovely when you’re interactive but for various reasons is problematic to use in a shell script (though, if you really want to, see set -m). You can keep it simple, however: assuming for a moment you have just one program that needs running to test whatever-it-is-you’re-working-on, you can simply capture the process id and use that:

    #!/bin/sh

    ./program &
    PID="$!"

Then you can later, in response to whatever stimuli, do:

    kill $PID

Said stimulus is, of course, our blocking call to inotifywait, returning because a file has been saved.

GOTO 10

Do a build. If it succeeds, run the specified program in the background then block waiting for an inotify ‘close_write’ event. When that happens, kill the program and loop back to the beginning. Easy, right? Sure. that’s why it took me all day.

I called it inotifymake. Usage is simple; throw it in ~/bin then:

$ cd ~/src/project/branch/
$ inotifymake -- ./program

make does its thing, building program as its result
program runs
waiting…

change detected!

kill program make does its thing, build failed :(
waiting…

change detected!

make does its thing, rebuilding program as a result
program runs
waiting…

the nice part being that the server or whatever isn’t running in the middle when the build is borked; http://localhost:8000/ isn’t answering. Yeah.

[The “--” back there separates make targets from the name of the executable you’re running, so you can do inotifymake test or inotifymake test -- ./program if you like]

So yes, that was a lot of effort for not a lot of script, but this is something I’ve wanted for a long time, and seems pretty good so far. I’m sure it could be improved; frankly if it needed to be any more rigorous I’d rewrite it as a proper C program, but in the mean time, this will do. Feedback welcome if you’ve got any ideas; branch is there if you want it.

AfC