Monday, 11 Dec 2006

Pushing a bzr branch with rsync

For some reason bzr rspush, a plugin from the “bzrtools” unstable command collection, doesn’t know how to push a branch that’s in a Bazaar repository.

Repository layouts

What’s a Bazaar repository? Good question. It’s not what you get if you do bzr init. You have to do bzr init-repo --trees . in a directory, then within that directory create and copy branches. Apparently this then allows the branches to share revision delta files in common.

Why oh why can’t they just be immutable like Git does and thus hard-linkable?

Anyway, I’ve ended up with a directory structure as follows. There’s a top level Bazaar repository here:

~/src/andrew/java-gnome

The bzr data actually lives in, as you might expect,

~/src/andrew/java-gnome/.bzr

But that repository is not a branch and not a working directory. In the repository directory, I create branches and clone them:

~/src/andrew/java-gnome/codegen ~/src/andrew/java-gnome/equivalence-fixes ~/src/andrew/java-gnome/mainline ~/src/andrew/java-gnome/primary ~/src/andrew/java-gnome/website

And so on. Each one also has a .bzr directory:

~/src/andrew/java-gnome/mainline/.bzr ~/src/andrew/java-gnome/website/.bzr

Apparently the deal is that if the actual revision data isn’t in the proximate branch’s .bzr, it’ll look up one to see if there’s a .bzr there (and finding itself in a repository, there is, and so the data it needs is in ../.bzr/repository).

Since I used the --trees option to bzr init-repository, I get working copies in each of those directories. I can then point a symlink to it from the place where my Eclipse IDE expects my working directory to be,

~/workspace/BindingsPrototype -> ~/src/andrew/java-gnome/primary

and finally I can get to work. [You go to a lot of effort to configure Eclipse to work on a given Java project in a specific place. Switching the basedir of that path from one place to another is a non-starter, except by doing a full refactoring. So, the symlink]

I’m liking bzr well enough, but doesn’t this strike you as a bit complicated? Granted git is a usability disaster zone, but the core concepts of repository storage and branching seem really rock solid. I really miss the create-a-branch-in-place and switch-between-branches-in-place aspects of Git. Those were killer features for me. Bazaar has a evolved in a few too many parallel directions, and they need a bit of good old GNOME 2.0 style refactoring to get out some of the cruft. Still, though, they have unit tests. That’s worth a lot in my book.

Pushing the wall

So now its time to publish your work. My experience trying to bzr push sftp:// to a server half way around the world was that the sftp implementation is brutally slow. Using their “Smart Server” with a bzr push bzr+ssh:// command was not much better.

Obviously this sort of thing is crying out for a little rsync lovin’, but unfortunately the bzr rspush command which supposedly does just that doesn’t know how to push a branch that’s in a Bazaar repository. Oh, great. So I had to do it manually.

I have a script that does a bunch of switching to figure what I want to send (source, website, photos, whatever) to where. For source code, I use it something like this:

upload source <project> <branch>

You get the idea. The trick is to figure out how to tell rsync to include the repository .bzr, the branch directory (and its `.bzr), and exclude everything else. I could have just done two separate SOURCEs, but I like to see all the traffic reported relative to a single root, so I use rsync’s include/exclude syntax instead. That stuff is notoriously voodoo; this time was no exception.

Here’s the magic smoke I used to get rsync to push a Bazaar branch within a Bazaar repository:

PROJECT=”$2” BRANCH=”$3”

note trailing slash

SOURCE=”/home/andrew/src/andrew/$PROJECT/” DEST=”andrew@centauri.lhr.operationaldynamics.com:/export/web/com/operationaldynamics/research/bzr/$PROJECT”

for some reason the escaping necessary eluded me; workaround:

s=’*’

now go to the trouble of including each directory in the chain

(otherwise exclude * nukes them) and the three things we care

about in : /.bzr, /, and /.bzr

OPTIONS=”—include=/ —include=/.bzr —include=/$BRANCH —include=/.bzr/$s$s —include=/$BRANCH/$s$s —exclude=/$BRANCH/tmp —exclude=/$BRANCH/.config —exclude=$s”

shift 3

…

exec nice rsync \ —verbose \ —recursive \ —update \ —links \ —hard-links \ —perms \ —times \ —sparse \ -e /usr/bin/ssh \ —partial \ —progress \ —compress \ $OPTIONS \ $* \ $SOURCE $DEST

So with a nice simple:

upload source java-gnome mainline

I publish (only) the “mainline” branch in my ~/src/andrew/java-gnome repository.

Hopefully that will be of help to someone; I can’t be the only person who has found themselves wanting to publish a Bazaar branch from within a project repository. What would be great, though, would be a nice and simple bzr push ssh:// that just assumes rsync-over-ssh and does all this for you. Shouldn’t be too hard for someone to hack in.

AfC

bzr-repository-rsync, posted by Andrew Cowie in software/version-control at 2006-12-10T14:12:41Z

---