Monday, 03 Aug 2009

One year and more

One of the things that saddens me is just how long it takes to write good code.

Havoc Pennington wrote a classic essay almost a decade ago about working on open source software. A lot has changed in 10 years, but his advice has surprising endurance:

Don’t start by launching your own project… it’s more educational to read and learn from other people’s code.

If you haven’t lurked and participated in a free software project, you won’t know how things are done and you’ll have an awful time trying to run your own…

If you do start a project, the all-important thing is to write code. You have to code enough to make the app useful pretty much by yourself; this can be months or years of lonely work…

When it comes down to it, writing a free application is a huge amount of work. If you’re writing your own, schedule 10-20 hours per week, at least… And schedule those 10-20 hours every week, for the foreseeable future. If you can’t commit this much time — don’t even bother starting the project. If you can’t write code, ditto.

It takes about a year

It takes about a year to write a serious application. A year! Sure, it’s easy to knock off a proof-of-concept or a mockup, but once you get down to the business of making all the internals actually work, ensuring it is well tested, and making it build for other people it all just becomes a long grind.

And frankly, that year is just to get far enough to know whether the thing is going to fly or not. It takes somewhere between two and three years to get a serious application to the point where it’s polished and well-rounded enough where other people are able to use it.

And as Havoc points out, then if your app becomes widely adopted you’re in for years and years of continued involvement.

Why on earth do we do it?

Writing software, writing poetry

This week a friend introduced me to Jacket Magazine, a poetry and literature periodical. Tons of cool & eclectic stuff. It’s published entirely online, and is clearly a labour of love. Early on the founder & editor, John Tranter, wrote an essay for the about page. The piece mostly focuses on the economics of periodical production and how the internet has opened revolutionary possibilities for the distribution of literature. Hot topics today, right? Not bad for a piece written in 1999.

The best part, though, was when he tried to describe his motivation:

The main cost is my time; which means I don’t get much poetry written these days. … it takes up most of my waking life. Why do I do it? Jacket is hard work, and I like hard work. I enjoy editing the poems and articles and taking photos of people and designing the pages, and I even enjoy writing the HTML that underlies the pages. Jacket exercises all my various talents - and it’s fun.

It has also enlarged my circle of friends by a factor of about ten. And I feel I’ve enabled a lot of writers to find a wider international audience for their work, especially younger poets. I received a lot of generous support and assistance when I was a young writer, and it’s good to be able to give something back.

10 years later, he and his contributors are still at it. Beautiful.

I’m not a poet, but the compulsive dedication that he talks about resonates.

I am ashamed to admit that of the five or so major software projects I have initiated since I returned to writing open source in 2002, only two of them have made it past the “1 year” proof-of-concept point, and only one of them has reached the “usable by real people” stage — indeed it has taken over 3 years, and that project is a library, not even an end-user application!

This emphasizes a point that the agile development community have been making for a while now — you have to get to the point of your app being usable by {you, your customer, whoever} as unbelievably fast as you can. If you’re not able to enjoy the fruits of your hard work, then you’ll likely reach the point where you’ve put tons of effort in and it’s still a demo harness that you can’t actually use for what you wanted to use it for. Live and learn.

Or maybe not :)

I’m at it again; I’ve got another one brewing, and it’s passed the 1 year mark. The competing pressures are almost overwhelming. One the one hand is the excitement that it’s starting to come together, and that drives me forward. But at other times there is intense frustration when I discover yet another bug and when it feels that the thing is no closer to being usable (by me, let alone anyone else) than it was months and months ago. Sometimes you just want to let it all go.

And I have no doubt that it will be at least two more years before it reaches the level of polish and general usefulness that is merely the starting point for other people to consider using this program.

What in God’s name am I doing spending time on this?

The demon that drives us

I was trying to explain all this to someone in a bar a few weeks back, and she asked that very same question.

Are you doing it to make money?

Well, no.

Then why are you doing it?

Well, the coding I do has paid off, at least sort of. Most of my software architecture work is consulting to large companies, but all the experimentation and new ideas that drive those engagements comes from my open source work. And some of my clients have found me through my public hacking.

Oh, so, it’s a promotional activity?

Well, I suppose it ends up that way, but that’s not why I’m doing it. It takes too much time to justify that way. Perhaps in the long term people will use my software and think well of me for having written it. Probably not, though; likely they’ll just bitch about how it doesn’t do this or that that they think it should. Doesn’t matter, I’m not really doing it for them.

Who are you doing it for?

Me.

For three years!

Hm, yeah, probably longer.

So you’re not going to make any money out of this, you aren’t building your reputation with people who hire you, and you may not ever finish it. Why are you doing it?

and I finally found an answer that made sense. The right answer:

Because it needs doing.

Needless to say that was pretty much the end of that flirtation. Apparently non-materialism isn’t sexy. :)

Doesn’t change the fact that it really is the reason we do it. It needs doing.

As for the thing I’m working on? The feeling of accomplishment that comes when I show people screenshots of my work is undeniable. The code inside’s not bad, not bad at all. And I’m actually dogfooding with it now (ok, ok, almost). I think this one is going to fly.

At last.

AfC

one-year-and-more, posted by Andrew Cowie in software/freedom at 2009-08-02T15:12:13Z

Friday, 25 Jan 2008

Free Range Software

Jon Hall writes of his experience in a restaurant talking with its owner about “Free Range Eggs”:

“… but we have to charge money for our eggs. People who don’t acknowledge that just do not want to understand the term ‘Free Range’ for what it really means … better eggs, and changing the term will not help that.”

The fact that the discussion started because of maddog’s suggestion that maybe they should be called “open range eggs” to eliminate the confusion is not the point (“that’s silly” the owner said, “everyone calls them free range eggs”). The term we use, Free Software, has a bigger problem. Consider the difference between:

• Free Range Eggs
• Free Eggs

Clearly, the term is “Free Range”, and applies as an adjective to “Eggs”, whereas the latter really does mean “free eggs”. Now consider this:

• Free Software

There’s something missing, and so the term free gets connotated as having to do with price.

No, I’m not about to say that we should call it Free Range Software [and while “let it run free!” is a lovely metaphor, I don’t quite think we want to be associating our work with chicken farming :)]. Perhaps someone will come up with an intermediate word that will do the trick. To be honest, though I’ve pretty much given up on the term Free Software; I write Open Source software, and the cause I advocate is Software Freedom.

And when people still stare at you blankly, you can say “you know, like Linux” and watch as comprehension dawns. To be sure, they probably still don’t get it, but chances are you’ve got more important things to talk about, and getting on with it is going to do you — and logiciel libre — a lot more good than getting lost because of the insufficiencies of the English language.

AfC

Thanks to Atul Chitnis for having passed on the link.

free-range, posted by Andrew Cowie in software/freedom at 2008-01-24T13:17:23Z

Wednesday, 16 Jan 2008

Reusing Experience

I came across an interesting comment yesterday:

The documentation doesn’t help you much though. First, it is not sufficient and second (and important) you do not learn much from the documentation.

Thankfully, you have the source code and I really appreciate the source code of Eclipse is open. That’s because only from source you can learn as much as necessary. And this fact leads me to think more and more that open source is not about “reusing software” (commercial companies do that as well) but about “reusing experience” which is hidden inside the source.

I have the strong belief that people who write the software are more important than the software itself and by looking to the source code you can gain at least part of experience of people who wrote it. That’s the power of open source, that’s “reusing experience” concept at work!

This observation was written a few years ago by one Alexander Dymo who was expressing his amazement at the whole Software Freedom thing, especially the hands-on side of it. And as for his “people are more important than software”, well, hey; I couldn’t agree more. Enlightened organizations that want to preserve their strategic advantage understand that the people who are capable of working with such code are their competitive advantage. They are the ones who can reuse experience and leverage the power of the most wide reaching and enabling social phenomenon we have ever seen: the global openness movement.

Towards a technical definition of Open Source

I’m on a bit of a kick at the moment working to elucidate a practical technical definition of Open Source (ie, complementary to the necessary legal foundation which enables Free Software and the requisite social interaction which is at the heart of global Open Source communities). Comments like the one above are helping refine my thoughts on the topic: yes the interaction of licence and copyright law gives us structure, and yes communication between people distributed the world over is the genesis of the sense of community, belonging, and accomplishment which is the rich social fabric within which our software development takes place, but there is also a pragmatic aspect: can you actually work with the source code, get right into it, experiment with it, break it, and do crazy things with it?

That the four GNU freedoms stipulate that this must be “permitted” doesn’t really change the fact that there are practical prerequisites. Can you easily get the sources under development? Do those sources actually build? Is there a clear mechanism for contributing source back to the project and are they actively facilitating such contributions?

The source tarball as primary release artifact

The biggest give-away is whether the primary release artifact is source or an opaque binary.¹ Especially in the Java world but elsewhere as well, there is a surprising amount of activity for which, despite the fact that it may legally be Free Software and its loud proclamations to being Open Source, it remains clear that they just don’t get it: there are a huge number of projects and products which only do their releases in binary form. Bad sign when you start calling it a “product”, I think. In a frustratingly large number of cases, if you try to actually work with their code you will discover that it doesn’t actually compile! In extreme, there are statements like:

We distribute source but never claimed that you can build it out of the box. We don’t have time for such things.

I didn’t believe it when I saw it, but one of the hackers of a supposedly Open Source project actually said that in response to a bug report. Astonishing.

Being able to duplicate the result is a rigour that goes far beyond software; indeed it has been the bedrock of science — and human progress — since the dawn of the age of reason. Back to the present day, it has suddenly become obvious to me that the fundamental technical difference between proprietary Unix from the commercial vendors (not to mention proprietary operating systems from Microsoft and Apple) and Linux is that in our world (taken to mean the entire continuum of FOSS communities) the primary release artifact for all upstream projects is a source release (these days it’s typically in a .tar.bz2, but whether it is that or .zip or whatever else isn’t terribly relevant) that you can build. Not a tarball full of already built .class files and .jar files. Not a “source .zip” for Eclipse. Not a self-extractable full of .exes and .dlls. No. A source release is source code that someone else can build, right out of the tarball. THAT, ladies and gentlemen, is the technical definition of Open Source.

I’ve started to realize that this area is a big stumbling block. People I collaborate with in (upstream) global projects like GNOME, Free Java, Bazaar and elsewhere take the efficacy and primacy of source releases entirely for granted. But a number of clients that my firm is working with to enable Open Source often just don’t get why they should be doing source releases — and resist it.

Whether you buy into Software Freedom or not is a different topic (and your decision, not mine to impose on you), but if you’ve got a software project for which you’ve decided to free the source, you want it to be successful, right? Success is a project that which inspires user enthusiasm, which the major distributions can package and ship without hassle, and around which a vibrant community of developers grows, ultimately fostering new contributions. There are a lot of steps along the way, but a buildable source release is, in our view, the technical bar you’ve got to reach. Otherwise you’re not making releases of the software, you’re abandoning it. And your users.

AfC

¹Note that this is different from a Linux or Unix distro shipping binary packages — no matter if it was Fedora, Debian, Gentoo, FreeBSD, OpenSolaris or anyone else, they should still have been able to build their package from source. It matters little whether that compilation took place in a build farm somewhere or on the user’s desktop (which is what happens in the “binary” distros when someone wants to work on the package or the project itself) — the key point remains that we can build the software if we want to, and are not forced into relying on someone else’s proprietary (and more to the point, unavailable) tooling. If you can’t build it, it’s not Open Source.

reusing-experience, posted by Andrew Cowie in software/freedom at 2008-01-16T09:38:12Z

---