Callisto maintenance release complete, but…

DTP released 0.9.1 with the rest of the Callisto maintenance stream this week. This has got me to thinking about what a “release” means at Eclipse.

I’m often asked about which build of DTP should be used, and usually there's some surprise when I say to use the latest nightly one: seems that a lot of people are in the habit of using only releases or at least milestones. I started wondering why this preference exists. In a closed development effort, there’s essentially a black box into which you insert requirements, and out pops some software. There isn’t a choice here – you wait until it is “done” and take what you get at that point (“beta” not withstanding). Another factor might be waterfall-ish development processes: there’s a long period in which new features are added, but stability is not a concern. Next, there is a (often much longer) period trying to stabilize the code line.

But a strong majority of projects at eclipse.org use agility-inspired methods, especially tight iteration cycles of plan-do-finalize. Also, the current state of the project is transparent through Bugzilla, the mailing lists, newsgroups, published documents, and so on. A consumer can see (granted, with some effort) the current and projected state of the project at any time. This visibility would seem to enable frequent integration.

So, what do you get from a “release” at eclipse.org? There is some ceremony around the release event, including a release and intellectual property (IP) reviews. Given the transparent nature of project day-to-day operations, I’d be surprised if anything was a revelation during a release review. Also, each eclipse.org committer has studied the IP policies and respective PMCs police these areas. Perhaps the notion of “release” is related to closure, a time to stop and pat ourselves on our virtual backs for a job well done.

Anyway, for DTP and projects I work with at eclipse.org, there is a strong incentive to stay as close to the current code line as possible. I can say that DTP gets better over time – more features, more bugs fixed – and so using the latest is desirable. Sure, we make mistakes and introduce new bugs, but it is a lot easier to find and fix these when more eyeballs are closer to the build cycle. Also, if you do find bugs, missing features, or other things that you don’t like, then telling a project about such issues close to a release date usually means that you get a response like “good point – maybe in the next release.” Now, I assume that you don’t like that kind of answer, and I can tell you that we don’t like having to say it. During the ebb and flow of a development cycle, though, we might not be able to address an issue immediately, but at least we have a lot more flexibility to weave it into the project plan.

But that’s just me, how about you? Do you use releases/milestones or current builds? If releases/milestones, but would prefer using current builds, then what makes it hard to do so?

The Open Source Hunt

Recently Don Smith has blogged about the Factors Contributing to the Success of Open Source (part 2, part 3). I share an interest in these sorts of questions about open source -- why engage in open source development? What is "success" in this context? and so on. In particular, I've found it very useful to compare open source business models, project management, and development methodologies with those used by commerical projects/products. The more I've worked on open source and with open source communities, the more I've been struck by the similarities, rather than the differences, with the more traditional alternatives. Not to say that open source doesn't innovate in a number of important ways, but rather that it is very easy to overlook shared aspects. Doing so, we run the very real risk of attributing capabilities to open source that are unlikely, creating more heat than light by over-hyping the benefits of a so-called "new world" (or "paradigm shift").

Anyway, I find these sorts of comparisons fascinating, and would appreciate discussing them with the community. To start off a series of posts thinking about open source in this way, I'd like to introduce the open source hunt for your consideration.

Game Theory has been studying collaboration, cooperation and competition in a variety of contexts and has proposed a number of models. Although Game Theory has been criticized for being too technical (mathematical/abstract/etc.) and not applicable to "real life," I believe it offers many contributions that can help in thinking about certain situations. For example, what about open source? Drawing on earlier philosophical discussions, Game Theory has the notion of a Stag Hunt.

Essentially, a Stag Hunt represents a situation in which collaboration, though tricky, produces a higher benefit to all involved rather than individual action. The challenge is that the benefit from collaboration requires more work and risk to attain the higher benefit, while the lesser benefit from individual action is much more certain. There are some dramatic, real world cases where the difference has a great impact (examples in later posts).

As I work with the Eclipse community on behalf of DTP, I often encounter Stag Hunts. Whether it is collaboration between eclipse.org projects, or collaboration with the eclipse community at large, it is not uncommon to indentify areas in which everyone agrees that more benefit would come from working together, but other factors (such as migration, time line, and so on) scuttle the effort. Part of the challenge of moving DTP forward for the benefit of Eclipse as a whole is to overcome these concerns and achieve broader and deeper collaborative efforts.

OK, so perhaps it is interesting that some aspects of life at Eclipse can be seen through the lense of Game Theory's Stag Hunt. Apart from "cool trivia" factor, what does this observation provide? Well, the Stag Hunt is just a prototype for many situations that occur, and Game Theory gives us an analytical tool for understanding it. Also by making this linkage, we can use the accumulated experience about what works and doesn't work in these sorts of situations. So, the Stag Hunt serves both as a tool for analysis and common term for linking a substantial amount of collective human understanding, which can then be brought to bear on specific challenges in DTP.