Eclipse Building Blocks
During last week's Eclipse Council meetings, we discussed evolution of the "base" eclipse build system to include additional features and encourage projects not already using it (such as DTP) to jump on board. I agree that DTP should be using as much standard eclipse.org infrastructure and best practices as possible, both since there usually are very good reasons (often based on hard-won experience) for the choices made and to provide a consistent context for those extending eclipse.
One thing I haven't convinced myself about yet is the complexity of builds. At eclipse.org we talk a lot about "transparency" (public discussions and decisions, etc.), and in general I think we do a good job with it. But builds tend to be a different creature. It goes something like this:
- Get code
- Magic configuration
- Compile
- More magic
- Build done!
In many projects I've worked on, only the release engineering team could do a build. No, I don't mean could == allowed, but could == capable. Any even then, if the stars were not aligned just right and the magic words were not spoken with enough gravity, the build wouldn't finish.
I don't think this is very transparent ("now you see this code, now you don't, here's the build!"), and that worries me for an open source project. Aside from the principle, there are practical reasons why tricky builds are bad: developers, whether committers on the project or not, should be able to get a code snapshot from source control, make minimal configuration settings, and have a build. Now, this might take a while to run, and sometimes you can't help that. But you should at least have the option. If I am changing/experimenting with some core functionality, I'd like to verify how it all hangs together in the build. (Doing so in a dev environment is good too, but there are many errors that can creep in between dev and "binary" environments too. Ever forget to update your build.properties file to include new resources?)
The build system for DTP started out with the main goal of "run for anyone," but with time and pressure, cruft slipped in. Going forward, we'd like to migrate to the eclipse "base" builder, but would also like to re-capture the "run for anyone" goal if possible too.
Any thoughts/experience out there? Anyone want to help DTP with this (I think it would benefit us all....)?
posted by John Graham at 3:56 PM
3 Comments:
Before going for the full automated build with automatic checkout, etc. I recommend you to start simple with only PDE Build (the eclipse base builder supersets pde build) and build the smallest feature you have.
To do so please try the instructions in the PDE Build section of the help system (http://help.eclipse.org/help32/topic/org.eclipse.pde.doc.user/guide/tasks/pde_feature_build.htm)
Once you are done with that, add automatic checkout and additional magic and finally move to the releng base builder (note that what base builder is mostly glue specific to the eclipse platform team with respect to build publishing and testing).
9:48 PM
If you are interested in learning more about pde build and the platform build process these documents are good starting points
http://www.eclipse.org/articles/Article-PDE-Automation/automation.html
http://wiki.eclipse.org/index.php/Platform-releng-faq#Eclipsecon_2006_presentation
For us, the releng team is the only one that runs the builds. The reason is that our builds are very long (two hours for build, four hours for junit tests and 14 hours for performance tests). If everyone ran builds there would be scheduling conflicts with the test machines.
This probably isn't an issue for other projects that produce a smaller number of zips and fewer tests. Several of our teams run their junit tests before submitting their content to an integration build which allows them to validate their submission without running a full build themselves.
If you have questions regarding the eclipse basebuilder, please ask questions on platform-releng-dev@eclipse.org or eclipse.platform.
9:35 AM
Kim, le scal:
Thanks for the links and advice: DTP will certainly be looking into the base PDE builder going forward, and will use the forums mentioned to post questions.
-- John
10:10 AM
Post a Comment
<< Home