On the Data Platform

Learning BIRT, part 2

2008-04-02T15:45:00.002-04:00

(This is the second half of my review for Practical Data Analysis and Reporting with BIRT).

Chapters six through ten cover various ways that reports can be enhanced. Each of these chapters is relatively self-contained, making it easy to refer back for details later when writing reports. Chapter six describes how reports can be parameterized, obviously a necessary capability to promote reuse. In particular the distinction between data set and report parameters in BIRT needs to be understood, and chapter six does a good job in explaining this difference. Also explained are more advanced parameter concepts, such as dynamic, cascading and group parameters. While a number of these parameter concepts are covered quickly, if you are a Java developer as the book assumes, the explanation should be a sufficient overview, and the BIRT documentation can serve to fill in the details.

Report project and libraries are described in chapter seven. By using the Eclipse projects configured for BIRT and report libraries, further reuse is enabled. For example, images that need to be shared across a team of developers creating reports can be stored in libraries, and these libraries are then referenced by each consuming report. As with the previous chapter, Java developers should have no problem understanding these concepts, and the use of specific project types will be very familiar to experienced Eclipse developers as well. Chapter seven also contains a tutorial about reusing resources, and this is useful for checking understanding.

An important aspect of report development is being able to customize rendering. Obviously we’d like to separate rendering instructions from main report data (if possible), so changes in rendering can be made independently. In chapter eight there are examples of several style options BIRT supports: BIRT built-in styles, custom styles, CSS, and style templates. For simple formatting requirements either the built-in style support or slight customizations of it will suffice. Style templates are more useful to apply over a range of reports, probably across groups or departments. Finally, the capability to use CSS allows BIRT users leverage vast resources from that style language. The examples in chapter eight are brief, but detailed enough to suggest the possibilities in each option.

Charts are a common requirement for reports. Luckily, BIRT’s Charting Engine supplies a number of thirteen popular charting options, including scatter, pie, bar and line charts. Further, drill-down (the ability to see more detailed information for a specific chart element) is supported by the Chart Engine. Chapter nine uses the pie, gauge and bar charts in simple, illustrative examples. While table reports are common and useful, you really get to see the power of BIRT as a reporting tool through these chart examples. Well designed charts can convey a lot of information in an attractive form, and drill-down allow you to present additional details without cluttering the initial chart presentation. Perhaps because of the visual appeal of charts, I found the examples in chapter nine more interesting than those in other chapters, and this made me wonder if incorporation of charts throughout the book might have been a good strategy.

BIRT includes scripting support using Java and JavaScript. Chapter ten discusses these capabilities, interestingly starting with a comment that knowledge of Java is useful for understanding the scripting examples. Yet the assumptions stated at the beginning of the book include being a Java developer and, as I’ve mentioned several times, those without Java experience will have to work hard to grasp much of the book’s content. Perhaps in an attempt to limit chapter size or to keep it accessible to those without Java experience, the script examples in this chapter only scratch the surface. A minor criticism: much of the code has pedestrian comments (about things that method names, etc. should suggest) and subsequent paragraphs have explanations similar to the comments. It would have been better to omit these comments, hence making the code more compact. Event handling, an integral part of BIRT scripting, is also covered briefly in a few examples. My feeling is that this chapter should assume a fair amount of experience with Java and the ability to pick up JavaScript while showing more detailed examples. Granted this would increase the length of the chapter and still would only show a fraction of the possibilities, but a more comprehensive example would be more instructive to the (stated) target readership.

The final two chapters deal with report deployment and a case study. The deployment material is good to get started with, and ideally your deployment requirements will fit within the basic cases. But, as is often the case with real world deployment scenarios, likely there will be complications requiring studying further BIRT documentation for alternatives. The working example chapter is only suggestive – to follow the example exactly requires a lot of set up and I doubt few readers will attempt it. As a summary of many of the concepts covered earlier, however, the case study is a nice summary and useful for pulling all of the previous threads together.

In conclusion, I believe John Ward’s book does a fine job of providing a quick start to BIRT. If you are a Java developing using Eclipse and want to take advantage of BIRT, starting with the BIRT “all in one” download and working through Practical Data Analysis and Reporting with BIRT will quickly get you up and running.

Reclipse

2008-04-01T12:26:00.003-04:00

Hello everybody out there using Eclipse -

I'm doing a (free)Ruby version of Eclipse (“Reclipse” Just a hobby, won't be big and
professional like regular Eclipse) for Ruby and scripting language clones.  This has
been brewing since last april 1st, and is starting to get ready.  I'd like any feedback
on things people like/dislike in Eclipse, as my Reclipse resembles it somewhat (same
physical layout of the widgets (due to practical reasons) among other things).


I've currently ported JDT(3.4M3) and PDE(3.4M5), and things seem to work.
This implies that I'll get something practical within a few months, and I'd like to
know what features most people would want.  Any suggestions are welcome, but I won't
promise I'll implement them :-)

Stepping Aside…

2008-03-26T12:22:00.003-04:00

Three years ago the DTP proposal was posted for community review and at EclipseCon in Burlingame, I first met many of the people who would form the core DTP team. For me EclipseCon this past week marked the completion of that circle, as I have decided to leave Sybase and step aside as the DTP PMC chair. My new position, starting the week of April 7, will be with the JBoss division of Red Hat. I will remain involved in DTP and the wider Eclipse community, working with DTP to help in the transition, and to assist in completing the DTP 1.6 (Ganymede) release.

Last week at EclipseCon there was a lot of talk about diversity in projects. Diversity is more than simply the number of committers or the number of organizations represented. Real diversity is about redundancy in project operations, so staffing changes can more easily be absorbed. All successful projects rely on having the right people, but no successful project should depend on a specific person. I am confident that DTP will succeed after this transition at least as much as it would have under my leadership.

I’d like to thank everyone who supported me and DTP up to this point -- the Eclipse community is really an amazing group of people to work with – and I look forward to continued collaboration in my new role.

Learning BIRT

2008-03-13T16:28:00.002-04:00

Recently Packt Publishing contacted me about their new book, Practical Data Analysis and Reporting with BIRT (John Ward), and asked if I would be willing to review it. While DTP and BIRT have enjoyed a close relationship since the beginning of DTP and the early days of BIRT, I have never actually had a chance to use BIRT. Always interested in learning new technology, I figured reviewing the book would be a good chance.

It only takes a few minutes roaming around the Eclipse web site to realize that there’s a lot going on at Eclipse. It seems like new projects appear every week, and existing projects continue to push forward with new features. On the verge of EclipseCon next week, a survey of the Eclipse landscape yield a variety and depth that can be overwhelming. This is true whether you are a new user of Eclipse, or someone involved in working on an Eclipse project everyday.

There’s also no shortage of information about Eclipse. Mature, popular projects like BIRT tend to have numerous articles, books, web casts, and other source of information. The availability of information is great, but it can be hard for new users of a specific Eclipse technology to get up and running quickly. That’s where John Ward’s book fits: in about 300 pages, we are promised a “fast-paced, task-driven, tutorial style” introduction to BIRT, concentrating first on the BIRT Report Designer and later the BIRT Charting engine. OK, this sounds good, and I decided to take it step by step like a new user would. In this post and a series of follow ups, I will discuss my progress and observations. I hope that my commentary is useful for those considering how best to approach BIRT.

The first chapter is an overview of Business Intelligence (BI) and the BI market, with an emphasis on open source options. The author explains how he became interested in reporting tools, and this story helps to put the need for BIRT into the context of challenges that developers face. Frankly, the material in this chapter is a bit disorganized (for example, jokes about the name “BIRT” appear almost verbatim in a couple of places, leading me at first to believe that either the book had duplicated pages or I was reading it backwards.) While the coverage of BI and BIRT is adequate to get the essentials, the sections comparing BIRT to other open source reporting solutions (JasperReports and Pentaho) was sketchy and did not add much. After finishing the chapter, I couldn’t help think that perhaps part of it could have been skipped: the fact that I’m reading tutorial book about using BIRT seems to imply that I know enough about BI/reporting to understand that I at least need to learn how to use BIRT, and the fact that I’ve chosen a book about BIRT itself means that I’ve already made a (perhaps preliminary) choice of which tool to use. I understand that some introduction is necessary, and the first part of the chapter (describing the author’s need for a reporting solution) fits the bill.

The next chapter describes how to obtain and install BIRT. Since Eclipse projects tend to move fast and there are a number of ways to obtain distributions (various download packages, the Eclipse Update Manager), this can be a very confusing area. In fact, we in DTP have often diagnosed problems that arise from installing mismatched versions of various Eclipse components. Luckily, BIRT has an “all in one” download and the chapter recommends using it. I did, and quickly I had a working BIRT installation. (Detail: the chapter talks about the need to download the iText libraries separately. In BIRT 2.2.2, this is not necessary.)

Once you have BIRT installed and running, the first thing you’d like is a quick tour of its main features, and perhaps a simple working example. Chapter three provides this. First the basics of Eclipse (perspectives, views, etc.) are covered, and then a simple report project using the BIRT “Classic Models” sample database is created. This chapter has a “getting started with BIRT” feel, and does a good job of covering the main points in a brief presentation. Within a couple of minutes I was able to generate my first report, and everything worked as expected. Sometimes the hardest part of using software is simply getting the first set of tasks done: once you understand enough to do that, then well-designed software seems to “fall into place” as you learn the rest. Quickly scanning through the books remaining chapters, I found that they seemed to make more sense, now that I understood how to create even this very simple first report.

Of course there’s more to reports than simple charts: we need a variety of components for labels, images, and so on. Chapter four provides a quick tour of these visual components using the BIRT Palette. If you’ve used visual layout tools before, the BIRT components will be familiar, and they interact with the Properties view to set the usual parameters. One interesting difference between the book’s instructions and my development environment appeared when adding an image to the report. The book has us entering the URL for the image directly into a text box, but (at least in BIRT 2.2.2 on my machine), the text box was read-only. I had to open the Expression Builder and enter the URL text there. This was confusing because it seems (according the dialog’s appearance) that you should be able to enter text directly, but in fact it doesn’t work. Anyway, these minor deviations caused me to explore BIRT more, and I quickly understood the options the Palette provides.

After chapter four, the “quick start” feeling fades and the chapters get into more detail. Chapter five covers various ways of working with data sources, and this gives a key insight into the potential of BIRT. Because BIRT has a good assortment of data sources available (for example: databases, XML files, flat files, web services), you can easily imagine aggregating these for business intelligence scenarios. Without going into a lot of detail (and chapter five does cover a lot of ground), by the end of the chapter you will have seen many features provided by BIRT data access through the Open Data Access (ODA) layer. My only concern about the chapter is that, at over forty pages and a lot of material, perhaps it would have been better broken into a couple of separate chapters (maybe one extended example using databases, then another chapter skimming the other data sources?).

The first five chapters cover about one-third of the books length. Introductory material is crucial, since readers will not persist to later chapters if the early learning curve is too steep. My experience was very good: in a few relatively short chapters, I was able to learn enough about BIRT to create some really basic reports, and I felt ready to continue on to the later chapters. In future posts, I’ll discuss my experiences with the rest of the book.

Is DTP 1.6M5 ready?

2008-02-11T09:58:00.000-05:00

As we wrap up the DTP 1.6M5 cycle, we thought of trying a more transparent way for deciding to promote the milestone build. We described this method in the Approved Build Sign-off Process for DTP, and it was accepted by the PMC and project leads. So, following this new process for DTP 1.6M5, we designated a candidate build date, produced a candidate build on the designated date and looked for sign-off notices to be posted on the DTP PMC mailing list by the end of last Friday. At best we would have received three sign-off emails, at worst, zero. It only takes a quick scan of the DTP PMC email archive to see that the grand total was zero.

Today we’re essentially back to how we’ve promoted DTP milestones in the past. That is, in the absence of any bug severe enough to delay the milestone, we promote. No severe bugs being found is a good thing, assuming there is testing. One sure way not to find any bugs is simply not to test and hope for the best. Works sometimes with luck, but generally is not considered a best faith effort, especially when your components are the foundation for others in the release train. During previous DTP milestones, I have been smoke testing the builds, and I rely on the project leads to work with their teams to do more complete testing. We think that this testing is taking place, but do we know? Do you want to know?

So, this leads me to my request for this week: I’d like to get feedback from the community about your expectations. Was the previous (“see no evil – promote”) process good enough, or would you like your DTP project leaders to say that they’ve completed a best faith testing effort and the build is milestone/release candidate/release ready? I have no interest in tracking and seeking conformance to policies that the community does not gain value from. In DTP 1.6M5 we’ve resolved 65 bugs. Does silence mean “ready to go?”

This week please send your thoughts about the sign-off process to the DTP PMC mailing list or the newsgroup. We’ll collect these and make a decision next week about whether to rescind the sign-off process or to delay promotion of future DTP milestones in the absence of sign-offs.

Of peguins and the end of the year

2007-12-21T12:48:00.000-05:00

Peter and Roman commented on my Penguin Design post, and I got a number of other comments/questions from other channels too. A number of people asked what I meant to say, and I had to answer honestly: I'm not sure. It just seems like such a natural connection to make, and I think there's a number of associations you can draw from it. Let's see... on my bookshelf there are two books with penguins on the front cover: Segaran's Collective Intelligence and Chamley's Rational Herds. Over the holiday break I think I'll look more at these to see if I can clear up my thinking about Penguin Design :-)

Speaking of holiday breaks, I've always enjoyed the end-of-year pause, since it gives me time to decompress a bit and take stock of the past year. One point is already becoming clear though: while I'm used to a fair amount of chaos, generally I've been able to point to unifying theme or purpose in the flow. But, looking back at this year, I'm not sure I can do that. Yes, it was hectic and chaotic, but probably not more than usual. Yet, I'm having a hard time identifying the how it all "hangs together." I'll need to fix this next year...

Penguin Design

2007-10-31T12:04:00.000-04:00

This weekend when trying to describe open source software to a friend, a neat association came to mind. Many of us have heard that "a camel is a horse designed by committee." What about open source. Well: "a penguin is a bird designed by open source!"

Assuming that I had heard this somewhere before, I searched but could not find it. Interesting: it seems like such an obvious connection to make :-) If anyone knows of a source for that statement, please let me know and I'll make the proper attribution here....

EclipseCon Submission Deadline: Sooner than you think

2007-10-24T10:57:00.000-04:00

Sure, November 19th sounds like it's a long time away. Especially if you're a PHB: "We could build an entire operating system by then!"

But really -- October 25 is tomorrow, people! And October 26 is right after that! (Work with me here...) and then it will be November 20th before you know it, and all your friends will be laughing at you for missing the EclipseCon deadline. We really don't want that: everyone looks foolish when one of us is diminished in that way.

So, come on -- there must be dozens of you Little Bobby Tables out there who could give an excellent talk about DTP! Let's have those submissions!

Open Source Silver Bullet

2007-10-23T11:51:00.000-04:00

Recently Jörg Siever made a very interesting keynote speech at Eclipse Summit Europe about why VCs like open source companies. Even if you're not involved with venture funding, I still recommend this presentation for understanding open source business models. One of the comments he made (about open sourcing unpopular applications) reminded me of an open source antipattern I've seen a number of times.

In brief, open source is not a silver bullet for software product development. In addition the productivity issues that Brooks discussed, open source does not absolve companies from the requirement of providing valuable products and services. Simply producing open source software does not mean that money from adoring users will flood in. This might seem obvious, but I assure you it is not in many cases. Rather, a very common mistake starts from wishful thinking and ends in disappointment when open source involvement does not automatically translate into product sales.

Open source is a method of production, one which of course has advantages and disadvantages. For me the most intriguing aspect of open source is that the full potential of its advantages is only now beginning to be understood and leveraged. And, yes, there are real disadvantages that need to be understood and accepted; otherwise unrealistic expectations will certainly be the source of later frustration.

DTP at OS Summit Asia

2007-10-05T15:43:00.000-04:00

OS Summit Asia is not too far away (26-30 November), and DTP team members will be giving a number of presentations:

Data Tools Platform (DTP) enablement training by Dafan Yang
Building Custom Data Sources with Eclipse DTP Open Data Access by Linda Chan
What can DTP Provide for You? - A Getting Started Guide by Hui Cao

It sounds like a great conference, so, if you're in the area, be sure to attend!

BTW: There's no picture for Linda? Hey, Linda -- where's your picture?

Eclipse Summit Europe

2007-10-02T11:43:00.000-04:00

I'm looking forward to next week's Eclipse Summit Europe in Ludwigsburg, Germany. I really enjoyed the format last year, and, from all indications, this year is going to be even better! If you're going to attend ESE, be sure to stop by my talk or take another chance to introduce yourself. I'd also be happy to schedule meetings, if you'd like to talk with me about DTP: please drop me a line on the Summit Network.

Innovation Shunned

2007-09-30T16:05:00.000-04:00

Adam Hanft has blogged about reasons why a company might have rejected proposals for the iPhone:

• “Consumers will reject a keyboard they can’t feel.”
• “No one will spend that much for a phone.”
• “You can’t expect people to get a new number online.”
• “You can’t succeed working with just one carrier.”

Now, whether or not the iPhone ultimately proves to be a success, Adam's point is very important. "Innovation" is talked about everywhere these days, and everyone wants some. Usually the concern is about idea generation: we've got to come up with some innovative ideas!

But, in my experience, there's no shortage of ideas around, and some of them are (naturally) quite good. Rather, I think the biggest barrier to delivering on innovation is the kind of reasoning cited by Adam. It's always easy to find reasons why something might not work, but it is very hard to grow a new idea into a successful product. The warm and fuzzy feeling of familiar technologies, business models, organization charts or corporate culture keeps people jumping in place, all the while chanting that innovation is important. To drive success from innovation requires shouldering of risk, and acceptance of failure, flexibility and persistence. These qualities are rare, and hence we really notice when a critical mass of them succeeds with the "next big thing."

Are you contributing to Eclipse?

2007-09-21T16:34:00.000-04:00

I just returned from the Eclipse Members’ meeting near Chicago, and there was some discussion about just what it means to contribute to Eclipse. This question has come up periodically, and seems to be more popular when projects are in planning mode (and hence trying to figure out what to do and who will do it). It certainly has been a recent topic of discussion in the DTP community, and I had more than one conversation at the Members’ reception around it as well. So, what does it mean to “contribute” to Eclipse?

There are many ways to make contributions, and we probably could debate a list for a long time without getting to a consensus. Luckily, the key point of my argument below doesn’t depend on a definitive list, so we can just make an estimate. Broadly speaking, we could classify contributions into three categories: definition, implementation, and use. By “definition” I mean things like use case and requirements gathering, design and architecture specification and so on. Clearly there are a number of ways to contribute to definition in Eclipse projects. Next, “implementation” is pretty obvious: mainly we’re talking about providing code (including unit tests, samples, etc.). Finally, “use” would cover contributions resulting from using the software. The most obvious candidate for this category is bug reports, but I’d also include efforts to drive adoption and help users/extenders succeed. Note that these are not strict categories and the process is interleaved. For example, during usage enhancement requests (definition) appear, or during implementation, certainly bug reports from usage are made.

Like I said, nothing really hangs directly on the exact list, though I think that most would agree with the rough boundaries suggested. Now, let’s get on the main point of this post: which contributions are most important (valuable, welcomed….)? Two types of answers immediately come to mind: I could give a feel-good “every contribution is valuable!” answer, but that’s just not my way. :-) Or I could propose a ranking, which would end causing more controversy than even the list of contribution types. From this controversy, and – no doubt – the lack of consensus about any one ranking, I would begin to suspect something. Next, I observe the feeling that ranking contribution types in a strict order somehow seems inflexible. Lack of agreement about ranking and inflexibility at even the thought of ranking tells me that something else is going on.

I propose that various types of contributions to Eclipse projects do have relative importance rankings but that these rankings change over time. I also claim that this is the case, regardless of what your list of contribution types is. Finally, the importance of particular contributions at a particular time varies depending on the “level” of your view. For example, DTP Ganymede as a whole might have one set of rankings, whereas specific components in DTP Ganymede would have others.

If we accept that contribution types vary in importance over time and level of granularity, then I think we can begin to understand some patterns depending on where a particular project (component, plug-in, etc.) is at a given time. For example, components undergoing substantial growth (including being born), such as the DTP Ganymede Visual SQL Query Builder, put an emphasis on implementation contributions over other types. Mature components, such as DTP ODA, might value usage contributions to refine example, API definitions and so on.
So, yes, all contributions are valued and welcomed. But certain types are more valuable and welcomed, depending on conditions at the given moment.

Finally, let’s explore an interesting implication of this idea. I think it’s fair to say that implementation contributions are generally perceived as being more valuable by project members. The appeal is that those producing the code seem to be doing – in some way – the crucial work. And – of course – if there is no implementation, then the definition was all talk, and usage is precluded. Following this, we tend to measure contributions to project based on commit event (or lines of code) to CVS. Those most active in this area (and by extension, their organizations) are contributing more to given project.

But the emphasis at Eclipse has been on ecosystem development: just slinging out code is not the goal. I think Bjorn in particular has mentioned this several times. So, turning the code-slinging logic on its head, I’d say that if your major effort in an Eclipse project is to push code out, you’re not contributing.

Before you flame me, realize that the preceeding statement was made simply to draw a sharp contrast to the code-king attitude discussed above. I know it seems ridiculous to say that only coding is not contributing to Eclipse, but you could also argue that code contributions as the main metric of overall contributions is equally wrong.

Visual SQL Early Access Download

2007-07-20T15:51:00.000-04:00

As I mentioned earlier, work is beginning in DTP for the Visual SQL Query Builder. You can now download an early access build of this component from our download page ("Early Access" section).

Please give it a try and let us know what you think!

Visual SQL

2007-05-25T14:44:00.000-04:00

Like many other Eclipse.org projects, DTP is now starting to discuss what happens post-Europa. One major new component post-Europa will be the eagerly awaited Visual SQL Query Builder (or "VisQ" as all the cool kids say).

Samir, Brian and Jeremy have been working hard on some initial documents and prototype code. This is a great time to get involved! Contributions from community -- requirements review/input, design help, offers to help develop, test, and so on -- are all very valuable, and will help VisQ to evolve in a way that is most useful to the entire Eclipse community.

For a start, head on over to the VisQ requirements wiki page and join the conversation...

The Importance of Being There

2007-05-23T17:45:00.000-04:00

One of the themes running through my thinking about open source has been the application of "traditional" economic and business principles to open source. At EclipseCon '07 I made a presentation about some of these ideas, and I have another blog in which I'm continuing the conversation. This week I noticed another of these parallels: the importance of location.

"Location, location, location" is an old adage about the importance of finding the right physical spot for your business. Likewise, in business strategy/marketing there's a lot of talk about "positioning," which can be seen as creating the impression of a location in the customer's mind. Online and in the virtual competitive landscape of open source, location matters too. There's a great advantage to being an Eclipse.org project, and there are a number of prime "real estate" spots as well.

So, I found it interesting that there's a bit of uproar about Sun's perceived attempt to conflate Java and NetBeans. Sun has a great location here: when you go to download Java, you're going to a site associated with Sun, and that's powerful since there's a lot of people interested in Java. Next, you see information about NetBeans in the same location, sort of being positioned like "this is the IDE for Java development." Especially if you're new to Java, you might not look elsewhere. Would the Java download site that offers NetBeans ever say something like "Here's a few free Java IDEs -- Eclipse, NetBeans, etc.?" Somehow, I doubt it, but the uproar cited in the article might bring some pressure to bear. I mean, if I owned the most obvious (but not only) gas station just outside the car rental return at an airport, wouldI put a map up highlighting my competitor's location as well?

Maybe the Java/NetBeans issue is different: the attempt to build an open source community might force that map to competitors, and that would be interesting to see...

Harmony and Eclipse?

2007-05-10T16:38:00.000-04:00

So, there's been a lot of interest in the Eclipse/Harmony combination at JavaOne as Ian reports. While I don't know enough about the Harmony project to have strong feelings one way of the other about it, there is certainly one thing I like: it offers the community more choices in open source, something that I've been growing increasingly interested in.

Make your mark on DTP

2007-05-10T16:32:00.000-04:00

DTP is hosting a logo contest, so here's your chance to make a mark on DTP. Wouldn't it be really cool to know that a graphic you created has spread far and wide with DTP, and countless people admire you for the contribution?

So, go ahead -- let's get some submissions!

Blog fork

2007-03-13T12:06:00.000-04:00

Over the past year and a half I've mixed discussions of DTP with more general notes about open source in this blog. Based on conversations last week at EclipseCon, I feel that it is time to separate (mostly) these streams, and hence have created the Open Source Musings blog. I do not plan to ask that this new blog be aggregated on Planet Eclipse.

I look forward to continuing discussions there as well, especially with a number of people in my Garden & Market talk who expressed an interest in such a forum.

DTP and Language Environments

2007-02-15T16:41:00.000-05:00

Although Eclipse's Java tools have been around since the beginning, other languages have been supported as well. For example, C/C++ tools were an early addition, and today we find strong initiatives around many other languages. As DTP continues to evolve, one area we've been looking at is how its tools can be leveraged in various language environments.

For example, I have a talk at EclipseCon that concentrates on integrating DTP and JDT. Though the emphasis will be on Java, we will also talk about possibilities for integration with other language environments. A good example of why this is a compelling direction for DTP appears in a recent blog entry by Bill Beebe. Bill, who has just returned to evaluate 3.3M5, discusses the strengths of ROR tools in Eclipse. He finds a strong case for Eclipse and its ecosystem of plug-ins, but he also mentions that "SQL database integration still leaves something to be desired." I'm not sure whether the gaps are in the core tooling or integration with languages like Java and Ruby, but we at DTP are very interested in exploring these areas. The EclipseCon talk will be an attempt to drive this conversation, and of course there are many channels -- mailing lists, bugzilla, newsgroups -- where additional discussion can occur.

Living, breathing projects

2007-02-09T14:29:00.000-05:00

Doug DeCarlo has argued that "Projects are living, breathing things. They are people."* Don't believe it? Want proof? Well, EclipseCon is a great place to find out! There's a huge number of interesting sessions this year, covering an amazing range.

There also is a variety of sessions related to DTP and I have listed them on Eclipse DTP wiki. I'm really looking forward to EclipseCon. If anyone would like to meet in person, please feel free to drop me a line.

* Extreme Project Management, p.33.

Eclipse Council Follow Up

2007-01-29T12:34:00.000-05:00

Back from the Eclipse Council (and other) meetings last week. As PE loaded this morning, I glanced over it and saw the Far Side reference from Bjorn. Naturally, I assumed he was using it to demonstrate something about the Architecture Council, but then I saw the bit about the elevators...

Eclipse Council Meetings

2007-01-19T17:16:00.000-05:00

Next week the Eclipse Councils will meet near San Francisco. Since this is the first face-to-face meeting of the year, and since it is Friday afternoon, let's run a small experiment.

The Requirements Council provides input to the other Councils, as well as Eclipse projects in general, about community themes, priorities, and specific requests. This input is very important, and Requirements Council does a great job in bringing a diverse set of perspectives to bear in forming their recommendations. The Planning Council does, well..., planning. Specifically, we work on the overall plans for the coordinated releases (Callisto last year, Europa this year). While a lot of planning does happen online, it is hard to imagine an effort the scope of Callisto or Europa if project representatives didn't sit down together in person once and a while to ~~enforce~~ agree upon a common vision and direction.

Now, here's an exercise for the reader: without repeating the text on the Council page, say a few words about what you imagine the Architecture Council does in practice. What does it bring to the Eclipse community?

DTP 1.0 released

2006-12-22T16:28:00.000-05:00

DTP 1.0 has been released and is now available on the download site.

I'd like to thank everyone in the community who helped DTP achieve this important goal. As Bjorn said in the release review, this is an "important step." Yes, it is important , because with 1.0 we move from incubating to mature status as a project. It is also a step, one move forward along the DTP path. We have a number of exciting ideas for DTP in 2007, not the least of which is participation in the Europa coordinated release. Also, there are a number of DTP presentations scheduled for EclipseCon, many of which will explore these directions.

We at DTP are looking forward to working with an expanding community in the coming months -- please keep telling us (newsgroup, Bugzilla, mailing lists, etc.) what we are doing well, where we need to improve, and what you'd like to see! And, making on final turn on a recent PlanetEclipse theme, since Eclipse is all of us, we really appreciate all the help the community can offer! :-)

Eclipse is you, and you are all different

2006-12-18T16:41:00.000-05:00

Eclipse is who? It's you, says Bjorn and Doug points out that this dovetails nicely with the Time Magazine Person of the Year. This sort of image resonates with the ideal of open source, but in practice runs into some complications, as Ian has discussed. Yet I'm not sure that I'd say, following Ian, that some are more important than others in the Eclipse ecosystem. Sure, committers can make decisions that others, say users, can not. But users can vote with their feet, and being a committer on a project without users is no fun. Rather, there are different roles, responsibilities and powers for different members of the community. While committers, project leads, the EMO, and so on tend to get a lot of the visibility, the plain fact is that Eclipse would be nowhere today without the other members of the ecosystem. And the impact of change on ecosystems is famously difficult to predict: large, unpredicted consequences can appears from seemingly small events.

Anyhow, I do take the original point, and I can understand the frustration that Ian expresses. This sort of thing has been bothering me for a while. So here it is: I think we on the committer side of projects at Eclipse make it much harder than necessary for others to participate. I have some ideas to help with this, but I'll wait to see if others agree that there is a problem in the first place, before taking blogspace with proposals.

The Eclipse Mine

2006-12-05T13:21:00.000-05:00

It's natural enough when thinking about IBM and open source to also think about Eclipse. When I think of "Eclipse" and "mining," my first association is usually BIRT. But now there's another link in the chain: Marc Fleury says that IBM's approach to open source is "strip mining." An interesting image, perhaps, but not one that makes a lot of sense to me. Assuming for a minute that Marc's argument holds, we still do not have something analogous to strip mining, because others, in addition to IBM, are able to obtain benefit from the open source in question. In strip mining, however, the resource is extracted only once. In other words, open source is nonrival.

But there's an interesting twist in Marc's argument. He says: "...dual strategy of proprietary products and low-end open source." What is this "low-end open source?" Does it apply to Eclipse and, if so, what are the implications?

Anyway, if there is an Eclipse mine, then who/what is the canary?

Integrating DTP and JDT

2006-11-29T16:54:00.000-05:00

Since almost the beginning of DTP, we've had multiple requests (examples: 113029, 105202) for integration of data tooling with Eclipse Java development components. For the first couple of releases of DTP we've had to focus on building foundations. As DTP 1.5 ("Europa") looms, the DTP PMC thought it might be a good time to revisit DTP/JDT integration and get a feel for what is possible and desirable from the community.

So, here's a proposal for an EclipseCon long talk -- I'd appreciate any comments on it using Eclipsezilla if possible.

Eclipse is smarter than me

2006-11-14T12:50:00.000-05:00

When a group of people* get together to perform a collective action, some interesting dynamics can emerge. For example we could end up with the Mob Madness or the Wisdom of Crowds. Since Eclipse is a group of people acting together in building out a common open source platform, let’s consider which alternative emerges. For me, though no doubt some will disagree, I think it is Wisdom of Crowds. In other words, Eclipse is smarter than me.

And this is good thing: we are able to leverage the collective knowledge, experience and wisdom of the Eclipse community (in the broadest sense) to shape future directions. One consequence is a self-correcting system in the long run. While there might be short-term deviations from the best path, the community has a way of adjusting course given enough time. So, criticisms and competition are good, and different perspectives shared in transparent communication channels are great.

What do you think? Is Eclipse smarter than you too?

* or "agents"

DTP Requirements Call

2006-11-03T15:52:00.000-05:00

The DTP PMC will be hosting a requirements call next week. Actually, there are two meetings scheduled, both covering the same material, in order to accomodate as many time zones as possible.

The purpose of this call isn't so much to discuss specific requirements in detail, but rather cover themes and priorities through our Europa release (and beyond), and to hear from the community. (Of course, general discussion about particular requirements is welcome.) Especially with WTP's recent move to DTP in their 2.0 stream, the DTP PMC wants to make sure that the community knows we're listening, and understands how best to express requirements to the team.

Using DTP with the JDT

2006-10-26T16:04:00.000-04:00

The 10/26 build of DTP 1.0M3 adds a new plug-in containing a first step in integration with Java tooling. This plug-in, org.eclipse.datatools.connectivity.jdt was created based on enhancement request 117318. So, what does it do?

If you go to the Java Build Path-->Libraries tab in the JDT (either when creating a new Java project, or on the properties of an existing Java project):

You then select Add Library:

And notice that there is a "Connectivity Driver Instance" category available. Selecting that category opens the driver page:

You then choose from the set of existing connection profiles, and the associated jar is added to your class path:

There's been interest using DTP with the JDT for a while (for example, bug 113029), and it is good to see a first step in that direction.

We'd be very interested in suggestions about how DTP could be used with the JDT, and of course we wouldn't turn away proposals for contributions :-)

The City Hall and the Forum

2006-10-18T10:58:00.000-04:00

I'm back from the Eclipse Summit Europe held last week in Esslingen, Germany. Also attended the Eclipse Council meetings at the same time, though in a different building. The Summit was held in the Neckar Forum, and the Council meetings in the historic city hall. This brought to mind the notion of the Cathedral and the Bazaar. In this case, however, we have the City Hall and the Forum! Is Eclipse the City Hall, a Forum, or something else?

Anyway, the Summit was great -- I really enjoyed the scheduling which gave ample time for informal meetings, without missing the main sessions.

At Eclipse Summit Europe

2006-10-10T19:21:00.000-04:00

I'm at the Eclipse Summit Europe this week. Or, rather, I just arrived near the location, and am trying to adjust to the time lag before the Summit and Eclipse Council meetings start. So far everything has been very good -- the hotel that I' staying at (in Stuttgart) is not particularly close to the meeting site, but the staff have been very friendly, even when I roam around the hotel and sit in strange places to get the best wireless connection. :-)

If anyone would like to meet at the Summit to talk about DTP or otherwise, please drop me an email and we'll set something up.

Thanks for sharing

2006-10-04T10:46:00.000-04:00

Bjorn recently blogged about the importance of Eclipse projects being “open, transparent and permeable.” I completely agree, but think there are some hard questions around this.

First, there’s the volume of information. For any given Eclipse project, there are a number of communication channels. In no particular order:

web site
mailing lists
wiki pages
newsgroups
Bugzilla
articles
source code
other mailing lists associated with the project (e.g. Eclipse Councils)
meeting minutes
conference presentations and discussions

There’s probably more – 10 seems like a reasonable example set. Now, let’s multiple these channels by the number of projects I might be interested in (some projects might not use a particular channel, others might have specific channel… you get the idea…) and that’s a lot of information to absorb. Too much: I’m deeply involved in DTP, and I’m running as fast as I can to keep up with the DTP channels, never mind other projects.

Certainly I pay attention to other projects (example: the platform… who doesn’t?), but my view is a specific slice of their information content. There’s simply not enough time for me to do otherwise.

What I need is information summaries or alerts for topics I care about. I appreciate knowing about technical discussions, and having input into decisions I care about. But how to I isolate this information? I submit that another mailing list, wiki page, or such is not the answer… Telling me “it is there, just read this or that” misses the whole point.

Secondly, there is a back-channel of communication for dealing with individuals or organization/companies who want to have a “private” conversation. There are a lot of reasons why, and maybe I’ll blog about it later if people are interested. Most Eclipse project members I know encourage these conversations to come “into the light” as soon as possible, but that doesn’t mean there isn’t a private component as well. Do you think this is a bad idea, out of the spirit of Eclipse? Maybe. Are all Eclipse project members willing to step into a panopticon with the community? I would, but only if I were convinced that everyone else would too (otherwise, I cut off an important and useful channel without corresponding benefit).

Callisto maintenance release complete, but…

2006-09-29T12:42:00.000-04:00

DTP released 0.9.1 with the rest of the Callisto maintenance stream this week. This has got me to thinking about what a “release” means at Eclipse.

I’m often asked about which build of DTP should be used, and usually there's some surprise when I say to use the latest nightly one: seems that a lot of people are in the habit of using only releases or at least milestones. I started wondering why this preference exists. In a closed development effort, there’s essentially a black box into which you insert requirements, and out pops some software. There isn’t a choice here – you wait until it is “done” and take what you get at that point (“beta” not withstanding). Another factor might be waterfall-ish development processes: there’s a long period in which new features are added, but stability is not a concern. Next, there is a (often much longer) period trying to stabilize the code line.

But a strong majority of projects at eclipse.org use agility-inspired methods, especially tight iteration cycles of plan-do-finalize. Also, the current state of the project is transparent through Bugzilla, the mailing lists, newsgroups, published documents, and so on. A consumer can see (granted, with some effort) the current and projected state of the project at any time. This visibility would seem to enable frequent integration.

So, what do you get from a “release” at eclipse.org? There is some ceremony around the release event, including a release and intellectual property (IP) reviews. Given the transparent nature of project day-to-day operations, I’d be surprised if anything was a revelation during a release review. Also, each eclipse.org committer has studied the IP policies and respective PMCs police these areas. Perhaps the notion of “release” is related to closure, a time to stop and pat ourselves on our virtual backs for a job well done.

Anyway, for DTP and projects I work with at eclipse.org, there is a strong incentive to stay as close to the current code line as possible. I can say that DTP gets better over time – more features, more bugs fixed – and so using the latest is desirable. Sure, we make mistakes and introduce new bugs, but it is a lot easier to find and fix these when more eyeballs are closer to the build cycle. Also, if you do find bugs, missing features, or other things that you don’t like, then telling a project about such issues close to a release date usually means that you get a response like “good point – maybe in the next release.” Now, I assume that you don’t like that kind of answer, and I can tell you that we don’t like having to say it. During the ebb and flow of a development cycle, though, we might not be able to address an issue immediately, but at least we have a lot more flexibility to weave it into the project plan.

But that’s just me, how about you? Do you use releases/milestones or current builds? If releases/milestones, but would prefer using current builds, then what makes it hard to do so?

The Open Source Hunt

2006-09-22T11:48:00.000-04:00

Recently Don Smith has blogged about the Factors Contributing to the Success of Open Source (part 2, part 3). I share an interest in these sorts of questions about open source -- why engage in open source development? What is "success" in this context? and so on. In particular, I've found it very useful to compare open source business models, project management, and development methodologies with those used by commerical projects/products. The more I've worked on open source and with open source communities, the more I've been struck by the similarities, rather than the differences, with the more traditional alternatives. Not to say that open source doesn't innovate in a number of important ways, but rather that it is very easy to overlook shared aspects. Doing so, we run the very real risk of attributing capabilities to open source that are unlikely, creating more heat than light by over-hyping the benefits of a so-called "new world" (or "paradigm shift").

Anyway, I find these sorts of comparisons fascinating, and would appreciate discussing them with the community. To start off a series of posts thinking about open source in this way, I'd like to introduce the open source hunt for your consideration.

Game Theory has been studying collaboration, cooperation and competition in a variety of contexts and has proposed a number of models. Although Game Theory has been criticized for being too technical (mathematical/abstract/etc.) and not applicable to "real life," I believe it offers many contributions that can help in thinking about certain situations. For example, what about open source? Drawing on earlier philosophical discussions, Game Theory has the notion of a Stag Hunt.

Essentially, a Stag Hunt represents a situation in which collaboration, though tricky, produces a higher benefit to all involved rather than individual action. The challenge is that the benefit from collaboration requires more work and risk to attain the higher benefit, while the lesser benefit from individual action is much more certain. There are some dramatic, real world cases where the difference has a great impact (examples in later posts).

As I work with the Eclipse community on behalf of DTP, I often encounter Stag Hunts. Whether it is collaboration between eclipse.org projects, or collaboration with the eclipse community at large, it is not uncommon to indentify areas in which everyone agrees that more benefit would come from working together, but other factors (such as migration, time line, and so on) scuttle the effort. Part of the challenge of moving DTP forward for the benefit of Eclipse as a whole is to overcome these concerns and achieve broader and deeper collaborative efforts.

OK, so perhaps it is interesting that some aspects of life at Eclipse can be seen through the lense of Game Theory's Stag Hunt. Apart from "cool trivia" factor, what does this observation provide? Well, the Stag Hunt is just a prototype for many situations that occur, and Game Theory gives us an analytical tool for understanding it. Also by making this linkage, we can use the accumulated experience about what works and doesn't work in these sorts of situations. So, the Stag Hunt serves both as a tool for analysis and common term for linking a substantial amount of collective human understanding, which can then be brought to bear on specific challenges in DTP.

Visions of data from the platform

2006-08-31T20:41:00.000-04:00

It's not uncommon for people to ask me what my long-term goals for DTP are. That is, they are asking about my vision for DTP. Now, "vision" is an interesting term... Looking on Wikipedia, I find the following two senses (among others):

a corporate long-term goal
A visual or generic hallucination

I'll leave you to decide which category the following belongs in...

The Eclipse platform has been a run-away success, and many projects (of course, DTP) ride its coat tails. This is all well and fine, with user reasoning being something like:

I want to use Eclipse! It's really cool!
Hmm... I need something to work with data sources like databases....
Well, here's DTP, and it is in Eclipse
So, I guess I'll use that too

No problems here. DTP thanks the platform for the comfortable ride.

But my vision for DTP brings an opposite reasoning process:

I want to use DTP! It's really cool!
Hmm... It's built on Eclipse
Ok, I'll use Eclipse then

Simple, no? In one sentence: I want some significant number of people to start using Eclipse because they wanted to use DTP.

Are we there today in DTP? We've worked really hard over the past couple of releases, and made a lot of progress. But, are we there yet? Honestly, no. Will we ever achieve this vision? Who knows (that's the sign of a good vision, I think), but it sure will be fun trying!

OK, I've stopped having visions...

EclipseWorld

2006-08-18T12:04:00.000-04:00

EclipseWorld will be held September 6--8 in Boston, and there's a lot of interesting sessions planned. Based on the web site's main page, Mike certainly seems more than happy to have a talk there. :-)

For those interested in learning more about DTP, Brian Payton and I will be giving a talk on Thursday morning, 8:30 -- 11:00:

How to Improve Database Connectivity With the Data Tools Platform

Loic Julien has a talk on Thursday afternoon, 1:15-2:45:

Architect and Build High-End Data Tools Solutions

While not directly related to DTP, my coworkers Maria and David will present along with Lee Anne Kowalski:

Developing Integrated Help and Documentation for Eclipse Applications

While Eclipse offers rich support for user assistance ("help"), these capabilities are not often exploited by eclipse.org projects. The lessons presented in the presentation above can help to flatten the learning curve for those interested in leveraging the full potential of the help components in Eclipse.

Finally, if anyone is interested in meeting with me to talk about DTP, please drop me a note, and we'll set something up!

Enablement now a project in DTP

2006-07-13T14:16:00.000-04:00

Yesterday the Enablement passed creation review, which means that it is now an official project in DTP. As I described in an earlier entry, the PMC sees Enablement as a very important part of DTP's future, and is looking forward to working with various groups to build out wide and deep data source support.

I'd like to take this chance to extend once again an offer to the community: If you have a data source that you'd like to use in Eclipse, why not come and work on the Enablement project with us? There are many benefits to doing so, not the least of which is being able to work with a vibrant and talented committer community at eclipse!

Eclipse Building Blocks

2006-07-06T15:56:00.000-04:00

During last week's Eclipse Council meetings, we discussed evolution of the "base" eclipse build system to include additional features and encourage projects not already using it (such as DTP) to jump on board. I agree that DTP should be using as much standard eclipse.org infrastructure and best practices as possible, both since there usually are very good reasons (often based on hard-won experience) for the choices made and to provide a consistent context for those extending eclipse.

One thing I haven't convinced myself about yet is the complexity of builds. At eclipse.org we talk a lot about "transparency" (public discussions and decisions, etc.), and in general I think we do a good job with it. But builds tend to be a different creature. It goes something like this:

Get code
Magic configuration
Compile
More magic
Build done!

In many projects I've worked on, only the release engineering team could do a build. No, I don't mean could == allowed, but could == capable. Any even then, if the stars were not aligned just right and the magic words were not spoken with enough gravity, the build wouldn't finish.

I don't think this is very transparent ("now you see this code, now you don't, here's the build!"), and that worries me for an open source project. Aside from the principle, there are practical reasons why tricky builds are bad: developers, whether committers on the project or not, should be able to get a code snapshot from source control, make minimal configuration settings, and have a build. Now, this might take a while to run, and sometimes you can't help that. But you should at least have the option. If I am changing/experimenting with some core functionality, I'd like to verify how it all hangs together in the build. (Doing so in a dev environment is good too, but there are many errors that can creep in between dev and "binary" environments too. Ever forget to update your build.properties file to include new resources?)

The build system for DTP started out with the main goal of "run for anyone," but with time and pressure, cruft slipped in. Going forward, we'd like to migrate to the eclipse "base" builder, but would also like to re-capture the "run for anyone" goal if possible too.

Any thoughts/experience out there? Anyone want to help DTP with this (I think it would benefit us all....)?

Callisto: name, effects and other such things

2006-06-23T15:45:00.000-04:00

Mike recently blogged about Callisto culture, and I'd like to add some observations from DTP. As he said, no project was forced to join Callisto and, to be honest, when the first offer came to DTP, I paused, for many of the same reasons Mike describes. But DTP signed up as well, and we're glad to have done so. Being a relatively new project, Callisto offered us a chance to learn about delivering software successfully at eclipse.org. As you might imagine, just getting the code in place (with all that entails) is only one part of the puzzle. Working with the entire Callisto team, especially going through the endgame together, was a great learning experience. While everyone involved did a lot of work, Kevin Haaland (of platform fame) provided invaluable experience, and David Williams (WTP) was relentless in execution while managing the update process, despite the fact I clearly remember that he was not present when volunteered to do that job. :-)

And then there's the name "Callisto." Alex recently made some interesting observations about naming at eclipse.org. I'll now confess: I liked the original name "Eclipse release train," since I imagined being able to use a particular Ozzie song in my presentations when talking about it... Well, maybe later.

As Alex says, words tend to end up meaning something other than the original intention. I think in some cases this happens because people take the word to mean what they hope for. And I think this is the case in the semantic shift of "get 10 projects to build at the same time" to "get 10 projects' code integrated, remove duplication, consistent UI, etc. etc." The question then becomes: if (certain segments of) the community want the second type of integration, who will deliver it? My answer: the community should! (DTP committers are one part of this community, of course.) I often have conversation like this:

Someone: I'd like to work on DTP....
Me: Cool, what are you interested in? We have lot's of areas.....
Someone: But, you know, I don't have a lot of time to devote to it so, maybe later.

Speaking at least for DTP (and so it seems, based on studies, most other oss projects), being a committer isn't as time consuming as you might think. Sure, there is a core team that spends a lot of time on DTP each day, but there are also a lot of people who contribute occasionally. All of these contributions are valuable, and all of them are appreciated!

Callisto Webinar coming soon!

2006-05-26T16:31:00.000-04:00

An interesting webinar, showing emergent functionality in Callisto using DTP and BIRT will be presented on June 5th (Friday). You can read more about this webinar, and others for Callisto, on Ian's blog.

The Device Driver Problem

2006-05-26T15:16:00.000-04:00

DTP is facing a particular challenge, which I'll call the "device driver problem." Essentially, the problem is this: DTP provides a set of extensible, vendor-neutral frameworks for using data sources (defining driver templates, making connections, getting data and so on). By the very nature of frameworks, they act as enablers rather than tools for end user tasks. But, what happens if we don't have an extension to the frameworks for your particular case?

Let's take a specific example: you use (or are required to use) database O, and are looking for some tools in Eclipse to help in application development. DTP seems like the logical place, and you eagerly download it. But, wait: It doesn't work with database O! What a disappointment! The fact that the DTP frameworks work fine with another database doesn't help you here. Basically, DTP does not work as far as you are concerned. This is the same as when you install an operating system, only to find out that it doesn't have a device driver for your printer, modem, or other vital hardware.

And you are right. Yet the problem is more complicated: Although we want DTP to be used by the widest possible community (both extenders and end users), DTP has neither the staffing level nor experience to create and support all of the data sources that would be required by this wide community. So, the situation is frustrating from both sides: you want DTP to work completely with your data source, and we want to make that happen.

One possible solution is simply to say something like this: Look, this is open source software -- extend it yourself and make it available to others! That could work, but has a number of drawbacks: (1) If you are a new user of DTP, how do you easily find the required extensions for your data source, (2) How do you know particular extensions for data sources work in certain DTP versions, (3) If you are an extender, how can you work closely with the DTP team, aside from relying only on newsgroups and mailing lists?

We at DTP hear this concern and want to address it. But we need to do this in partnership with the community -- we simply don't have the staff and expertise to cover all necessary data sources. And hence, we have proposed the Enablement project as a new subproject of DTP.

Take a look at the Enablement proposal: let us know what you think. And, more importantly, come and get involved to make DTP everything that it should be. I am convinced that together we can make DTP a truly compelling part of the Eclipse ecosystem.

You want to do what with Eclipse?

2006-05-22T16:42:00.000-04:00

It's my birthday, and I'm giving myself the present of playing Devil's Advocate about Eclipse.

At eclipse.org we talk a lot about "extensible frameworks and exemplary tools." Frameworks are meant to be built on, and are not end-user tools by definition. Some projects even use this word in their name:

EMF: Eclipse Modeling Framework
GEF: Graphical Editor Framework

We also use the word "platform" a lot, especially in project names. Examples:

DTP: Data Tools Platform
WTP: Web Tools Platform
TPTP: Test & Performance Tools Platform
STP: SOA Tools Platform
DSDP: Device Software Development Platform

I'll take the notion of "platform" to mean something like what Michael Cusumano has written: "...an evolving system made of interdependent pieces that can each be innovated upon." ("Platform Leadership," p.2-3).

Yet we also find an emphasis on tools:

JDT: Java Development Tools
CDT: C/C++ Development Tools
BIRT: Business Intelligence and Reporting Tools

Those of us who work on "platform" or "framework" projects, however, often get requests and questions from end users, typically trying to use the example (exemplary) tools for a particular task. Now (and here's the Devil's Advocate): Why would you try to use a platform or framework to do a end user task? It's not a coincidence, I think, that the last three projects above have strong user communities and the earlier examples tend to have much strong extender communities. In the end, it is really is an expression of the committer base and those who "fund" the open source project by enabling committers to work on them. If platforms or frameworks are primary, then that's great for extenders, but not so good for end users.

Is this obvious? Even if so, if this the direction that eclipse.org should be moving? If you don't think so, then what can you do to motivate change?

Mixing Eclipse projects -- getting to where you want to be

2006-04-25T15:59:00.000-04:00

Foo Yung Chang commented on my earlier "Fractured Eclipse?" post with a perspective that I hadn't previously considered: "These users don't read newsgroups and post to Bugzilla; they just want to build their database-driven webapp without learning how to operate a nuclear reactor. These tasks reside at the intersection of WTP (75%), DTP (25%) and TPTP (25%)."

I think this calls out the need for cross-project use cases delineating the types of things that users want to do, and the part of eclipse.org projects they need to accomplish these tasks. Callisto is a first step in this area, simply trying to get everyone on the same page, platform-wise, and to see what happens from that. What sorts of cross-project uses cases are out there? Which ones should eclipse.org address, and which ones should be handled by the extender community?

What motivates an eclipse?

2006-04-25T15:32:00.000-04:00

Mike Milinkovich has an interesting response to my earlier "Fractured Eclipse" entry. He points out that the Eclipse Bylaws emphasize frameworks, and this is clear from the section that he quotes. He also notes that exemplary tools are crucial as well, as called out in the Eclipse Development Process Quality section. I think it is telling that two sources are used here. I don't disagree with Mike's comments at all, but I think the juxtaposition of these two sources is a very nice example of the multiple voices present in the Eclipse stakeholder community. Roughly, we have the extender and user groups. The extenders are more interested in the frameworks typically, and in fact might not want to see strong exemplary tools, since that is one area in which they'd like to make money. Users want to use Eclipse to perform some other task, such as developing enterprise or RCP applications. One looks inward to Eclipse and builds on it, and the other looks outward from Eclipse and builds with it. IMHO both are valid positions, but the framework centric one has tended to dominate more (notice that the framework comments come from the bylaws, whereas the tooling comments come from the development process guidelines).

So, as the Development Process Quality section says, "Neither frameworks without users nor tools without frameworks are interesting points along the software development spectrum," and this is certainly true. The challenge is to recognize the multiple stakeholder voices in the Eclipse community and strike a balance. How are we, as members of the Eclipse community, doing at this?

Fractured Eclipse?

2006-04-24T16:14:00.000-04:00

So, there's been a lot of discussion today about the level of integration among projects at eclipse.org. As the PMC chair for a top-level project at eclipse.org, I've certainly been aware of these sorts of issues, and have heard a number of discussions about them in various forums, ranging from the more formal (architecture council) to the casual (chance gatherings at conferences, user groups, etc.). I have to confess that I'm of two minds here.

On one side, the early history of Eclipse set a very high bar (too high?) with the JDT and PDE integration with the platform. Both of these technologies, especially JDT, are the result of concentrated, expert work by a strong committer base over a relatively long time (don't let the tip of the eclipse.org iceberg fool you here). Few, if any, other projects at eclipse.org have the benefit of this depth and length of experience (BIRT is perhaps close to being an exception here). Yes, there are a lot of really cool things going on at eclipse.org, but clearly, as the blogs and discussion mentioned above point out, there are a number of fractures too. I think the real question is: Which groups, and in which priority order, is eclipse.org trying to serve?

On the one hand there are ISV having the experience, staff, and willingness to pick up the pieces, and work the rough edges so they fit together nicely. Maybe add some additional pieces along the way. Nice way to build a product, leveraging open source, and there are a number of companies doing so in a big way.

On the other hand, there's the end users and companies wanting to take eclipse.org code as-is (or essentially as-is) and just use it. While groups in the first category can afford to work away the rough edges, groups in this category tend to cut themselves. And this is a very valid concern. I'd really like to see DTP be as full featured and easy to use for its domain as, for example, JDT is for Java.

The question you have to answer, and the position you have to get commitment to, is, however, what do the main players at eclipse.org want to spend their money on? I'm afraid that, if we start trying to solve the "user" problem before this foundational agreement is reached, then any efforts will be half-baked and only further serve to annoy exactly those we were supposed to be helping.

A visit to LinuxWorld

2006-04-07T17:15:00.000-04:00

Had a chance this week to help staff the eclipse.org booth at LinuxWorld. Yes, this was the first day, and the one with the now famous flaming booth. I'm ashamed to say this was my first LinuxWorld. While I use Linux on my personal machines, my open source involvement has been around Java and eclipse.org so far. It was really interesting to see the diversity of projects people are working on in Linux, and I was pleasantly surprised that so many Linux users also really like Eclipse (not just for CDT too).

Sustainable software development?

2006-04-02T11:39:00.000-04:00

I've been reading Sustainable Software Development by Kevin Tate. I've found it to be extremely interesting and right on target. I'm a little surprised that there hasn't been more discussion about this book in the community, given that unsustainable development is so common and debilitating.

EclipseCon, Day 2: Meeting with MySQL

2006-03-22T18:15:00.000-05:00

Had a really great meeting with Jay Pipes from MySQL. Looks like this is the beginning of a strong and productive collaboration between the MySQL and Eclipse DTP communities, and I'm really pleased about it! In DTP we've been watching the MySQL community, especially some of the recent tools, with great interest, and are looking forward to seeing how we can join forces to make DTP stronger and drop-dead easy to use with MySQL.

Stay tuned... I'm sure this is going to be very interesting!

EclipseCon, Day 2: Fun with demos.

2006-03-22T18:06:00.000-05:00

So, here I am in front of 50 or so people doing my "Data Tools for Rich Clients" presentation at EclipseCon. The sample app is pretty cool -- I think -- and it was just working (no, seriously, I swear it was just working!) in my hotel room 15 minutes before the presentation. It is an RCP application that reads the song list from an IPod....

Open the app, create the connection profile, click connect and... stack trace!
So, my first thought is "Fine, this is about building RCP apps with DTP, so I'll debug and fix this live, and it will be a good lesson." Then I looked at the stack trace, and realized that I didn't have a clue what had gone wrong. So, I did the natural thing: I tried it several times more, just to be sure that is was, in fact, really, truly broken...

Then someone from the audience asks if I have an internet connection, and it hit me: the SAX parser was trying to resolve off the box. Connected to the internet, and, finally, it worked! I don't know who gave me that suggestion, but -- seriously -- thanks!

Day 1, EclipseCon

2006-03-20T19:31:00.000-05:00

Day 1, EclipseCon. Sold out again, they say. It amazing how this conference just gets bigger and bigger each year.

Helped out on the DTP tutorial session this morning, and it was great. A small audience, but really engaged and a lot of great discussions and questions.

Thinking about Open Source

2006-03-06T10:36:00.000-05:00

I've been reading a lot of books/articles/research papers about open source recently, and (perhaps naturally) one frequent question is why developers participate in open source. The question is typically posed something like "What motivates developers to contribute time and code to open source?" or "Why do they give away their work for free?" Within the scope of such studies, a variety of motives and incentives is kept in the foreground.

But then, perhaps as a simplifying assumption, variation in motives and incentives is ignored when studying projects themselves, especially large (sets of) projects like Eclipse.org, Apache, or Linux. Let's ask the question: Why do companies contribute to Eclipse.org? And let's deal with the facts of variation and change of motivation and incentives over time. I think that would be very interesting to look at...

Camping out with Eclipse

2006-03-02T11:20:00.000-05:00

On Tuesday I helped staff the Raleigh, NC Eclipse Code Camp. It was an interesting chance to meet a wide variety of people who are using, or interested in using, Eclipse.

A few years ago when I went to these sorts of events, the type of people you'd meet and the questions they'd have were very predictable. What struck me most about this Code Camp is how much this is not true anymore. With the growth of the Eclipse ecosystem and widespread adoption, Eclipse is appearing in unexpected places.

Can't wait for EclipseCon this year -- it should be really interesting to see the diversity in this larger group...

A Year of DTP

2006-02-22T17:08:00.000-05:00

So, there I was, flipping bits at a local software company when the chance to propose the Eclipse Data Tools Platform (DTP) project appeared. We went live with the proposal on eclipse.org a year ago today... Yes, a year ago...

And a lot has happened during that year. I went to EclipseCon and met a lot of great people. The interest in DTP was amazing, and the result was a core group that would eventually form the DTP committer team. DTP progressed from proposal to top-level project at eclipse.org. We went to EclipseWorld. We wrote a number of articles and did various presentations. And we have even built some software along the line.

Now, I had worked on a variety of software projects, in a variety of roles. I've seen the typical range of outcomes, and I've had my share of blurried-eye, three AM coding marathons too. I've worked with small teams and large, and quite a bit in virtual teams as well.

But nothing quite prepares you for open source development, especially in a rich ecosystem like eclipse.org. Most people assume that being the chair of such a project is like being a engineering manager. Or maybe a project manager. But the truth is that (as someone, who I can't remember, told me early on) that running a top-level project at eclipse.con is more like running a start-up company. You get to do it all: you are HR, project management, product management, build management, businesse development... Yeah, the whole lot of it...

And it has been fascinating. As we close in on this year's EclipseCon, we expect to have a solid release for the community. We'll have a tutorial and two long talks at EclipseCon. We'll have a lot of informal meetings. DTP will be there in force.

Finally, we'll be talking to the community to learn about what DTP has done well in the past year, what we have done poorly and, most importantly, how we can grow and improve in the coming year. I look forward to it...

Administrative Thoughts

2006-01-30T16:10:00.000-05:00

It is important to realize that, though I work for Sybase: "The postings on this site are my own and don't necessarily represent Sybase's positions. strategies or opinions". Now, let's get on with it.... :-)