Having worked on open source projects for years, Doug Lea is convinced transparency is a far superior approach for the JCP. An active professor in the Computer Science Department, State University of New York at Oswego, Lea serves on the Executive Committee and leads JSR 166. When he submitted the Java Specification Request for 166, he was eager to use his expert group to illustrate precisely how much better the process could be. The result is that the JSR 166 Concurrency Utilities expert group worked in open view of the public and those who truly matter: future users and implementors of the spec.
 
After JCP 2.6 goes final early in 2004, the transparency showcased by JSR 166 will reflect the new order of the day. It won't be just a good idea; it'll be the new rule for all JSRs that are new or currently operating under JCP 2.5. But the reason it will be the rule is because specs created under it will be so much better.
 
A case in point, Lea's open source Application Programming Interface (API), dl.util.concurrent, had been in use for a few years by many systems and middleware developers, and he wanted to improve and standardize it. So he submitted the package as a JSR, gathered a team of experts and used the JCP program to improve it. Lea says, "The java.util.concurrent package is better than the package it evolved from by virtue of thousands of e-mail comments/discussions, design and code reviews, systematic tests, and so on. What we have are saner APIs, better specs, better tests, more user experience, more discussion, more code reviews, more design reviews than what we started with."
 
 
In selecting a team of experts, Lea made several deliberate choices. Because he wanted the expert group to have a role more substantial than merely advisory in producing the spec, RI, and TCK, Lea knew he must limit group size -- ten maximum -- to keep it manageable. Therefore, he made it a priority to invite only those who were committed to spending the time and effort. But Lea says his attempt didn't completely pan out, since one immediately dropped out and two others "have been lurkers who we forget even exist."
 
Networking is an essential tool for recruitment, and the best networking extends backwards over the years. Lea and Sun engineer Joshua Bloch have almost identical backgrounds and interests, but they didn't know each other well until they worked together on Java technology projects, which led to their involvement in the JCP. "Josh Bloch and I are among the principal co-conspirators for JDK 1.5 (JSR 176), and between us we're on almost all the components of the next platform release. And we're great friends, too. One of the nice things about being on these groups is you get to work with so many great people."
 

Tim Peierls

Tim Peierls, president of the three-person company Prior Artisans, LLC, was recruited on the recommendation of Bloch, his long-time professional acquaintance and a member of the JSR 166 expert group. In fact, Peierls had already been questioning Lea about his util.concurrent library. Almost all the other members were selected and tentatively agreed to serve before the JSR was officially proposed. David Holmes and Joe Bowbeer had made notable contributions in Java concurrency, and so were obvious choices. Brian Goetz had experience writing about and consulting on concurrent Java systems. A few other members were also selected from responses to the JCP invitation to participate.
 
When everything settled down, the expert group comprised six "very active smart people who have the ability and interest to work on specifications, who reflect a good range of interests and competence, and who all know each other at least a little bit. It's been absolutely great," says Lea. In terms of expertise, his goal was to strike "a balance between people expert in implementing the spec versus those expert in creating the kinds of software that would use the implementation once it existed." He made sure everyone agreed with the open-source development model and was experienced in small-group remote collaboration.
 
Besides corporate members Sun Microsystems and SAP AP, the remaining members were all individual members, further proof of the validity of JCP 2.5 changes. The company Bowbeer works for isn't a JCP member, but he personally joined anyway and has been a valued participant in multiple expert groups. Lea himself is one of the individual members hailing from the academic world for whom the JCP offers an opportunity to put theory into practice. He notes that some individuals and academics have vast expertise in particular niche areas and "are usually very happy to share it."
 
Peierls adds, "Individual members are more likely to act in the best interests of the language and the community of its users than are members beholden to an employer with product to move. For instance, if my company were a JCP member instead of just me, I would have felt a strong obligation to push for compatibility of JSR 166 with J2SDK 1.4, even though it wouldn't be in the best interest of most users."
 
 
Lea expected the experts to step in to do important parts of the work. Although he ended up creating practically all of the RI, and he coordinates, by default, nearly everything else, there has been quite a bit of informal role-taking, where people take charge of an API or issue for a while. He says, "We're small enough that we don't need to plan this out too much, and we're all experienced enough with small-group collaboration that we don't encounter logistics problems or personality clashes."
 
The expert group has never met altogether in the same place, but communicates through emails and phone calls, and uses a fairly standard infrastructure:

• An internal expert group-only mailing list is the main place where the team discusses issues, proposes solutions, deals with logistics, and so forth. The expert group found it helpful to include some guests on the list, such as Bill Pugh, spec lead for JSR133, who watches for issues relevant to his group. Message frequency ranges from zero in a slow month to thirty on a hectic day.
• An external interest list permits anyone to post questions, comments, and suggestions.
• The Concurrent Versions System (CVS) is a standard revision control system used to coordinate changes to the specs, RI, and TCK. Any expert group member can check in changes, although nontrivial changes are usually discussed before committing (http://www.cvshome.org/). Committed versions of source and test files can be made viewable from the web using viewcvs (http://viewcvs.sourceforge.net). Snapshots of entire builds can be replicated using rsync (http://samba.anu.edu.au/rsync).
• Mailman, the GNU mailing list manager, keeps good archives and is easy to maintain. (http://sourceforge.net/projects/mailman)
• JUnit is a popular framework that the group used to conduct TCK tests. (http://junit.org)
• Ant, a Java technology-based build tool, compiles code and generates proper javadocs. (http://ant.apache.org/)
• Anthill OS, a tool that makes use of both Ant and CVS to perform automated builds with email notification. (http://www.urbancode.com/projects/anthill/default.jsp).

The expert group appreciates the 300 subscribers of its external interest list. Lea says, "They have contributed a lot to JSR166 by asking questions that didn't have answers until we clarified or changed specs, complaints that showed that we missed important concerns, bug reports on prereleases that we could fix before J2SE release, and suggestions that showed that users sometimes wanted things different than we thought they would. Even when we can't directly address concerns, we've profited by finding out those places where we needed to be clearer about rationale."
 
The expert group keeps the two lists separate because it is "impossible" to make decisions among 300 list members, the experts need the freedom to think out loud about ideas and make mistakes in a protected environment, and it's nicer to shield non-experts from the tedium of mundane internal details such as deadlines. Lea, a consensus-driven spec lead, can't recall any instances where he made a decision over the serious reservations of any expert, although a few decisions ran against reservations of interest-list participants.
 
The CVS sources and spec are on the Internet, made transparent so people can see changes as the expert group commits to them. "Sometimes we 'uncommit' too, changing our minds. We were at first afraid that people might be confused about this, and think that the available javadocs, etcetera, were more final than they were, but we've never had anyone complain about this," says Lea. In another move toward transparency, the group also put out a preliminary RI release of main functionality, to let people test and react to it.
 

David Walend

David Walend is a software engineering leader at Alphatech and a graduate student at Tufts; he reads the interest list for work and to support his open source project SomnifugiJMS. After Lea introduced the list to him at JavaOne2002, Walend hoped to be a "kibitzer, but really I'm more of a pectator." He feels Concurrency Utilities is "one of the hardest topics to cover, and one of the best specifications. The problem attracts sharp people, and encourages them to park their egos at the door." Transparency helps illuminate the topic, and people on the list contribute by finding bugs and asking "tough questions about real-world problems." When Walend commented on JSRs of more traditional expert groups, he "never heard anything back. It was like shouting down a well," but his comments on JSR166 always elicited a "serious reply," Everyone's input counts, he says, and feedback is transparent to all, even when it is, "Good idea, but we're not doing that."
 

David Biesack

David Biesack's interest in continued improvement of the Java platform to increase "developer expressiveness, productivity, and program correctness" prompted him to join the external list. Biesack, R&D Java strategist of SAS Institute, Inc., is pleased that by monitoring the list, he mined nuggets to apply "right now" before the specs are even final. Since he wasn't a member of the expert group, he hesitated to comment much at first, but after having one of his method-naming suggestions incorporated, he feels more confident the expert group is "open to all ideas." He believes that because experts often lack the time to thoroughly review an RI, it's "critical that the RI be made available and as many as possible work with it in diverse domains and scenarios to validate it." Biesack appreciates that JSR 166 puts out a preliminary RI to encourage such validation.
 
A self-described external list "gadfly," John Mitchell laments he lacks time for deeper involvement, what with running his consulting company Non.net and writing books and articles (Java.net) on Java technology. Mitchell has been "pestering" Lea, Sun, and the rest of the Java industry on various fronts "forever," has employed Java technology since it was first released, and has also used Lea's original util.concurrent library. Mitchell can't say enough about how happy he is with the JSR 166 expert group's commitment to "as much transparency as they have been able to provide," and he's pleased that the JCP supported the group's openness. "We all benefit from the ideas of wider development community early in the process. The more open process has made this a much better result than any of the other JSRs that I know anything about, which is a fair number. It's clear from the relatively quick updates of various facets of the spec in response to both private and public comments that Doug and the rest of the expert group have taken community feedback to heart," responding with "real reasons backed up with code instead of big corporate BS."

Integrating Platform Specifications   Doug Lea reports that finding and working with Sun engineers is an important ingredient in the successful integration of platform specifications such as JSR 166 into Java 2 Standard Edition (J2SE). Because JSR 166 was part of the J2SE 1.5 platform release, the specifications, RI, and TCK had to be integrated into Sun's HotSpot release (which also serves as the J2SE 1.5 RI). This entailed interactions with Sun engineers throughout the process.   For example, Cliff Click (originally Sun server VM lead, now at Azul systems) and Dave Dice (Sun HotSpot runtime engineer) were guests on the internal mailing list and contributed advice and assistance integrating native support features. Sun J2SE engineer Martin Buchholz helped set up helpful mechanisms to coordinate commits from JSR 166 CVS into the full J2SE sources. Sun test engineer Andrey Chernyshev similarly coordinated integrating TCK tests as part of the Java Compatibility Kit (JCK). And expert member Josh Bloch helped represent JSR 166 interests in other release planning and delivery processes.

 
 
Lea believes the best way to learn more about spec leadership is to network, "to find a handful of other spec leads and call them up, talk to them. You find out a lot about whether you want to do something similar, all sorts of minor things that never get written about." That's practically an open invitation, and spec leads could learn a lot from this "acknowledged expert among experts in the field," says Peierls.
 
It doesn't hurt that Lea is perceived to have clout gained from his continuing service on the Executive Committee, but Mitchell feels Lea's practical understanding of the political situation within the JCP is even more valuable. In addition, Lea commands a certain respect just by virtue of his independence, free to follow his own professional instincts, with no corporate pressure. Biesack believes having a spec lead "who is committed to serving the wider Java community, not a personal or corporate agenda" is a substantial factor in the success of the expert group.
 
This factor, however, may have more to do with a spec lead's perceived attitude than with his or her actual chain of command. Walend appreciates Lea for "running the group because he thinks it's the coolest problem on the coolest platform."
 
Spec leads have another source of help in the JCP Program Management Office. In reflecting on his own interactions, Lea says, "The PMO folks truly are nice and helpful about logistics matters. The sooner the spec leads find that out, the better off they are." For example, Lea felt grateful that Kluyt spent some time wording the JSR 166 open source licensing terms and dealing with other people to help Lea succeed in pioneering the new practice.
 
That an "extremely" transparent process works very well in practice is clear from the outcome of JSR 166, and Lea looks forward to seeing current and future spec leads take similar approaches.
 
JSR 166 Home Page