The open source community is remarkable in many ways. For me, one of the most significant aspects of it is exactly that: it IS a community. It’s composed of people who communicate and share in deep and productive ways. One of the most interesting manifestations of that spirit I’ve run across is the Eigenbase project, an extensible platform being used by some very creative folks for the creation and continuing development of databases for data warehousing (the LucidDB DBMS) and stream processing (the SQLstream continuous query engine). I haven’t posted about either of those yet but will, and I’m watching their continuing evolution with great interest.
Organizationally, Eigenbase is a 501(c)(3) tax-exempt public charity. It has had contributions of various kinds – servers, IP, etc. from numerous companies and individuals since its founding in 2004. John Sichi, president and chairman, worked for Quadbase Systems and Broadbase Software, and was the original developer of Fennel and Farrago, two Eigenbase framework sub-projects. John contributes to a number of other open source projects inside and outside of Eigenbase, including JGraphT and LucidDB. Julian Hyde, treasurer, also sports a database pedigree from Oracle, Broadbase and Pentaho; he founded Mondrian and other projects. Julian is Chief Architect for SQLstream and Chief Architect, OLAP for Pentaho. He and John were together at Broadbase when John came up with some of the key Eigenbase constructs. Other key players include director Damian Black, whose career includes stints at Hewlett-Packard Research Laboratories in Europe where he worked on combining logic programming, object-oriented and relational database systems. Damian also took on business roles, which helped prep him for his role today as CEO of SQLstream.
The notion that “one size fits all” DBMSs are an artifact of an old model, and that it’s time to move on, is one that has gained a great deal of academic momentum, and the ADBMS vendors are aggressively pitching the idea that they have identified at least one use case that proves it. Mike Stonebraker has a few other ones in his thinking, and is commercially involved with two in addition to Vertica. But Eigenbase takes a rather different tack: it’s factored the data management tasks in such a way that different use cases can be supported by grafting different “features” onto it, as illustrated in this figure.
The premise is that as ideas for specialized use cases emerge, the developer can construct the extensions desired. In the case of LucidDB, it’s the column store bits (pun intended) that support the ADBMS case; in SQLstream, it’s SQL-based support for event processing. The platform includes a methodology for the assembly of these innovations, and a community of programmers work together to advance the model.
Platform capabilities include a source code repository, a release management process that uses Sourceforge, a wiki for documentation, a defect tracking system, mailing lists, standards enforcement and a great deal more. These things are not without precedent, but they form a coherent whole that attracts some very talented people. “The factoring is why Eigenbase is a vibrant community,” John told me recently over a breakfast in Half Moon Bay. “Lucid DB is beginning to get to a wider audience: people are kicking the tires,” Julian added. Lucid continues despite the closure of Lucidera, and development is now moving from core stuff to the integration side.
Quite a crew. There are others, whose brief bios on the Eigenbase site read like a who’s who, but these are the folks I’ve had the pleasure of meeting in several settings. Their commitment – to quality, and to the open source model – is extraordinary. If you’re interested in these technology categories, open source and/or collaborative processes you should get to know them. Follow the links and say hello. As they continue their work, don’t be surprised if we see some commercialization, evangelism as the dozens of implementations gain their own footholds, and the emergence of yet another contender.