Kickfire Disrupts DW Economics, Targets Mainstream ADBMS Opportunities

In just 18 months, Kickfire has established itself as one of the most intriguing of the ADBMS insurgents. It espouses a radical go-to-market strategy: target the overwhelming majority of the market in the sub-5Tb space, and let others battle over who’s doing best at the top end, fighting over a small group of prospects. Kickfire also takes a radically different architectural approach: it uses an “SQL chip” to run much of its work in hardware, to dramatic effect in performance.

In April 2008, the Kickfire data warehouse appliance was announced at a MySQL conference, and simultaneously the company released 100Gb and 300Gb TPC-H benchmarks  that transformed price-performance expectations at the low end of the market. 6 months later the appliance became generally available, and 6 months after that had its first production reference. Since then, the company has had two encouraging quarters, and the product is now in the hands of some two dozen early adopters, a half dozen of whom are referenceable production sites. I spent some time recently with Kickfire CEO Bruce Armstrong to discuss the story so far, and Kickfire’s recent announcement of Kickfire 1.5 and the 3000 series appliance.

Armstrong is clear about the company’s focus:

90% of the market is in deployments of 10 Tb and less. It’s a $5B market; we’d be happy to take a piece with a low-cost alternative while other players fight for the biggest deals they can find.”

Kickfire quotes IDC’s Dan Vesset to affirm that two-thirds of all data warehouses are in the sub-5 TB size range.  In that market, MySQL, which is a key component of Kickfire’s architecture, is the 3rd most widely deployed database, and surveys indicate that 25% of its 12 million licensees are using it for data warehousing and BI applications.  Armstrong is comfortable with his firm’s multi-year contract with MySQL and is looking forward to Kickfire’s participation in the upcoming Oracle Open World as another opportunity to reach that huge community.

Kickfire has also allied itself with several key open source leaders to craft an end-to-end value proposition, key at the low end of the market. The appliance is greatly enhanced by including 3 key components of a data warehousing strategy:  warehousing engine (Kickfire); ETL capability (Talend, which also offers significant MDM functionality) and BI (Jaspersoft). Other elements are either also open source or very widely commoditized: Intel hardware, CentOS (one of the more interesting Linux distributions), Pentaho (for another BI alternative), and  zmanda for backup and recovery. This makes for a strong, fairly complete story, and so far the response seems to have been steady. I’m hoping to talk to some clients about the end-to-end experience soon.

Kickfire’s technical architecture addresses two bottlenecks in processing: the CPU’s handling of instructions generated from SQL and the I/O challenges of large volume retrieval from storage. The latter is familiar to ADBMS followers, and is handled in a fashion similar to that of other emerging vendors: with a columnar database that only sends back the columns needed to resolve a query. Where Kickfire differs from all of its competitors is in its proprietary SQL processing chip. Seem unprecedented? It’s not.  Your Cisco or Juniper router uses processors that handle networking primitives, your PC has a graphics chip that does the same thing for polygons, surface mapping and the like. Why not handle SQL  in a similar way, offloading some key processing from the CPU? That’s exactly what Kickfire’s chip does, and it operates on compressed data, in memory, without registers through interesting memory pipelining strategies with on-board memory next to the chip. Some gory details: [skip this if you don’t want to get into the hardware side):.

What’s an FPGA? Briefly, a chip that can be designed to implement desired logical functions in hardware. The includes database operations like SELECT and JOIN. The benefits? Speed and lower power consumption – you can run at lower clock speeds, saving power. As Daniel Abadi’s blog will tell you ( read it often – it’s always worth the effort,) “operations that take hundreds to thousands of CPU instructions can be performed in a single clock cycle in FPGA logic.”  Hundreds of these operations can execute in parallel. There’s more work in the surrounding memory architecture; you can read about it in Daniel’s blog and in a great discussion thread on Curt Monash’s blog post.   As a result of all this work, the system can keep intermediate data sets in the chip’s onboard storage, eliminating a huge amount of overhead.

MySQL’s role in all this is integrated ingeniously: MySQL parses incoming SQL, and then Kickfire’s optimizer generates an execution plan that uses either or both its field-programmable gate array (FPGA) SQL chip and its software SQL execution engine. The majority of queries run natively in hardware or with just a small component in software, and with the new product release, Kickfire claims that 95% is running there, including a good chunk of DML Kickfire claims that for the kind of processing it targets, the chip is equivalent of 30 CPUs.

The resulting device delivers an order of magnitude saving. That’s right: one-tenth the storage. One-tenth the cost (per the TPC benchmark data) of the Oracle/HP configuration, one-fifth of the Microsoft/IBM one. One-tenth the power– and space. When you add it all up over three years it’s  a half-million dollars worth of savings in the TPC configurations compared to what Kickfire used. That’s a powerful value proposition for the price-sensitive low end of the market, and departments in large enterprises as well, where the simple installation of a true appliance completes the pitch very strongly. The 2000 series starts at just $32K for a full appliance, with low rack costs (2-3RU). The 2^nd unit is the SQL chip coprocessor. Yes, this a full appliance play. There’s a great deal of integration and pre-installation, although not as much as there will be as the company gains more experience. Already, it includes a high-speed bulk loader and an incremental loader, which does micro-batch updating.

The new  3000 Series release, starting at $154K, is targeted for mid-range data warehouses, with more on-board storage as well as external RAID support. It ships with up to 14.4 TB of disk, to assure headroom for customers who begin to crave more data. It is multi-node enabled, and it can deploy on an HA environment. Kickfire asserts that it can comfortably handle a “query while loading” workload. A faster and easier Migration Wizard  allows customers to simply drag and drop objects onto the Kickfire box.

Like its ADBMS brethren, Kickfire has a small but growing set of happy customers raving about its performance. Like other emerging firms I’ve discussed here, Kickfire needed to address funding early, and it secured a $20M Series B funding from some of the industry’s more astute and succesful investors, Greylock, Accel and Mayfield Fund among them. The management team has used its resources well – SI and software partnerships, marketing events of various kinds, influencer outreach, and a new product launch are evidence of strong management that’s covering all the bases. Kickfire is aiming to change the game. Let’s overload the metaphor counter: sometimes you skate to where the puck is going to be, and sometimes you go after the low hanging fruit. Kickfire is doing both.