Database Benchmarks – The Gift That Keeps on Giving

Yes, I know – not everyone believes database benchmarks are useful. My position is that there is value in benchmarks’ role in helping engineers wring out bottlenecks, bugs and performance impediments in their products. Berni Schiefer, Technical Executive , Information Management Performance and Benchmarks for DB2, MDM and SolidDB, recently told me that “every time we run [TPC-C] we are astonished at how effectively it hammers every element of the system. We always find bugs, room for tuning. It’s the nastiest, most punishing combination there is.” Read more of this post

Oracle’s TPC Assertions Don’t Help Its Credibility

Oracle has been making much of its recent benchmark results. Its new TPC campaign may backfire, however; its deceptive assertions do it no credit, and obscure some interesting technical advances (such as its first use of flash technology) behind mislabeling and deliberate omission of important facts. The “benchmark wars” are far less active than they were in their heyday, when new leapfrogging results occurred quarterly, or even more often. TPC-C, the transaction processing measure, has long been understood to be a poor representation of today’s real transaction types. At various times, most of the DBMS vendors have stopped issuing them – but they come back when they think they can get a headline or two. Some hardware vendors have also been dismissive of the benchmark; in fact, until this one, Sun had been a skeptic for a number of years. Read more of this post