Symposium Notes – Day Four Returns to Data Security, and to Hadoop

Thursday, the final day, reinforced a theme for the week: data security is heating up, and organizations are not ready. It came up in half of today’s final 10 meetings.

“Is my data more secure, or less, in the cloud?”

“Does using open source software for data management compromise how well I can protect it?”

“I’m a public utility – can I put meter data in the cloud safely? What about if it is used to drive actions at the edge?”

“I’m using drones for mapping and the data is in the cloud – am I exposed?”


DBMS 2015 Numbers Paint a Picture of Slow but Steady Change

Gartner recently published “Market Share: All Software Markets, Worldwide, 2015” (for clients) and the story the DBMS data tells continues themes we have been observing for some time in the market. Overall, the DBMS space continued to grow in high single digits, coming in at $35.9 Billion in US dollars – an 8.7% growth over the prior year’s $33.1 Billion, which itself represented growth of 8.9% over 2013. The picture is changing, and though the effects are just beginning to be significant, they will grow substantially through this decade.


Strata Standards Stories: Different Stores For Different Chores

Has HDFS joined MapReduce in the emerging “legacy Hadoop project” category, continuing the swap-out of components that formerly answered the question “what is Hadoop?” Stores for data were certainly a focus at Strata/Hadoop World in NY, O’Reilly’s well-run, well-attended, and always impactful fall event. The limitations of HDFS, including its append-only nature, have become inconvenient enough to push the community to “invent” something DBMS vendors like Oracle did decades ago: a bypass. After some pre-event leaks about its arrival, Cloudera chose its Strata keynote to announce Kudu, a new columnstore written in C++, bypassing HDFS entirely. Kudu will use an Apache license and will be submitted to the Apache process at some undetermined future time.


Perspectives on Hadoop Part Two: Pausing Plans

By Merv Adrian and Nick Heudecker 

In the first post in this series , I looked at the size of revenue streams for RDBMS software and maintenance/support and noted that they amount to $33B, pointing out that pure play Hadoop vendors had a high hill to climb. (I didn’t say so specifically, but in 2014, Gartner estimates that the three leading vendors generated less than $150M.)

In this post, Nick and I turn from Procurement to Plans and examine the buying intentions uncovered in Gartner surveys.


–more in Gartner blog–

Perspectives on Hadoop: Procurement, Plans, and Positioning

I have the privilege of working for the world’s leading information technology research and advisory company, covering information management with a strong focus for the past few years on an emerging software stack called Hadoop. In the early part of 2015, that particular technology is moving from early adopter status to early majority in its marketplace adoption. The discussions and published work around it have been exciting and controversial, so in this post (and a couple to follow) I describe three interlocking research perspectives on Hadoop: procurement (counting real money actually spent); plans (surveys of intentions to invest) and positioning (subjective interpretations of what the first two mean.)

Procurement Perspective: Hadoop is a (Very) Small Market Today

–more on Gartner blog–



Which SQL on Hadoop? Poll Still Says “Whatever” But DBMS Providers Gain

Since Nick Heudecker and I began our quarterly Hadoop webinars, we have asked our audiences what they expected to do about SQL several times, first in January 2014. With 164 respondents in that survey, 32% said “we’ll use what our existing BI tool provider gives us,” reflecting the fact that most adopters seem not to want to concern themselves overmuch with the details.

–More on my Gartner blog

Prediction Is Hard – Especially About the Future

OK, I admit it – I stole the title from a much smarter man. I thought that man was Yogi Berra, but maybe not – more about that at the end of this post.

Every year, Gartner issues a series of Predicts documents. This year I had the pleasure of doing one for my team on Information Infrastructure Technology. Now, I’m a software guy, and the team I’m on is all software people, so a document assigned to our team would typically be about – well, information software technology. But that would have missed the point rather dramatically, so I connected with a few colleagues and got their OK to use some of their predictions in the small set any document can include.

— more on Gartner blog —

DBMS Legacies are Very Sticky

Donald Feinberg (@Brazingo) & Merv Adrian (@merv)

Every so often, there’s a wave of interest in the “imminent retirement” of one or more legacy database management systems (DBMS). Usually, it’s because someone with very little knowledge of the actual use and distribution of the products becomes enthusiastic about someone’s sales pitch, or an anecdote or two. Sometimes it’s the result of a “replacement” marketing campaign by a competitor. And so far, it’s usually as illusive- and as far off – as the “death of the mainframe”.

Recently, a financial analyst report stated that in 2015, the industry would begin retiring Sybase products (owned now by SAP) and Informix (owned now by IBM). We and our colleagues have since had several inquiries about this and our response is simple: poppycock. DBMS market data, and our thousands of interactions with customers, do not support any of this.

—more on my Gartner blog–

Hadoop Investments Continue: Teradata, HP Jockey For Position

Interest from the leading players continues to drive investment in the Hadoop marketplace. This week Teradata made two acquisitions – Revelytix and Hadapt – that enrich its already sophisticated big data portfolio, while HP made a $50M investment in, and joined the board of, Hortonworks. These moves continue the ongoing effort by leading players. 4 of the top 5 DBMS players (Oracle, Microsoft, IBM, SAP and Teradata) and 3 of the top 7 IT companies (Samsung, Apple, Foxconn, HP, IBM, Hitachi, Microsoft) have now made direct moves into the Hadoop space. Oracle’s recent Big Data Appliance and Big Data SQL, and Microsoft’s HDInsight represent substantial moves to target Hadoop opportunities, and these Teradata and HP moves mean they don’t want to be left behind.



Aspirational Marketing and Enterprise Data Hubs

In the Hadoop community there is a great deal of talk of late about its positioning as an Enterprise Data Hub. My description of this is “aspirational marketing;” it addresses the ambition its advocates have for how Hadoop will be used, when it realizes the vision of capabilities currently in early development. There’s nothing wrong with this, but it does need to be kept in perspective. It’s a long way off.