Merv Adrian's IT Market Strategy

Open Source “Purity,” Hadoop, and Market Realities

Merv Adrian — Sat, 09 Mar 2013 23:43:39 +0000

I don’t often do a pure opinion piece but I feel compelled to weigh in on a queston I’ve been asked several times since EMC released its Pivotal HD recently. The question is whether it is somehow inappropriate, even “evil,” for EMC to enter the market without having “enough” committers to open source Apache projects. More broadly, it’s about whether other people can use, incorporate, add to and profit from Apache Hadoop.

–more–

Hadoop 2013 – Part Four: Players

Merv Adrian — Fri, 08 Mar 2013 20:20:59 +0000

The first three posts in this series talked about performance, projects and platforms as key themes in what is beginning to feel like a watershed year for Hadoop. All three are reflected in the surprising emergence of a number of new players on the scene, as well as some new offerings from additional ones, which I’ll cover in another post. Intel, WANdisco, and Data Delivery Networks recently entered the distribution game, making it clear that capitalizing on potential differentiators (real or perceived) in a hot market is still a powerful magnet. And in a space where much of the IP in the stack is open source, why not go for it? These introductions could all fall into the performance theme as well – they are all driven by innovations intended to improve Hadoop speed.

– more –

Hadoop 2013 – Part Three: Platforms

Merv Adrian — Sat, 23 Feb 2013 22:52:14 +0000

In the first two posts in this series, I talked about performance and projects as key themes in Hadoop’s watershed year. As it moves squarely into the mainstream, organizations making their first move to experiment will have to make a choice of platform. And – arguably for the first time in the early mainstreaming of an information technology wave – that choice is about more than who made the box where the software will run, and the spinning metal platters the bits will be stored on.There are three options, and choosing among them will have dramatically different implications on the budget, on the available capabilities, and on the fortunes of some vendors seeking to carve out a place in the IT landscape with their offerings.

– more –

Hadoop 2013 – Part Two: Projects

Merv Adrian — Thu, 21 Feb 2013 03:16:56 +0000

In Part One of this series, I pointed out that how significant attention is being lavished on performance in 2013. In this installment, the topic is projects, which are proliferating precipitously. One of my most frequent client inquiries is “which of these pieces make Hadoop?” As recently as a year ago, the question was pretty simple for most people: MapReduce, HDFS, maybe Sqoop and even Flume, Hive, Pig, HBase, Lucene/Solr, Oozie, Zookeeper. When I published the Gartner piece How to Choose the Right Apache Hadoop Distribution, that was pretty much it.

–more–

Hadoop 2013 – Part One: Performance

Merv Adrian — Sat, 16 Feb 2013 17:27:30 +0000

It’s no surprise that we’ve been treated to many year-end lists and predictions for Hadoop (and everything else IT) in 2013. I’ve never been that much of a fan of those exercises, but I’ve been asked so much lately that I’ve succumbed. Herewith, the first of a series of posts on what I see as the 4 Ps of Hsdoop in the year ahead: performance, projects, platforms and players.

– more –

Hadoop and DI – A Platform Is Not A Solution

Merv Adrian — Sun, 10 Feb 2013 17:16:10 +0000

“Hadoop people” and “RDBMS people” – including some DBAs who have contacted me recently – clearly have different ideas about what Data Integration is. And both may differ from what Ted Friedman and I were talking about in our Gartner research note Hadoop Is Not a Data Integration Solution , although I think the DBAs’ concept is far closer to ours.

- more -

Stack Up Hadoop to Find Its Place in Your Architecture

Merv Adrian — Wed, 30 Jan 2013 19:14:55 +0000

2013 promises to be a banner year for Apache Hadoop, platform providers, related technologies – and analysts who try to sort it out. I’ve been wrestling with ways to make sense of it for Gartner clients bewildered by a new set of choices, and for them and myself, I’ve built a stack diagram that describes the functional layers of a Hadoop-based model.

– more –

2013 Data Resolution: Avoid Architectural Cul-de-Sacs

Merv Adrian — Thu, 27 Dec 2012 18:35:17 +0000

I had an inquiry today from a client using packaged software for a business system that is built on a proprietary, non-relational datastore (in this case an object-oriented DBMS.) They have an older version of the product – having “failed” with a recent upgrade attempt.

The client contacted me to ask about ways to integrate this OODBMS-based system with others in their environment. They said the vendor-provided utilities were not very good and hard to use, and the vendor has not given them any confidence it will improve. The few staff programmers who have learned enough internals have already built a number of one-off connections using multiple methods, and were looking for a more generalizable way to create a layer for other systems to use when they need data from the underlying database. They expect more such requests, and foresee chaos, challenges hiring and retaining people with the right skills, and cycles of increasing cost and operational complexity.

My reply: “you’re absolutely right.”

—more–

Amazon Redshift Disrupts DW Economics – But Nothing Comes Without Costs

Merv Adrian — Sat, 08 Dec 2012 00:23:04 +0000

At its first re:Invent conference in Late November, Amazon announced Redshift, a new managed service for data warehousing. Amazon also offered details and customer examples that made AWS’ steady inroads toward enterprise, mainstream application acceptance very visible.

Redshift is made available via MPP nodes of 2TB (XL) or 16TB (8XL), running Paraccel’s high-performance columnar, compressed DBMS, scaling to 100 8XL nodes, or 1.6PB of compressed data. XL nodes have 2 virtual cores, with 15GB of memory, while 8XL nodes have 16 virtual cores and 120 GB of memory and operate on 10Gigabit ethernet.

Reserved pricing (the more likely scenario, involving a commitment of 1 year or 3 years) is set at “under $1000 per TB per year” for a 3 year commitment, combining upfront and hourly charges. Continuous, automated backup for up to 100% of the provisioned storage is free. Amazon does not charge for data transfer into or out of the data clusters. Network connections, of course, are not free - see Doug Henschen’s Information Week story for details.

This is a dramatic thrust in pricing, but it does not come without giving up some things.

More…

Diary of an Asian Swing: Day 5

Merv Adrian — Thu, 08 Nov 2012 13:56:48 +0000

Second day of Singapore meetings. APJ market conversations with IT vendors, watching the emergence of big data here. Seeing activity in the field is always a fascinating counterpoint to the briefings and conferences back home. But the big data phenomenon is surprisingly rapid. Certainly the user conversations have been similar in some ways to those I have in North America, but the players and the details have not made their way here. Yet, we met with a government-funded think tank, building IP for an aggressive thrust into new business opportunities. Different from Silicon Valley think tanks, but no less intriguing – or aggressive.

Capper to the day was dinner with two colleagues in a neighborhood crab restaurant. I have enough opportunities for fine cuisine, but this was a chance to let our hair down, eat like normal people and talk about the day. And Singapore cuisine is unique and marvelous. A wonderful evening and back to the hotel. Next day: Kuala Lumpur.