Hadoop 2013 – Part Four: Players

March 8, 2013 by Leave a comment

The first three posts in this series talked about performance,  projects and platforms as key themes in what is beginning to feel like a  watershed year for Hadoop. All three are reflected in the surprising emergence of a number of new players on the scene, as well as some new offerings from additional ones, which I’ll cover in another post. Intel, WANdisco, and Data Delivery Networks recently entered the distribution game, making it clear that capitalizing on potential differentiators (real or perceived)  in a hot market is still a powerful magnet. And in a space where much of the IP in the stack is open source, why not go for it? These introductions could all fall into the performance theme as well – they are all driven by innovations intended to improve Hadoop speed.

– more – 

Filed under Data Management, Gartner, Hadoop, Industry trends, Mapreduce, Software infrastructure Tagged with , , , , , , ,

Hadoop 2013 – Part Three: Platforms

February 23, 2013 by Leave a comment

In the first two posts in this series, I talked about performance and projects as key themes in Hadoop’s watershed year. As it moves squarely into the mainstream, organizations making their first move to experiment will have to make a choice of platform. And – arguably for the first time in the early mainstreaming of an information technology wave – that choice is about more than who made the box where the software will run, and the spinning metal platters the bits will be stored on.There are three options, and choosing among them will have dramatically different implications on the budget, on the available capabilities, and on the fortunes of some vendors seeking to carve out a place in the IT landscape with their offerings.

– more –

Filed under Data Management, Gartner, Hadoop, Hardware platforms, Industry trends, Mapreduce, Software infrastructure Tagged with , , , , , , , , , , ,

Hadoop 2013 – Part Two: Projects

February 20, 2013 by Leave a comment

In Part One of this series, I pointed out that how significant attention is being lavished on performance in 2013. In this installment, the topic is projects, which are proliferating precipitously. One of my most frequent client inquiries is “which of these pieces make Hadoop?” As recently as a year ago, the question was pretty simple for most people: MapReduce, HDFS, maybe Sqoop and even Flume, Hive, Pig, HBase, Lucene/Solr, Oozie, Zookeeper. When I published the Gartner piece How to Choose the Right Apache Hadoop Distribution, that was pretty much it.

–more–

Filed under Gartner, Hadoop, Industry trends, Mapreduce, Software infrastructure

Hadoop 2013 – Part One: Performance

February 16, 2013 by Leave a comment

It’s no surprise that we’ve been treated to many year-end lists and predictions for Hadoop (and everything else IT) in 2013. I’ve never been that much of a fan of those exercises, but I’ve been asked so much lately that I’ve succumbed. Herewith, the first of a series of posts on what I see as the 4 Ps of Hsdoop in the year ahead: performance, projects, platforms and players.

– more –

Filed under Gartner, Hadoop, Industry trends, Mapreduce, Software infrastructure Tagged with , , , , , , , ,

Stack Up Hadoop to Find Its Place in Your Architecture

January 30, 2013 by Leave a comment

2013 promises to be a banner year for Apache Hadoop, platform providers, related technologies – and analysts who try to sort it out. I’ve been wrestling with ways to make sense of it for Gartner clients bewildered by a new set of choices, and for them and myself, I’ve built a stack diagram that describes the functional layers of a Hadoop-based model.

– more –

Filed under Data Management, Hadoop, Mapreduce, Software infrastructure Tagged with , ,

Amazon Redshift Disrupts DW Economics – But Nothing Comes Without Costs

December 7, 2012 by Leave a comment

At its first re:Invent conference in Late November, Amazon announced Redshift, a new managed service for data warehousing. Amazon also offered details and customer examples that made AWS’  steady inroads toward enterprise, mainstream application acceptance very visible.

Redshift is made available via MPP nodes of 2TB (XL) or 16TB (8XL), running Paraccel’s high-performance columnar, compressed DBMS, scaling to 100 8XL nodes, or 1.6PB of compressed data. XL nodes have 2 virtual cores, with 15GB of memory, while 8XL nodes have 16 virtual cores and 120 GB of memory and operate on 10Gigabit ethernet.

Reserved pricing (the more likely scenario, involving a commitment of 1 year or 3 years) is set at “under $1000 per TB per year” for a 3 year commitment, combining upfront and hourly charges. Continuous, automated backup for up to 100% of the provisioned storage is free. Amazon does not charge for data transfer into or out of the data clusters. Network connections, of course, are not free  - see Doug Henschen’s Information Week story for details.

This is a dramatic thrust in pricing, but it does not come without giving up some things.

More…

Filed under Data Warehousing, DBMS, Hadoop, Mapreduce, On Business Intelligence, Software Licensing, Vendor image and communications Tagged with , , , , , , , ,

Diary of an Asian Swing: Day 1

November 3, 2012 by Leave a comment

I’ve never been a diarist. But as an exercise, I’m going to document this trip: two weeks on the road to Asia and Australia. Almost all work, though there is one day of weekend and recovery time built in.

Friday, Nov 2. Cathay Pacific flight to Hong Kong. Business class. Comfortable, well-appointed cabin. Friendly, courteous staff.

Learned system, table, storage, seat. NO WIFI! OMG. 14 unconnected hours – that’s what stimulated the idea to blog. Otherwise I would have been tweeting a lot. Which is fun, but this will be a change. Let’s start with some entertainment and settle in….

Video presentation of Sir Paul McCartney’s Kisses on the Bottom. Diana Krall. John Pizzarelli. Joe Walsh! It’s delightful – commentary, moments of studio play, tenderness. No real edge, but that’s not what it’s about. Luscious melody, beautiful jazz harmonies. Diana is a sideman here and a brilliant one. McCartney is who he is – always a touch too smooth and a touch too sweet, but it works so well here. Wonderful selection of songs and I love his own Valentine one.

Next up is a video piece about the Who’s Quadrophenia. Clearly part of Townshend’s current campaign to remind the world of his brilliant work – and well done. A good narrative sprinkled with revelatory bits about Moon, and Daltrey, and the terrible chaos they endured. Live performance footage and some nice stuff at the control board with Pete and the engineers highlighting dimensions of the music.

Now, it’s time to get to work – pick some music for background, get the computer out. Found Xuefei Yang – guitarist, playing Bach. Lovely tone, tempos, clarity. Don’t know her. Will fix that. And now to work – Magic Quadrant writing.

Wait. First a survey by Cathay Pacific. Why not?

A few Magic Quadrant hours reviewing interviews, surveys, briefing content, and our own scoring and it’s on to writing up a draft for one of the vendors and sending it off for my colleagues to comment and collaborate on the content.

Now, a break. I’ve earned one. Stretch the legs, a few exercises…

Treated to a reboot of the airplane’s computer system. Red Hat, it turns out. A parade of system level messages marched across the screen, unfathomable to anyone unfamiliar with Linux. The darkness, and eventually, a blessed progress bar, followed by the return of the flight map. We’ve passed over Siberia and the Gulf of Shelekhova; now we’re over the Sea of Okhotsk, only 6 and a half hours to go.

A moment to update this diary and a little reading – music choice Wynton Marsalis & Eric Clapton. Really? Didn’t know about this one. Would love to look it up on the net, but NO WIFI. Oh well, a little Time magazine, and then some Hadoop Operations by Eric Sammer.

Eric’s book is highly recommended – lucid, well written and doesn’t demand extraordinary technical depth to understand. Find it here: http://www.amazon.com/Hadoop-Operations-Eric-Sammer/dp/1449327052/ref=sr_1_1?s=books&ie=UTF8&qid=1351944059&sr=1-1&keywords=hadoop+operations 

Arrived on time. Hong Kong continues to impress: efficient, clean, modern as I remembered. Nice hotel. Good wifi. Sent off MQ writeups to colleagues and synchronized all the email I did while offline. Time to crash. Day One is done.

Filed under Data Management, Hadoop, Mapreduce Tagged with , , , ,

Follow

Get every new post delivered to your Inbox.

Join 110 other followers