Open Source “Purity,” Hadoop, and Market Realities

March 9, 2013 by Leave a comment

I don’t often do a pure opinion piece but I feel compelled to weigh in on a queston I’ve been asked several times since EMC released its Pivotal HD recently. The question is whether it is somehow inappropriate, even “evil,” for EMC to enter the market without having “enough” committers to open source Apache projects. More broadly, it’s about whether other people can use, incorporate, add to and profit from Apache Hadoop.

–more–

Filed under Gartner, Hadoop, Open source Tagged with , , , , , , ,

Hadoop 2013 – Part Four: Players

March 8, 2013 by Leave a comment

The first three posts in this series talked about performance,  projects and platforms as key themes in what is beginning to feel like a  watershed year for Hadoop. All three are reflected in the surprising emergence of a number of new players on the scene, as well as some new offerings from additional ones, which I’ll cover in another post. Intel, WANdisco, and Data Delivery Networks recently entered the distribution game, making it clear that capitalizing on potential differentiators (real or perceived)  in a hot market is still a powerful magnet. And in a space where much of the IP in the stack is open source, why not go for it? These introductions could all fall into the performance theme as well – they are all driven by innovations intended to improve Hadoop speed.

– more – 

Filed under Data Management, Gartner, Hadoop, Industry trends, Mapreduce, Software infrastructure Tagged with , , , , , , ,

Hadoop 2013 – Part Three: Platforms

February 23, 2013 by Leave a comment

In the first two posts in this series, I talked about performance and projects as key themes in Hadoop’s watershed year. As it moves squarely into the mainstream, organizations making their first move to experiment will have to make a choice of platform. And – arguably for the first time in the early mainstreaming of an information technology wave – that choice is about more than who made the box where the software will run, and the spinning metal platters the bits will be stored on.There are three options, and choosing among them will have dramatically different implications on the budget, on the available capabilities, and on the fortunes of some vendors seeking to carve out a place in the IT landscape with their offerings.

– more –

Filed under Data Management, Gartner, Hadoop, Hardware platforms, Industry trends, Mapreduce, Software infrastructure Tagged with , , , , , , , , , , ,

Hadoop 2013 – Part One: Performance

February 16, 2013 by Leave a comment

It’s no surprise that we’ve been treated to many year-end lists and predictions for Hadoop (and everything else IT) in 2013. I’ve never been that much of a fan of those exercises, but I’ve been asked so much lately that I’ve succumbed. Herewith, the first of a series of posts on what I see as the 4 Ps of Hsdoop in the year ahead: performance, projects, platforms and players.

– more –

Filed under Gartner, Hadoop, Industry trends, Mapreduce, Software infrastructure Tagged with , , , , , , , ,

Hadoop and DI – A Platform Is Not A Solution

February 10, 2013 by Leave a comment

“Hadoop people” and “RDBMS people” – including some DBAs who have contacted me recently –  clearly have different ideas about what Data Integration is. And both may  differ from what Ted Friedman and I were talking about in our Gartner research note Hadoop Is Not a Data Integration Solution , although I think the DBAs’ concept is far closer to ours.

- more -

Filed under Data Integration, Data Management, Gartner, Hadoop, Magic Quadrant Tagged with , , , ,

Diary of an Asian Swing: Day 4

November 7, 2012 by Leave a comment

Halfway across the world you go to breakfast and see a neighbor is in your hotel too. How often does it happen? Today I saw an SAP colleague I worked with two decades ago at Sybase – and his colleague, with whom I’ll meet while in Singapore. Great start to the day.

This day was all business. Met several Gartner clients to talk Big Data (since that was my billing.) Interest is high, and like North American firms, one of the key questions, as always, is Value. “What are people doing? What is proving useful from a business perspective?”

Gartner’s local office is beautiful – two floors in a thriving business neighborhood in one of the world’s most vibrant cities. I was told per capita income here is the second highest in the world, and the way the city is kept continues to impress: clean, efficient, beautifully designed and planted with fabulous flora everywhere. Our people here are professional, motivated, friendly and prepared for all our meetings, making sure I know who we’re meeting with and why.

It was a busy, stimulating day capped with dinner with my colleague Arun Chandrasekaran in the Pan Pacific Hotel’s restaurant. Multiple serving stations with different cuisines: Indian, Cantonese, Japanese…. that marvelous Singaporean polyglot cuisine I love. And if the food was good, the conversation was even better. Arun and I talked about how his infrastructure research and my software focus converged in big data and what our next collaboration should be after the Hadoop pilots piece we’re nearing completion on now.

Closing the day with a little BBC World in my room, I watched the pre-election coverage, amused by the overloading of the “battleground states” metaphor when I switched to CNN. They even referred to reporters “embedded” there. Please. Thank goodness this overpriced, overheated exercise will soon be complete. And after all the sound and fury, I don’t expect much will have changed.

Filed under Hadoop, Industry trends, Research Industry Tagged with , , , , , ,

Diary of an Asian Swing: Day 3

November 6, 2012 by Leave a comment

This was a day of transition. No meetings in Hong Kong, so after a leisurely breakfast and a look at the news, I settled down for a rare session of uninterrupted writing. It was still Sunday back home, so the email was relatively caught up and I could focus. Finished first drafts of some Gartner Magic Quadrant DW DBMS content and sent them off to colleagues for review and assembly into our eventual document.

This MQ is my second, and I’m really enjoying the process this time now that I’m not trying to figure out what happens next. I’m especially pleased with the process of combining interview data from customer interviews and analysis of our inquiry traffic – hundreds for each of the four authors – with surveys we conducted specifically for the report.

Mark Beyer built a fantastic link for feeding survey criteria measured by numeric scores from customers directly into relevant cells on our underlying spreadsheet. We had already done some collective scoring of our own in those cells, and the new exercise showed us how customers read the same issues. And it moved some of the scores significantly, with some vendors doing better than we expected in some areas, and others getting hammered. When a sizable number of survey respondents highlight an issue like support as a serious weakness, one has to take notice.

Several hours of uninterrupted time, a luxury that made the work move quickly, gave way to a decision about what to do with a free afternoon. I decided to use it for more work, so instead of an excursion I headed to the airport hours ahead of schedule to work in the attractive Cathay Pacific lounge. But I was surprised by a helpful check-in agent who told me there was an earlier flight I could get onto. As a result, I arrived in spectacular Singapore late in the evening instead of well into the night, and was in my hotel for a good night’s rest before early morning meetings the next day.

And of course, working on the place – even without wi-fi – was just as good as working in the lounge. So I had the chance to complete a new draft of a forthcoming Hadoop Pilot Best Practices piece and send it off to a collaborator. A good day indeed.

Filed under Data Management, Data Warehousing, DBMS, Hadoop, The Computing Life Tagged with , , , , , , ,

Diary of an Asian Swing: Day 1

November 3, 2012 by Leave a comment

I’ve never been a diarist. But as an exercise, I’m going to document this trip: two weeks on the road to Asia and Australia. Almost all work, though there is one day of weekend and recovery time built in.

Friday, Nov 2. Cathay Pacific flight to Hong Kong. Business class. Comfortable, well-appointed cabin. Friendly, courteous staff.

Learned system, table, storage, seat. NO WIFI! OMG. 14 unconnected hours – that’s what stimulated the idea to blog. Otherwise I would have been tweeting a lot. Which is fun, but this will be a change. Let’s start with some entertainment and settle in….

Video presentation of Sir Paul McCartney’s Kisses on the Bottom. Diana Krall. John Pizzarelli. Joe Walsh! It’s delightful – commentary, moments of studio play, tenderness. No real edge, but that’s not what it’s about. Luscious melody, beautiful jazz harmonies. Diana is a sideman here and a brilliant one. McCartney is who he is – always a touch too smooth and a touch too sweet, but it works so well here. Wonderful selection of songs and I love his own Valentine one.

Next up is a video piece about the Who’s Quadrophenia. Clearly part of Townshend’s current campaign to remind the world of his brilliant work – and well done. A good narrative sprinkled with revelatory bits about Moon, and Daltrey, and the terrible chaos they endured. Live performance footage and some nice stuff at the control board with Pete and the engineers highlighting dimensions of the music.

Now, it’s time to get to work – pick some music for background, get the computer out. Found Xuefei Yang – guitarist, playing Bach. Lovely tone, tempos, clarity. Don’t know her. Will fix that. And now to work – Magic Quadrant writing.

Wait. First a survey by Cathay Pacific. Why not?

A few Magic Quadrant hours reviewing interviews, surveys, briefing content, and our own scoring and it’s on to writing up a draft for one of the vendors and sending it off for my colleagues to comment and collaborate on the content.

Now, a break. I’ve earned one. Stretch the legs, a few exercises…

Treated to a reboot of the airplane’s computer system. Red Hat, it turns out. A parade of system level messages marched across the screen, unfathomable to anyone unfamiliar with Linux. The darkness, and eventually, a blessed progress bar, followed by the return of the flight map. We’ve passed over Siberia and the Gulf of Shelekhova; now we’re over the Sea of Okhotsk, only 6 and a half hours to go.

A moment to update this diary and a little reading – music choice Wynton Marsalis & Eric Clapton. Really? Didn’t know about this one. Would love to look it up on the net, but NO WIFI. Oh well, a little Time magazine, and then some Hadoop Operations by Eric Sammer.

Eric’s book is highly recommended – lucid, well written and doesn’t demand extraordinary technical depth to understand. Find it here: http://www.amazon.com/Hadoop-Operations-Eric-Sammer/dp/1449327052/ref=sr_1_1?s=books&ie=UTF8&qid=1351944059&sr=1-1&keywords=hadoop+operations 

Arrived on time. Hong Kong continues to impress: efficient, clean, modern as I remembered. Nice hotel. Good wifi. Sent off MQ writeups to colleagues and synchronized all the email I did while offline. Time to crash. Day One is done.

Filed under Data Management, Hadoop, Mapreduce Tagged with , , , ,

Guest Post: Leading the Logical Data Warehouse Charge Has its Challenges

February 24, 2012 by Leave a comment

From my colleague Mark Beyer, who speculates about how leadership in moving toward the logical data warehouse (LDW) will be received: 

The logical data warehouse is already creating a stir in the traditional data warehouse market space. Less than 5% of clients with implemented warehouses that we speak with are pursuing three or more of the six aspects of a logical warehouse: 

  • repositories
  • data virtualization
  • distributed processes
  • active auditing and optimization
  • service level negotiation
  • ontological and taxonomic metadata

That means we are in a very early stage regarding the adoption trend, and vendors who are aggressively moving toward it are ahead of their customers.

..more…

Filed under Data Management, Data Warehousing, Industry trends Tagged with , , , , ,

Apache Hadoop 1.0 Doesn’t Clear Up Trunks and Branches Questions. Do Distributions?

January 22, 2012 by Leave a comment

In early January 2012, the world of big data was treated to an interesting series of product releases, press announcements, and blog posts about Hadoop versions.  To begin with, we had the announcement of Apache version 1.0 at long last, in a press release. Although there were grumblings here and there in the twittersphere that changes to release numbers are meaningless, my discussions with Gartner’s enterprise customers indicate otherwise. Products with release numbers like 0.20.2 make the hair on Procurement’s neck stand on end, and as Hadoop begins to get mainstream attention (Gartner’s clients, see Hype Cycle for Data Management 2011), IT architects and executives find such optics quite important. Hadoop is moving beyond pioneers like Amazon, Yahoo! and LinkedIn into shops like JP Morgan Chase, and they pay attention to such things.

…more…

Filed under Data Management, Industry trends Tagged with , , , , , , , , , , , , , ,

← Older posts

Follow

Get every new post delivered to your Inbox.

Join 110 other followers