Open Source “Purity,” Hadoop, and Market Realities

I don’t often do a pure opinion piece but I feel compelled to weigh in on a queston I’ve been asked several times since EMC released its Pivotal HD recently. The question is whether it is somehow inappropriate, even “evil,” for EMC to enter the market without having “enough” committers to open source Apache projects. More broadly, it’s about whether other people can use, incorporate, add to and profit from Apache Hadoop.

–more–

Hadoop 2013 – Part Four: Players

The first three posts in this series talked about performance projects and platforms as key themes in what is beginning to feel like a  watershed year for Hadoop. All three are reflected in the surprising emergence of a number of new players on the scene, as well as some new offerings from additional ones, which I’ll cover in another post. Intel, WANdisco, and Data Delivery Networks recently entered the distribution game, making it clear that capitalizing on potential differentiators (real or perceived)  in a hot market is still a powerful magnet. And in a space where much of the IP in the stack is open source, why not go for it? These introductions could all fall into the performance theme as well – they are all driven by innovations intended to improve Hadoop speed.

– more – 

Hadoop 2013 – Part One: Performance

It’s no surprise that we’ve been treated to many year-end lists and predictions for Hadoop (and everything else IT) in 2013. I’ve never been that much of a fan of those exercises, but I’ve been asked so much lately that I’ve succumbed. Herewith, the first of a series of posts on what I see as the 4 Ps of Hsdoop in the year ahead: performance, projects, platforms and players.

– more –

Hadoop and DI – A Platform Is Not A Solution

“Hadoop people” and “RDBMS people” – including some DBAs who have contacted me recently –  clearly have different ideas about what Data Integration is. And both may  differ from what Ted Friedman and I were talking about in our Gartner research note Hadoop Is Not a Data Integration Solution , although I think the DBAs’ concept is far closer to ours.

- more -

2013 Data Resolution: Avoid Architectural Cul-de-Sacs

I had an inquiry today from a client using packaged software for a business system that is built on a proprietary, non-relational datastore (in this case an object-oriented DBMS.) They have an older version of the product – having “failed” with a recent upgrade attempt.

The client contacted me to ask about ways to integrate this OODBMS-based system with others in their environment. They said the vendor-provided utilities were not very good and hard to use, and the vendor has not given them any confidence it will improve. The few staff programmers who have learned enough internals have already built a number of one-off connections using multiple methods, and were looking for a more generalizable way to create a layer for other systems to use when they need data from the underlying database. They expect more such requests, and foresee chaos, challenges hiring and retaining people with the right skills, and cycles of increasing cost and operational complexity.
My reply: “you’re absolutely right.”

Diary of an Asian Swing: Day 5

Second day of Singapore meetings. APJ market conversations with IT vendors, watching the emergence of big data here. Seeing activity in the field is always a fascinating counterpoint to the briefings and conferences back home. But the big data phenomenon is surprisingly rapid. Certainly the user conversations have been similar in some ways to those I have in North America, but the players and the details have not made their way here. Yet, we met with a government-funded think tank, building IP for an aggressive thrust into new business opportunities. Different from Silicon Valley think tanks, but no less intriguing – or aggressive.

Capper to the day was dinner with two colleagues in a neighborhood crab restaurant. I have enough opportunities for fine cuisine, but this was a chance to let our hair down, eat like normal people and talk about the day. And Singapore cuisine is unique and marvelous. A wonderful evening and back to the hotel. Next day: Kuala Lumpur.

Diary of an Asian Swing: Day 4

Halfway across the world you go to breakfast and see a neighbor is in your hotel too. How often does it happen? Today I saw an SAP colleague I worked with two decades ago at Sybase – and his colleague, with whom I’ll meet while in Singapore. Great start to the day.

This day was all business. Met several Gartner clients to talk Big Data (since that was my billing.) Interest is high, and like North American firms, one of the key questions, as always, is Value. “What are people doing? What is proving useful from a business perspective?”

Gartner’s local office is beautiful – two floors in a thriving business neighborhood in one of the world’s most vibrant cities. I was told per capita income here is the second highest in the world, and the way the city is kept continues to impress: clean, efficient, beautifully designed and planted with fabulous flora everywhere. Our people here are professional, motivated, friendly and prepared for all our meetings, making sure I know who we’re meeting with and why.

It was a busy, stimulating day capped with dinner with my colleague Arun Chandrasekaran in the Pan Pacific Hotel’s restaurant. Multiple serving stations with different cuisines: Indian, Cantonese, Japanese…. that marvelous Singaporean polyglot cuisine I love. And if the food was good, the conversation was even better. Arun and I talked about how his infrastructure research and my software focus converged in big data and what our next collaboration should be after the Hadoop pilots piece we’re nearing completion on now.

Closing the day with a little BBC World in my room, I watched the pre-election coverage, amused by the overloading of the “battleground states” metaphor when I switched to CNN. They even referred to reporters “embedded” there. Please. Thank goodness this overpriced, overheated exercise will soon be complete. And after all the sound and fury, I don’t expect much will have changed.

Guest Post: Leading the Logical Data Warehouse Charge Has its Challenges

From my colleague Mark Beyer, who speculates about how leadership in moving toward the logical data warehouse (LDW) will be received: 

The logical data warehouse is already creating a stir in the traditional data warehouse market space. Less than 5% of clients with implemented warehouses that we speak with are pursuing three or more of the six aspects of a logical warehouse: 

  • repositories
  • data virtualization
  • distributed processes
  • active auditing and optimization
  • service level negotiation
  • ontological and taxonomic metadata

That means we are in a very early stage regarding the adoption trend, and vendors who are aggressively moving toward it are ahead of their customers.

..more…

Apache Hadoop 1.0 Doesn’t Clear Up Trunks and Branches Questions. Do Distributions?

In early January 2012, the world of big data was treated to an interesting series of product releases, press announcements, and blog posts about Hadoop versions.  To begin with, we had the announcement of Apache version 1.0 at long last, in a press release. Although there were grumblings here and there in the twittersphere that changes to release numbers are meaningless, my discussions with Gartner’s enterprise customers indicate otherwise. Products with release numbers like 0.20.2 make the hair on Procurement’s neck stand on end, and as Hadoop begins to get mainstream attention (Gartner’s clients, see Hype Cycle for Data Management 2011), IT architects and executives find such optics quite important. Hadoop is moving beyond pioneers like Amazon, Yahoo! and LinkedIn into shops like JP Morgan Chase, and they pay attention to such things.

…more…

PDF X-Change – Still The One

Nearly a year ago, I mentioned a wonderful product called PDF X-Change, from Tracker Software,  in a post. It allows me to annotate PDF files, which many vendors maddeningly insist on using for briefings. Why “maddeningly”? Because for me at least, the best place for my notes is in the presentation – it provides the context and I don’t need another window open. In PowerPoint I just use the notes at the bottom of the window. PDF X-Change is a free download, and takes care of the rest of the pitches I see. Read more of this post

Follow

Get every new post delivered to your Inbox.

Join 110 other followers