Hadoop Investments Continue: Teradata, HP Jockey For Position

Interest from the leading players continues to drive investment in the Hadoop marketplace. This week Teradata made two acquisitions – Revelytix and Hadapt – that enrich its already sophisticated big data portfolio, while HP made a $50M investment in, and joined the board of, Hortonworks. These moves continue the ongoing effort by leading players. 4 of the top 5 DBMS players (Oracle, Microsoft, IBM, SAP and Teradata) and 3 of the top 7 IT companies (Samsung, Apple, Foxconn, HP, IBM, Hitachi, Microsoft) have now made direct moves into the Hadoop space. Oracle’s recent Big Data Appliance and Big Data SQL, and Microsoft’s HDInsight represent substantial moves to target Hadoop opportunities, and these Teradata and HP moves mean they don’t want to be left behind.

more

 

Aspirational Marketing and Enterprise Data Hubs

In the Hadoop community there is a great deal of talk of late about its positioning as an Enterprise Data Hub. My description of this is “aspirational marketing;” it addresses the ambition its advocates have for how Hadoop will be used, when it realizes the vision of capabilities currently in early development. There’s nothing wrong with this, but it does need to be kept in perspective. It’s a long way off.

–more–

Hadoop Summit Recap Part Two – SELECT FROM hdfs WHERE bigdatavendor USING SQL

Probably the most widespread, and commercially imminent, theme at the Summit was “SQL on Hadoop.” Since last year, many offerings have been touted, debated, and some have even shipped. In this post, I offer a brief look at where things stood at the Summit and how we got there. To net it out: offerings today range from the not-even-submitted to GA – if you’re interested, a bit of familiarity will help. Even more useful: patience.

–more–

That Exciting New Stuff? Yeah… Wait Till It Ships.

A brief rant here: I am asked with great frequency how this RDBMS will hold off that big data play, how data warehouses will survive in a world where Hadoop exists, or whether Apple is done now that Android is doing well. There is a fundamental fallacy implicit in these questions.

–more–

Amazon Redshift Disrupts DW Economics – But Nothing Comes Without Costs

At its first re:Invent conference in Late November, Amazon announced Redshift, a new managed service for data warehousing. Amazon also offered details and customer examples that made AWS’  steady inroads toward enterprise, mainstream application acceptance very visible.

Redshift is made available via MPP nodes of 2TB (XL) or 16TB (8XL), running Paraccel’s high-performance columnar, compressed DBMS, scaling to 100 8XL nodes, or 1.6PB of compressed data. XL nodes have 2 virtual cores, with 15GB of memory, while 8XL nodes have 16 virtual cores and 120 GB of memory and operate on 10Gigabit ethernet.

Reserved pricing (the more likely scenario, involving a commitment of 1 year or 3 years) is set at “under $1000 per TB per year” for a 3 year commitment, combining upfront and hourly charges. Continuous, automated backup for up to 100% of the provisioned storage is free. Amazon does not charge for data transfer into or out of the data clusters. Network connections, of course, are not free  – see Doug Henschen’s Information Week story for details.

This is a dramatic thrust in pricing, but it does not come without giving up some things.

More…

Diary of an Asian Swing: Day 3

This was a day of transition. No meetings in Hong Kong, so after a leisurely breakfast and a look at the news, I settled down for a rare session of uninterrupted writing. It was still Sunday back home, so the email was relatively caught up and I could focus. Finished first drafts of some Gartner Magic Quadrant DW DBMS content and sent them off to colleagues for review and assembly into our eventual document.

This MQ is my second, and I’m really enjoying the process this time now that I’m not trying to figure out what happens next. I’m especially pleased with the process of combining interview data from customer interviews and analysis of our inquiry traffic – hundreds for each of the four authors – with surveys we conducted specifically for the report.

Mark Beyer built a fantastic link for feeding survey criteria measured by numeric scores from customers directly into relevant cells on our underlying spreadsheet. We had already done some collective scoring of our own in those cells, and the new exercise showed us how customers read the same issues. And it moved some of the scores significantly, with some vendors doing better than we expected in some areas, and others getting hammered. When a sizable number of survey respondents highlight an issue like support as a serious weakness, one has to take notice.

Several hours of uninterrupted time, a luxury that made the work move quickly, gave way to a decision about what to do with a free afternoon. I decided to use it for more work, so instead of an excursion I headed to the airport hours ahead of schedule to work in the attractive Cathay Pacific lounge. But I was surprised by a helpful check-in agent who told me there was an earlier flight I could get onto. As a result, I arrived in spectacular Singapore late in the evening instead of well into the night, and was in my hotel for a good night’s rest before early morning meetings the next day.

And of course, working on the place – even without wi-fi – was just as good as working in the lounge. So I had the chance to complete a new draft of a forthcoming Hadoop Pilot Best Practices piece and send it off to a collaborator. A good day indeed.

Diary of an Asian Swing: Day 1

I’ve never been a diarist. But as an exercise, I’m going to document this trip: two weeks on the road to Asia and Australia. Almost all work, though there is one day of weekend and recovery time built in.

Friday, Nov 2. Cathay Pacific flight to Hong Kong. Business class. Comfortable, well-appointed cabin. Friendly, courteous staff.

Learned system, table, storage, seat. NO WIFI! OMG. 14 unconnected hours – that’s what stimulated the idea to blog. Otherwise I would have been tweeting a lot. Which is fun, but this will be a change. Let’s start with some entertainment and settle in….

Video presentation of Sir Paul McCartney’s Kisses on the Bottom. Diana Krall. John Pizzarelli. Joe Walsh! It’s delightful – commentary, moments of studio play, tenderness. No real edge, but that’s not what it’s about. Luscious melody, beautiful jazz harmonies. Diana is a sideman here and a brilliant one. McCartney is who he is – always a touch too smooth and a touch too sweet, but it works so well here. Wonderful selection of songs and I love his own Valentine one.

Next up is a video piece about the Who’s Quadrophenia. Clearly part of Townshend’s current campaign to remind the world of his brilliant work – and well done. A good narrative sprinkled with revelatory bits about Moon, and Daltrey, and the terrible chaos they endured. Live performance footage and some nice stuff at the control board with Pete and the engineers highlighting dimensions of the music.

Now, it’s time to get to work – pick some music for background, get the computer out. Found Xuefei Yang – guitarist, playing Bach. Lovely tone, tempos, clarity. Don’t know her. Will fix that. And now to work – Magic Quadrant writing.

Wait. First a survey by Cathay Pacific. Why not?

A few Magic Quadrant hours reviewing interviews, surveys, briefing content, and our own scoring and it’s on to writing up a draft for one of the vendors and sending it off for my colleagues to comment and collaborate on the content.

Now, a break. I’ve earned one. Stretch the legs, a few exercises…

Treated to a reboot of the airplane’s computer system. Red Hat, it turns out. A parade of system level messages marched across the screen, unfathomable to anyone unfamiliar with Linux. The darkness, and eventually, a blessed progress bar, followed by the return of the flight map. We’ve passed over Siberia and the Gulf of Shelekhova; now we’re over the Sea of Okhotsk, only 6 and a half hours to go.

A moment to update this diary and a little reading – music choice Wynton Marsalis & Eric Clapton. Really? Didn’t know about this one. Would love to look it up on the net, but NO WIFI. Oh well, a little Time magazine, and then some Hadoop Operations by Eric Sammer.

Eric’s book is highly recommended – lucid, well written and doesn’t demand extraordinary technical depth to understand. Find it here: http://www.amazon.com/Hadoop-Operations-Eric-Sammer/dp/1449327052/ref=sr_1_1?s=books&ie=UTF8&qid=1351944059&sr=1-1&keywords=hadoop+operations 

Arrived on time. Hong Kong continues to impress: efficient, clean, modern as I remembered. Nice hotel. Good wifi. Sent off MQ writeups to colleagues and synchronized all the email I did while offline. Time to crash. Day One is done.

Guest Post: Leading the Logical Data Warehouse Charge Has its Challenges

From my colleague Mark Beyer, who speculates about how leadership in moving toward the logical data warehouse (LDW) will be received: 

The logical data warehouse is already creating a stir in the traditional data warehouse market space. Less than 5% of clients with implemented warehouses that we speak with are pursuing three or more of the six aspects of a logical warehouse: 

  • repositories
  • data virtualization
  • distributed processes
  • active auditing and optimization
  • service level negotiation
  • ontological and taxonomic metadata

That means we are in a very early stage regarding the adoption trend, and vendors who are aggressively moving toward it are ahead of their customers.

..more…

Mark Beyer, Father of the Logical Data Warehouse, Guest Post

Another guest post, this time from my colleague and friend Mark Beyer.

My name is Mark Beyer, and I am the “father of the logical data warehouse”. So, what does that mean? First,  if like any father, you are not willing to address your ancestry with full candor you will lose your place in the universe and wither away without making a meaningful contribution. As an implementer in the field, I was a student and practitioner of both Inmon and Kimball. I learned as much or more from my clients and my colleagues during multiple implementations as I did from studying any methodology. My Gartner colleagues challenged my concepts and helped hammer them into a comprehensive and complete concept. Simply put, I was willing to consider DNA contributions from anyone and anywhere, but through a form of unnatural selection, persisted in choosing to include the good genes and actively removing the undesirable elements.

more…

IBM Fills Out Netezza Lineup With High Capacity Appliance

In the months since IBM closed its Netezza acquisition, the data warehouse appliance pioneer has been busy, if the announcements at this week’s Enzee are any indication. An enthusiastic crowd – 1000 strong – heard CEO Jim Baum deliver the news: new hardware, software and partnerships.The biggest news was The Appliance Formerly Known As Cruiser, now known as the Netezza High Capacity Appliance (HCA). A wag made up some t-shirts bearing the acronym TAFKAC and did quite well. IBM is aiming to push the size perception for Netezza higher. How high? Half a PB in a rack. You can scale it to 10PB.

more

Follow

Get every new post delivered to your Inbox.

Join 140 other followers