Microsoft Plays Where’s Waldo? With BI – Good Idea

In April, I was critical of the BI messaging I heard from Microsoft – as told, it was long on benefit adjectives and short on architectural clarity. But things have changed since then, and the Combined Tech Ed/Business Intelligence Conference made that very clear. Do I see more clarity because I now know more of the detail, and have internalized my own narrative? Likely. But it would be outrageously self-centered of me to think that was all. The fact is, the story is being told better, and there is much to tell. It needs to be told well because it’s complex behind the “simple” descriptions that underlie Microsoft’s ultimate value proposition: BI should be an easily consumed, context-aware service available to everyone, not a separate offering. Microsoft is tackling the continuing grand challenge of BI – expanding usage beyond the 20% of potential users that are BI consumers today.Microsoft’s BI vision hasn’t changed: the three key words (there are always three key words) remain familiar, collaborative, and managed. As Ted Kummert, SVP of the Business Platform Division put it: “Managed self-service BI will enable an order of magnitude more users.” To begin with, there is no “center” to the BI story: Microsoft has carefully packaged BI functionality across three key products: SQL Server, Office and SharePoint – each of which requires a license. The attractive scenarios being painted – and make no mistake, they are very compelling – will require all three. That’s a nice revenue enhancement model. It’s also a delivery challenge: it simultaneously suggests less IT involvement in usage while it offers new management capability and an ever-increasing set of behind the scenes features that IT pros will be required to master and deploy to deliver on Microsoft’s seductive promises.

The trinity involved begins with Office, and specifically with Excel – that’s where users and BI professionals alike are most likely to actually interact with data today. It’s also the source of Microsoft’s definition of a power user: “power users know pivot tables and 15-20% of our customers do,” said Herain Oberoi, Group Product Manager. (The twitterverse erupted when I quoted this; my fellow analysts think this percentage is far too high.) And it’s interaction Microsoft is focused on – not just consumption of reports.

End users are increasingly familiar with SharePoint; 70% of Microsoft’s enterprise customers are deploying it, Tom Casey, General Manager for SQL Server BI, asserted. This creates an opportunity: Microsoft believes that getting people to participate in a “trusted information environment” gives the organization a hook to use SharePoint for managing the environment and governing usage as well. Microsoft’s preoccupation with eye candy dominated this discussion. They showed attractive metaphors like a “carousel” of report screens using Silverlight – companies can build their own as well. Users will be able to build reports out of existing ones using the new SharePoint Report Part Gallery – another way of making BI resident everywhere – and requiring ownership of all the parts to do the cool stuff being shown.

SharePoint also provides reports to be used by IT for tracking usage, data sources, etc.  But it appears that governance remains an exercise left to the implementer; Master Data Services (MDS), a new and useful feature of SQL Server 2008 R2, was not mentioned during the analyst day or the BI event keynote. (There were sessions during the conference and the twitter traffic on them was mixed.) In response to a question during the analyst event, I learned that in this version there are no pre-built alerts to identify new data sources being used – “you’d have to build that workflow.” If a user creates a relationship with an outside data feed, there is no automated quality assurance.

How quickly will this connection take hold among Office users, who were out in force and even got their own water cooler recognition? Today, it’s likely that the two populations: SharePoint-using IT admin folk on the one hand, and BI producers (and consumers who experiment) on the other are separate. Even at the conference, the SharePoint sessions were in a different part of the complex. The piece parts for the solution are certainly there; there is a role-based interface that is authorization-oriented. There is DRM. And SharePoint has useful templates to help organizations get started; one is a BI Center that, while incomplete, can be easily customized and will give its customers a head start. It includes a library of content and widgets, and leverages Performance Point Services, itself a part of SharePoint Enterprise.

Interestingly enough, there seems to have been some evolution of the logos involved that reflects this approach. You may have already noticed that the familiar SQL Server logo is also used in slightly modified form by Forefront, the security product. At this event, the SharePoint logo, which used to look like the Office logo, now looks like the SQL Server one, with the turned over flap in a different color.

The Power of Ubiquity in the Services Model

This vision of BI everywhere begins in my view with the powerful model of services in SQL Server, where much of the data Microsoft BI users want obviously will live. From a product architecture perspective, SQL Server licenses give the customer access to a series of Services. Reporting Services (SSRS), first released in 2004 as an add-on to SQL Server 2000, got some new tweaks with the 2008 R2 release. Its Report Designer is hosted inside of Business Intelligence Development  Studio (BIDS), a specialized Visual Studio that ships with SQL Server. So SQL Server users are able to of deliver SSRS output without SharePoint or Office in a variety of ways: HTML, PDF, TIFF, XML, CSV and Atom among them. And interacting with it doesn’t require other products either. But that’s not how Microsoft tend to show people using it. 

This illustration shows how the Services interact as part of the overall SQL Server offering; it’s not up to date and will need some tweaking. I’ll skip additional pictures for the other Services, but briefly: Integration Services is an ETL platform which replaced Microsoft Data Transformation Services (DTS), in the SQL Server 2005 release. In the 2008 R2 release, it got new connectors for Oracle and Teradata, as well as SAP BW. Analysis Services underlies the OLAP capabilities: it builds cubes, is deployed on multiple servers if desired while sharing a single database, and provides a place to do data mining to boot.

Donald Farmer, a man who most Microsoft executives say needs no title, showed the analysts how the services model in Microsoft’s vision creates synergies that facilitate new and powerful opportunities. He discussed the data mining capabilities that have been available within SQL Server since 2005 – features that are based on typical approaches, and some original ideas built on algorithmic research from Microsoft Research labs.

Analysis Server hosts the algorithms. The data is passed to the algorithms to create and train models. In addition to SQL Server, Excel tables can be the source; so can an external connection, in which case Excel is just brokering the exchange. Client apps can send data in and can get data out. SQL Server has  PMML support, so models can be exchanged with non-Microsoft products. BI Developer Studio supports all these and can leverage DM Extensions for Excel (DMX), a free download. These are functions installed in SQL Server for Excel. DMX can be used “just like T-SQL.” You INSERT data into a model, SELECT from it.

SSIS also comes into play here; its components can perform training and testing in parallel with a database load in the flow. This is powerful for champion/challenger processes. Again, note that getting the value out of all this requires both Excel and SQL Server – more interleaving.

There is a progression of analytic types as one moves from fixed reporting through ad hoc analysis into data mining, and it becomes more predictive. Microsoft has not been absent from the industry movement toward predictive analytics, though it may not have been as visible as other firms in promoting it. Farmer didn’t claim that this offering is competitive with SAS or IBM’s SPSS, but the users of those products are not the target.

The power of the services-based model also becomes visible in Microsoft’s first, albeit hesitant, steps in using data mining elsewhere in the product portfolio. Dynamics CRM Analytics Foundation has embedded data mining capability. Farmer showed a customer example: a telco in the Philippines selling ringtones. Applying  modeling to their marketing data has been very successful, and the resulting system is now used to manage all their marketing. Microsoft has done very little of this cross-business integration with Dynamics – it was absent from the messaging. So, for that matter was discussion of business process – strange given that the Business Platform Division, is part of Bob Muglia’s Server and Tools Business unit, which owns Windows Server, SQL Server,Visual Studio, Azure and others – he keynoted TechEd. Ted Kummert’s remit specifically focuses on Microsoft’s “integrated application platform,” which means he has BizTalk as well – which makes the absence of business process focus in the BI story all the more disappointing .

Bottom Line: Microsoft Is Moving Steadily Up The Stack

It’s no accident that Microsoft has continued to grow revenue and market share in the database business while both Oracle and IBM have delivered flat or slightly down results there over the past two years. And with each release, Microsoft has made another step forward. SQL Server 2008 R2 is no exception, notwithstanding the oddity of the nomenclature for a 2010 product. Its services focus has created a strong foundation for binding in new use cases for BI, and this wedding of SQL Server to SharePoint and Office leverages the technical foundation provided to continue the assault on the market.

There’s always more to do – Microsoft simply doesn’t participate in the conversation about bringing business analytics to the established packaged application infrastructure. The high-end Parallel Data Warehouse is still MIA. The metadata story, the “stored procedures in the database” story, in-memory away from PowerPivot, the NoSQL counterthrust – all are yet to be delivered. I’ll post about several of those topics in the days ahead. But Microsoft is moving up and defining the battle on its own terms, with the likelihood of strong market support for its vision.

Disclosures – Microsoft is not a client of IT Market Strategy