What Drives Microsoft’s Data Platform Vision?
FEATURED POST BY: Quentin Clark, Corporate Vice President, The Data Platform Group, Microsoft Corporation
If you follow Microsoft’s data platform work, you have probably observed some changes over the last year or so in our product approach and in how we talk about our products. After the delivery of Microsoft SQL Server 2012 and Office 2013, we ramped-up our energy and sharpened our focus on the opportunities of cloud computing. These opportunities stem from technical innovation, the nature of cloud computing, and from an understanding of our customers.
In my role at Microsoft, I lead the team that is responsible for the engineering direction of our data platform technologies. These technologies help our customers derive important insights from their data and make critical business decisions. I meet with customers regularly to talk about their businesses and about what’s possible with modern data-intensive applications. Here and in later posts, I will share some key points from those discussions to provide you with insight into our data platform approach, roadmap, and key technology releases.
Microsoft has made significant investments on the opportunities of cloud computing. In today’s IT landscape, it’s clear that the enterprise platform business is shifting to embrace the benefits of cloud computing—accessibility to scale, increased agility, diversity of data, lowered TCO and more. This shift will be as significant as the move from the mainframe/mini era to the microprocessor era. And, due to this shift, the shape and role of data in the enterprise will change as applications evolve to new environments.
Today’s economy is built on the data platform that emerged with the microprocessor era—effectively, transactional SQL databases, relational data warehousing and operational BI. An entire cycle of business growth was led by the emergence of patterns around Systems of Record, everything from ERP applications to Point of Sale systems. The shift to cloud computing is bringing with it a new set of application patterns, which I sometimes refer to as Systems of Observation (SoO). There are several forms of these new application patterns: the Internet of Things (IoT), generally; solutions being built around application and customer analytics; and, consumer personalization scenarios. And, we are just beginning this journey!
These new application patterns stem from the power of cloud computing—nearly infinite scale, more powerful data analytics and machine learning, new techniques on more kinds of data, a whole host of new information that impacts modern business, and ubiquitous infrastructure that allows the flow of information like never before. What is being done today by a small number of large-scale Internet companies to harness the power of available information will become possible to apply to any business problem.
To provide a framework for how we think applications and the information they generate or manage will change—and how that might affect those of us who develop and use those applications—consider these characteristics:
Data types are diverse. Applications will generate, consume and manipulate data in many forms: transactional records, structured streamed data, truly unstructured data, etc. Examples include the rise of JSON, the embracing of Hadoop by enterprises, and the new kinds of information generated by a wide variety of newly connected devices (IoT).
Relevant data is not just from inside the enterprise. Cross-enterprise data, data from other industries and institutions, and information from the Web are all starting to factor into how businesses and the economy function in a big way. Consider the small business loan extension that accounts for package shipping information as a criteria; or, companies that now embrace the use of social media signals.
Analytics usage is broadening. Customer behavior, application telemetry, and business trends are just a few examples of the kinds of data that are being analyzed differently than before. Deep analytics and automated techniques, like machine learning, are being used more often. And, modern architectures (cloud-scale, in-memory) are enabling new value in real-time, highly-interactive data analysis.
Data by-products are being turned into value. Data that were once considered as by-products of a core business are now valuable across (and outside of) the industries that generate this data; for example, consider the expanding uses of search term data. Perhaps uniquely, Microsoft has very promising data sets that could impact many different businesses.
With these characteristics in mind, our vision is to provide a great platform and solutions for our customers to realize the new value of information and to empower new experiences with data. This platform needs to span across the cloud and the enterprise – where so much key information and business processes exist. We want to deliver Big Data solutions to the masses through the power of SQL Server and related products, Windows Azure data services, and the BI capabilities of Microsoft Office. To do this, we are taking steps to ensure our data platform meets the demands of today’s modern business.
Modern Transaction Processing—The data services that modern applications need are broader now than traditional RDBMS. Yes, this too needs to become a cloud asset, and our investments in Windows Azure SQL Database reflect that effort. We recognize that other forms of data storage are essential, including Windows Azure Storage and Tables, and we need to think about new capabilities as we develop applications in cloud-first patterns. These cloud platform services need to be low friction, easy to incorporate, and operate seamlessly at scale—and have built-in fundamental features like high availability and regulatory compliance. We also need to incorporate technical shifts like large memory and high-speed low latency networking—in our on-premises and cloud products.
Modern Data Warehousing—Hadoop brought flexibility to what is typically done with data warehousing: storing and performing operational and ad-hoc analysis across large datasets. Traditional data warehousing products are scaling up, and the worlds of Hadoop and relational data models are coming together. Importantly, enterprise data needs broad availability so that business can find and leverage information from everywhere and for every purpose—and this data will live both in the cloud and in the enterprise datacenter. We are hearing about customers who now compose meaningful insights from data across Windows Azure SQL Database and Windows Azure Storage processed with Windows Azure HDInsight, our Hadoop-based big data solution. Customers are leveraging the same pattern of relational + Hadoop in our Parallel Data Warehouse appliance product in the enterprise.
Modern Business Intelligence—Making sense of data signals to gain strategic insight for business will become commonplace. Information will be more discoverable; not just raw datasets, but those facets of the data that can be most relevant—and the kinds of analytics, including machine learning, that can be applied—will be more readily available. Power BI for Office 365, our new BI solution, enables balance between self-service BI and IT operations—which is a key accelerant for adoption. With Power BI for Office 365, data from Windows Azure, Office, and on-premises data sources comes together in modern, accessible BI experiences.
Over the coming months, we are going to publish regular posts to encourage discussions about data and insights and the world of modernized data. We will talk more about the trends, the patterns, the technology, and our products, and we’ll explore together how the new world of data is taking shape. I hope you will engage in this conversation with us; tell us what you think; tell us whether you agree with the trends we think we see—and with the implications of those trends for the modern data platform.
If you’d like more information about our data platform technologies, visit www.microsoft.com/bigdata and follow @SQLServer on Twitter for the latest updates.
