SQL Server 2012 and the New World of Data

Editor's Notes

1. There has never been a more exciting time with respect to the world of data. We are seeing the convergence of three significant trends that are fundamentally transforming the industry. The first is the explosion of data, which is being discussed quite broadly today. The second is the ushering in of a new era of tech innovation in areas like social, mobile, advanced analytics and machine learning. And finally, we are in the midst of a radical reshaping of the economics of computing.
2. Storage Costs: Industry standard hardware has radically reduced the cost of compute, storage and memoryVolume, Variety & Velocity: More types, shapes and sizes than ever before…Where does it come from? 1) Transactional growth in your business. 2) New data sources – image files, sensors driven 3) Moving business online – entirely new channels, with different pace at which the business needs to operate. entirely new scale and scope of the kinds of data we are trying to gain insights fromConsumerization of IT: New ways in which we are discovering and collaborating on data – The total number of social networkers in Q4 CY11 exceed all internet users in Q4 2006…and the technology is moving into Enterprise IT. New questions are being asked of the business – it’s now pretty common to want to know what’s being said in social media about your products, services and companies… Web (Bing) + Social Media (sentiment) + Business Data = new value. Employees want to be empowered wherever they are, we are all carrying around devices with powerful computational capabilities. With mobile phones, tablets and slates. Driven by 3G adoption at 35% Y/Y there are now 835m smartphone subscribers vs. 5.6B mobile phone users.Cloud Computing: Cloud computing holds the potential of a 10x TCO advantage over traditional systems and we can now do things we could not do before… This sea change presents new challenges for organizations who wish to remain agile and innovative.
3. To tap into these new opportunities, you need a Modern Data Platform…A Data Platform that delivers the following:Embraces the new scope, new scale, and new diversity of the data to really be in a position to manage and process, any data of any size, Big Data and Small Data across on-premises and the cloud. Delivers new capabilities in how todiscover, combine and collaborate with your data and the world’s data to harness the power of all data. Where information is delivered to you based on who you are, who you’re with, where you are and what you’re doing. Bringing the right data, to the right person, at the right time to make the right decision.Projecting immersive experiences for applications and data to new form factors in an appropriate and rich way delivers new insights, to all users via familiar tools, wherever they are – and on any device.This strategy puts customers on an immediate path to maximize their business opportunities and we are confident there is no other vendor in the industry with the vision or the assets necessary to deliver on it. We are bringing together the power of SQL Server 2012, world’s most popular RDBMS, with Office, the world’s most popular productivity suite, and connecting that into our online platform.
4. Any Data, Any Size, Anywhere embraces the new scope, new scale, and new diversity of the data to really be in a position to manage a process, any data of any size, Big Data and Small Data across on-premises and the cloud. An organization needs to be able to do this anywhere, whether it be in their data center, or a Private Cloud and/or in the Public Cloud. Bringing together data of all types/sizesIt's about processing any data of any size, anywhere. And this has been a continuous theme for SQL Server. Even when we weren't using these words, this has been a core part of our strategy going all the way back to SQL Server 7.0. Microsoft had the belief that the platform was more than just a relational database, organizations needed to support the other types of data and then bring all of the services needed to really process and gain insight from it in one integrated platform. And that's a continuous theme for Microsoft looking back, and going forward.Mission-critical scale, on-premises to cloudWith today’s world of continuous services and connected devices, it’s simply expected that applications and data will be up and available on the spot. HA, Security, Performance – these are all table stakes across the platform, on-premises to public cloud. The cloud design point enables organizations to better manage SLAs and scale to uniquely fit their needs and with the needed level of agility and TCO. Anywhere is really acknowledging the delivery of this with the cloud. Again, the Microsoft model is about choice, it's about a hybrid world, services from our data center, as well as from your data center, and connecting those environments to meet your needs.Common management and developmentAnd it's about a consistent platform, consistent programming models, behaviors, tools, consistent no matter where you deploy. Microsoft can offer customers a common management and development platform across the hybrid world, so organizations get the ultimate in flexibility and choice and productivity out of their dev and IT teams.In today’s world and looking ahead, support for any data, any size anywhere, is the baseline set of requirements an organization should come to expect; a data platform needs to do new things – discover, explore, enrich data that is living with the organization to help the user, IT Pro, or business owner bridge out to the external world to use that data to enrich the data from the organization. Customer Example BetOnSoft develops and manages more than 110 online casino games, played every day by thousands of players worldwide. The company needed to ensure that its games are highly available, because players are online around the clock. BetOnSoft also wanted to prepare for business growth by scaling its database while maintaining application responsiveness. In addition, its applications must perform key business-critical analytics in real time. In November 2011, the company deployed a hybrid application solution that takes advantage of the high-availability features in Microsoft SQL Server 2012 AlwaysOn and the scalability of Windows Azure SQL Database. Now, the company’s infrastructure can exceed 10 times its previous peak loads while running intensive real-time data analytics. BetOnSoft has achieved the availability it needs and can use its hybrid infrastructure to scale to meet unexpected business growth.“The firewall and security configurations in Windows Azure SQL Database are great. Our cloud-based services are now as secure as our data-center-based services”
5. ERP, SCM, CRM, and transactional Web applications are classic examples of systems processing Transactions. Highly structured data in these systems is typically stored in SQL databases. Web 2.0 are about how people and things interact with each other or with your business. Web Logs, User Click Streams, Social Interactions & Feeds, and User-Generated Content are classic places to find Interaction data.Ambient data trends is becoming “Internet of Things”. Mary Meeker has predicted 10B connected devices by 2015. Sensors for heat, motion, pressure and RFID and GPS chips within such things as mobile devices, ATM machines, and even aircraft engines provide just some examples of “things” that output ambient signals…There are multiple types of data personal - > organizational - > public - > private So we should NOT minimize our thinking to just data that flows through an organization. Ex. The mortgage-related data you may have COULD benefit from being blended with external data found in Zillow, for example.Moreover, the government has the Open Data Initiative. Which means that more and more data is being made publicly available.
6. Microsoft offers an end to end Data Platform for all data and the easiest to use tools to analyze it. Microsoft’s data platform seamlessly manages any data (relational, non-relational and streaming) any size (GB -> TB -> PB), anywhere (on premises and cloud), enriches existing data sets by connecting to the world’s data and enables all users to gain insights with familiar and easy to use tools through Office and SharePoint.Today, customers manage relational data with our industry leading database products - SQL Server 2012, SQL Server Parallel Data Warehouse appliance, the industry’s first data marketplace – Azure Marketplace Datamarketand familiar BI tools such as PowerPivot for Excel and Power View for SharePoint. In addition, we support new capabilities such as streaming data via SQL Server StreamInsight we will include a new Hadoop-based distribution on Windows Azure and Windows Server for processing petabyte scale unstructured data andenable high speed data movement to and from your relational data on SQL server and unstructured data in Hadoop.
7. The majority of CIOs we speak with today ask fundamental questions about their application portfolio 1) Which apps should be virtualized 2) which apps do I put in a private cloud either in my DC or someone else’s 3) which applications do I write for the new Public Cloud design point to take full advantage of the promise of the cloud in terms of scale and TCO?Ultimately we believe that it is a combination of all three, and this philosophy of choice is driving our strategy. We call this Hybrid IT.There are really three key dimensions upon which to make this decision. 1) Technology i.e. perf, scale, latency 2) Economics – capex/opex 3) Compliance – data sovereignty, regulatory and corporate compliance Let’s take a look at how these are playing out across a set of common database application scenarios….TraditionalOrganizations will primarily keep their largest and most complex mission critical applications in a Traditional non-virtualized environment, such as OLTP, data warehouses as well as Big Data.The primary reasons come down to data compliance as well as the complexity of achieving performance SLAs for a large mission critical workload. Customers are experimenting and looking to take some of these workloads into a Private Cloud where the application is virtualized, but today the majority of these mission critical applications continue to sit in a non-virtualized environmentPrivate CloudWhat we see today are departmental or workgroup applications, especially OLTP and BI workloads getting virtualized and customers looking further in reducing OPEX by investing in building out a Private Cloud. The primary driver here is economics through Improved infrastructure utilization, management, and operations. Or compliance if the application is subject to restrictions around where the data resides, preventing the move to Public Cloud.Public CloudIn the Public Cloud we see customers either consuming finished applications or building new applications. The database scenarios we often see here are Tier 2 and Tier 3 OLTP, BI workloads and now Big Data. The primary drivers here include rapid application development, where time to solution is a key issue. Leveraging the economies of scale a of a Public Cloud provider like Microsoft. Finally the need for elastic scale where Windows Azure and Windows Azure SQL Database can scale dynamically to meet demand without the customer having to worry about provisioning hardware or retooling the application to scale the database…
8. Delivers new capabilities in how to discover, combine and collaborate with your data and the world’s data to harness the power of all data. Where information is delivered to you based on who you are, who you’re with, where you are and what you’re doing. Bringing the right data, to the right person, at the right time to make the right decision.Enrich your data with the world’s dataA lot of the historical conversation has really been about your data, the data within the walls of your organization, and giving you the tools to be able to process that, manage that, and gain insight. But, there's a world of data out there. There are new questions being posed within and to organizations now. When we talk to customers, it's not uncommon for us to hear customers say, ‘look, I'm being asked different kinds of questions now. I'm asked now to understand what's being said in social media sites, what is the sentiment around these brands and these products. And that's a different set of information and a different set of questions than I've ever been asked before.’ Empower developers to build new services and applicationsProviding an ecosystem of APIs and building block services to support and accelerate developers to quickly build those apps and connect to APIs. OData is an example of this…Vibrant marketplace ecosystem for the world’s dataNow, we do bring other data into our walls today, to enhance the work that we're doing. Often this is reference data, reference content that we're using. We established the Windows Azure marketplace for data around the idea of making that easy. Having a one-stop shop where you can go find all the reference content that you can need, bringing the world's data together, so you can have it to enrich the work that you're doing.Microsoft launched Windows Azure Marketplace over 18-months ago. It's a very unique one-of-a-kind service, but there are some interesting scenarios we think about beyond what we're doing today. (ADD BING?) We provide a search experience today and we provide visualization so you can explore the data, but the fact is we understand something by knowing your data, by being able to classify your data semantically, we can bring data to you and make recommendations. You have an address. What would you be interested in? Address verification data that works with data quality services, demographic data, geospatial data, using technologies like recommendation engines we can help bring the world's data to you. That's a very exciting capability and really enhances that search experience.CUSTOMER EXAMPLEA large American automaker – is looking to launch a battery-powered electric passenger vehicle. Fully charging the vehicle at home or a business should take just over 3 hours to complete, however as the cost of electricity can vary by the time of day, when you charge the vehicle can have an important impact on costs of ownership. So, every new electrical vehicle will offer a Value Charging system powered by Microsoft, to help owners in the US charge their vehicles at the cheapest utility rates, lowering cost of ownership. To do this, the automaker will rely on an electric utility rates dataset available on the Windows Azure Marketplace that offers current information from 100 utilities covering more than 10,000 US zip codes and 1,500 Canadian Postal Codes. Combining the world’s data with their own product creates a unique value proposition for customers.
9. Microsoft introduced Windows Azure Marketplace DataMarket, the first Global Marketplace for connecting you with the World’s data. A one-stop shop for bringing the world’s data to your doorstep…The Marketplace is now available in 26 markets across the globeThere are more than 40 providers and 100’s of data sets available to youBringing power and breadth of Bing to the Marketplace – New Petabytes of Data and trillions of data points across a complete range of content categories. Creates massive opportunity – the value of one dataset increases with its connection to another Car dealer - correlate avg price of gas, sales of hybrids v. SUVs, the core demographic of my best customers, and choose the right time and areas in which to advertiseStart-up developer – builds apps that consumes data from a local public transportation service and connects it with data on coffee shop locations in New York – provides vast community of commuters with a way to get to work on time AND be fully caffeinatedResearcher - use sensors from ocean floor to collect data related to earthquake activity. That data increases in value when combined with weather data so that we can better predict the need to send emergency alerts to local populations.
10. So where are we going next? There are some interesting scenarios beyond what we deliver today… We provide a search experience for finding data and visualization experiences for exploring data….But it’s not enough…we are stacked to the rafters with data, and its too hard to find interesting and relevant dataThe Day is coming when Information will be streamed to you based on who you are, who you’re with, where you are and what you’re doing. And it turns out that meta data and semantic classification are keys to unlocking more valueIt’s because we understand the meta data of the data you’re using that we can bring new interesting data to you and help you integrate it with your data…For example, you’re working with an address…what data would you be interested in? Possibly address verification data from one of our providers…Using meta data and recommendations engines Microsoft can bring the right data to you at the right timeBuilding on the huge success of the marketplace, Microsoft is delivering a new class of data services for discovering, exploring, refining and publishing your data and the world’s data. We have recently demonstrated a prototype this technology….codename Data Explorer, as seen in the screenshot here. This is going into our Azure Lab environment for public consumption by the end of the year. Easy to discoverEasy to transform and aggregate data – yours and the worldsEasy to share across the orgEasy to publish backJust a couple things I will point out on the screenshot of the tool…Top left shows some how we are classifying your dataTop right shows a visual view of where your data sources are coming fromBottom left shows some recos based on the classification of the data you are usingMicrosoft will use context and intent to help you easily and quickly get the “right data where and when you need it to maximize” it’s value. We are the only company who can deliver active data because we know who, how, and when you are using data…. meta data + context + intent = active data Microsoft is also moving forward to add new capabilities to this service – to use the context that we have about your data and the data sets you’re using to bring more data sets of interest to you. Have an address? Here’s data sets that can work with data quality services to cleanse your data? Need demographic data, here’s a few choices? How about geo spatial data? Technologies such as recommendation engines can greatly enhance the discovery of data sets that can immediately be put to use.Another set of capabilities are about making it easy to transform and aggregate this information but also to discover what others have done. Chances are that within the walls of your business today, multiple people are taking the same data sets and transforming and aggregating them in similar ways in order to complete their analysis. Our vision is to make those operations easier for end users but also to create marketplace capability to be able to share and discover what others have done.
11. Projecting those experiences for those applications and data to these other form factors in an appropriate and rich way delivers new insights, to all users via familiar tools, wherever they are – and on any deviceImmersive experiences with any dataThe New World of Data lands with the ability for people to gain the insights they need to move forward, to get their questions answered – wherever they are. The Microsoft strategy is, no matter what role the end user is in, to give them the ability to get questions answered in the tools they use every day. Getting insights should be easy; whether a professional BI user, or a user of line of business applications, or whether an avid user who lives in Excel every day. Now, we remain committed to the capabilities needed for professional BI solutions but, Microsoft has also been investing very much in a story for the rest of the users, the rest of the end users, to really make BI technologies and solutions available to them without them having to understand the technology and the terminology behind it.Empower all users through Office and SharePointThis was a journey we started in SQL Server 2008 R2, and Office 2010, a scenario we called Managed Self-Service Business Intelligence, and it brought together three things. It brought together a user experience in Excel and PowerPivot for Excel, to make it easy for the Excel user to build analytical solutions. We brought the environment into SharePoint, which enabled sharing, and collaboration, so I could discover the work that other people had done and build on top of that. And then by bringing it into SharePoint, it gives IT a point of visibility and management, to get the benefits of IT's governance on the whole thing. And what you get is this balance of end-user empowerment with the benefits of IT's governance, and that really makes this a solution that's ready for the enterprise.Accessible wherever you areNow, let's get back to talking about devices. There's a lot of computational power here and as users we're already accessing a ton of business information on these devices. We can access email, documents and social media, for example. Now, why is any other type of business information different? Why are BI applications any different? End users should be able to access them wherever they are, using whatever device they choose; this includes smart phones and tablets across all mobile platforms and devices.CUSTOMER EXAMPLEA large mobile carrier – This company wanted to tackle churn…They are trying to correlate call quality (dropped calls, call volumes, etc) with customer retention. They have 100TBs of call log data…stored in Hadoop. Customer acct information is in operational systems and data warehouses. Ultimately they want a simple way to correlate this information in a way that makes sense to them without technology, physical location of data, how it is stored to get in the way.
12. We started our Business Analytics journey with a unique differentiated perspective…which at the time was not shared by the competition… It was founded on a simple set of principles….It started with the notion of empowering all users with self-service BIBut balancing that with the right IT governance and controlAnd valuable insights are created by individuals, but are only maximized when they are easily shared and collaborated on by teams We pioneered the idea of Self-Service BI with SQL Server 2008 R2 when we first introduced the popular PowerPivot Add-In for Excel, enabling users of all levels to quickly access data from virtually any source, to analyze and model that data in Excel.  Then we enabled collaboration and sharing these insights through SharePoint. After all, BI is about decision making and decision making is an inherently collaborative process. By integrating BI more seamlessly into our existing collaboration platform, and existing business workflows, we enabling a more collaborative and effective decision making process.. Through SharePoint we also provide what we refer to as managed Self-Service BI – providing IT with the tools they need to manage and protect the data and content that end users are creating. Microsoft uniquely provides IT administrators with the insight and oversight they need via tools for monitoring and managing user-generated content, as well as for transforming that content into corporate grade solutions that are professionally managed by the IT department. This simplifies compliance without hampering user agility and creativity. With the new release of SQL Server 2012 we built on this foundation with the introduction of a new tool in our self-service line up – Power View. Power View is a highly interactive, browser-based data exploration, visualization, and presentation experience for users of all levels—from business executives to information workers. Now anyone can create a report in just seconds, transform the “shape” of data with a single click, add powerful timed animation sequences to quickly identify trends or anomalies…With Power View customers can get insights at the speed of thought.Looking ahead: Recently, Microsoft announced that we will enable delivery of solutions on Microsoft BI across a range of devices. Our goal is to provide great touch based experiences on all devices including iPads and the best in class experience on Windows….
13. So we spent some time today talking about some real world challenges and opportunities around Data. And how Microsoft can help address those challenges and help you maximize the value of your data to grow your business and gain a competitive advantage in the market…We have a compelling vision for this new world of data, and we have an incredibly strong set of offerings today across Appliances, traditional box software and Cloud and they get even stronger with a new set of product releases right around the corner…And we are investing in new capabilities, like Hadoop, to put you into a position to address a broader range of your data challenges with one consistent data platform….
14. Today, Microsoft took a major step toward the New World of Data with the delivery of SQL Server 2012. As the foundation to the cloud-ready information platform, SQL Server 2012 is helping organizations unlock breakthrough insights across the organization, including Big Data, as well as helping build and extend solutions from server to private or public cloud backed by enterprise-ready capabilities for mission critical confidence.