Today, 80% of
organizations
adopt cloud-first
strategies
AI investment
increased by
300% in 2017
Data will grow to
44 ZB in 2020
(1 ZB = 1 trillion GB)
Cloud AIData
Let’s start by talking about how the world is changing
We all aspire to create disruption that
constructs new realities for customers and
builds a distinct advantage for our organizations.
In order to make that leap, we have to look across trends and decide:
which variables and trends we need to care about
which ones will prevail
which ones we invest in
We’ve identified 3 major trends we believe will heavily shape and shoulder disruption in the future
The 1st is the astronomical explosion of data
By 2020, data will reach 44ZB, quadruple what it is today
The 2nd trend is cloud adoption
4 out of every 5 companies invest in public cloud technologies
The 3rd major trend is artificial intelligence
AI gives life to all that data we’re creating
This year alone, investment in AI has increased 300%
You’re probably not surprised that we’ve identified data, cloud and AI as the seeds for growing transformation.
However, this isn’t a simple list of things you go out and buy
The key to true business transformation – or the magic behind it – doesn’t come from any one of these on their own. It comes from the dynamic and fluid intersection
You might be thinking enough of the theory, so let’s look at the facts.
Organizations that harness data, cloud, and AI OUT PERFORM
<click>and we’re not the ones saying this keystone research results that companies who invest out outperforming. Companies need to start using their data.
Through a Keystone research study we learned companies in the top quartile for “investing in their data platform” vastly outperformed companies in the bottom quartile
With double the operating margin, they are crushing the competition
Investments ALONE don’t make you money and they don’t give you a competitive edge.
Investments enable innovation. This leads us to our second point that….
What prevents companies from capitalizing on that data growth?
Data exists in silos: There are all sorts of data coming in, stored across databases that power the applications or lifeblood of a company, data from sensors and devices, external data sources from media, and more. But harnessing that data is difficult and costly. Furthermore, connecting that data in ways that drives deeper insights poses even greater challenges.
Incongruent data types: Different applications , different data formats. There is tremendous complexity in bringing these formats together in a way where companies can leverage varietal data sources to derive insights that are richer than the single source.
Performance constraints: On-premises systems reach capacity. Customers are having to make strategic decisions about how to scale and how to do it cost effectively.
Complexity of solutions: Over time, companies have adopted a variety of products or solutions to solve specific business needs. They now find that they have a diverse set of tools, on premise and in the cloud, all with their unique tool sets and cost models, and skill sets requires to use them. It is hard to skill up and scale these offerings for the whole company to take advantage of, and costing is no longer transparent…the invoice has many line items to manage.
This comes to rising costs. Scaling up and adding hardware, paying for multiple toolsets that have some overlapping capabilities but lacking the scale, or ubiquity to do more than a purpose driven task well, and hiring resources with the specific skills needed adds up. And suddenly, companies looking to get ahead and maximize ROI with data are finding it challenging to harness it for transformation.
To derive real value from your data, msft provides a comprehensive platform with familiar tools and a robust ecosystem of partners and ISVS to deliver the solutions you need.
One hub for all data: This is one of the most significant pain points we hear from customers. How can I get access to all of the data internally and externally to my organization. Each system needs its own database, that is true. Microsoft’s analytics solution is able to connect to those data sources and bring that data into a cloud scale data warehouse solution for deriving insights, from a variety of data sources.
Diverse data types: This data can be in various formats, structured, and unstructured. It doesn’t matter. Microsoft’s solution can process the data into formats that can be leveraged in new and exciting ways.
Unlimited data scale: Not only can Microsoft’s solution process these diverse types of data, it can do it at scale, without constraints typically associated with existing on premises solutions.
Familiar tools and ecosystem: And, you don’t have to hire a number of specialists to manage a niche tool. Microsoft’s offerings provide familiar tools and ecosystem to help you leverage your investment quickly.
Lower TCO: The benefits noted all accrue to a lower TCO.
Only Microsoft delivers a comprehensive cloud scale analytics solution.
First, you need to be able to ingest data where ever that data is, whatever type of structured, unstructured, streaming data.
Once you’ve ingested this data you need a cloud scale storage solution that can handle all data types at scale and cost effectively.
Once stored data, you want to be able to prepare and process your data, to transform unstructured data and serve it with your structured data into the data warehouse.
Finally, you can leverage that data in a number of ways. Deliver it to the organization with rich visualizations for improved BI and reporting, deliver more advanced analytics such as personalized customer experiences, and access real-time data all at your fingertips for better business outcomes (for example, predictive maintenance).
In this example, we’ve integrated data into a single hub to provide intelligent insights with rich BI, Advanced Analytics, and Real Time analytics to drive the business forward.
Productive: quickly build solutions and focus on biz value:
all the tools and services you need on a single platform, reducing complexity. (familiar tools)
Familiar tools to quickly go to market: example
Provision your data warehouse & spark environment in minutes with a single click
Accelerate your data integration with consistent 70+ native data connectors
Empower your data scientists, data engineers, and business analysts to use the tools and languages of their choice
Hybrid: meet you where you are: cloud is a journey – customers environments on premise and cloud, evolve with your needs.
How we are Hybrid:
Leverage SQL Server’s proven performance and security consistently, in public or private cloud
Reduce cost & complexity of managing your existing data transformations by consistently running SQL Server Integration Service packages in Azure
Enable consistent user experience with common identity across on-premises and Azure
Intelligent: Tools that drive intelligence not only in the cloud but at the edge. You can move your analytics and machine learning closer to your data.
Need more detail
Our customers understand the value the of intelligent cloud and are using it to connect their businesses and customers globally. For certain scenarios, they are also recognizing the need to process and analyze data close to where the data originates. This is the power of intelligent edge. Customers are increasingly packaging applications in containers and deploying them onto their devices themselves, closer to where the data is created. This enables organizations to build machine learning and AI solutions that not only scale out with their needs, but also provide immediate inferences from intelligence built on the edge.
Trusted:
Local availability enables you to be compliant where your data resides. This allows mission-critical work of crucial organizations to take advantage of local availability. This addresses the requirements of governments and national infrastructure, including banks, utilities, transport and telecommunications. We want to ensure that we build our cloud infrastructure to serve the needs of customers by driving innovation and making it accessible globally.
50+ industry and geographical compliances
Azure Active Directory: role based admin control on your data access.
SQL Server & GDPR Compliance
Rich partner network to accelerate time to value.
A modern data warehouse lets you bring together all your data at any scale easily, and to get insights through analytical dashboards, operational reports, or advanced analytics for all your users.
1 Combine all your structured, unstructured and semi-structured data (logs, files, and media) using Azure Data Factory to Azure Blob Storage.
2 Leverage data in Azure Blob Storage to perform scalable analytics with Azure Databricks and achieve cleansed and transformed data.
3 Cleansed and transformed data can be moved to Azure SQL Data Warehouse to combine with existing structured data, creating one hub for all your data. Leverage native connectors between Azure Databricks and Azure SQL Data Warehouse to access and move data at scale.
4 Build operational reports and analytical dashboards on top of Azure Data Warehouse to derive insights from the data, and use Azure Analysis Services to serve thousands of end users.
5 Run ad hoc queries directly on data within Azure Databricks.
Transform your data into actionable insights using the best-in-class machine learning tools. This architecture allows you to combine any data at any scale, and to build and deploy custom machine learning models at scale.
1 Bring together all your structured, unstructured and semi-structured data (logs, files, and media) using Azure Data Factory to Azure Blob Storage.
2 Use Azure Databricks to clean and transform the structureless datasets and combine them with structured data from operational databases or data warehouses.
3 Use scalable machine learning/deep learning techniques, to derive deeper insights from this data using Python, R or Scala, with inbuilt notebook experiences in Azure Databricks.
4 Leverage native connectors between Azure Databricks and Azure SQL Data Warehouse to access and move data at scale.
5 Power users take advantage of the inbuilt capabilities of Azure Databricks to perform root cause determination and raw data analysis.
6Run ad hoc queries directly on data within Azure Databricks.
7 Take the insights from Azure Databricks to Cosmos DB to make them accessible through web and mobile apps.
Get insights from live streaming data with ease. Capture data continuously from any IoT device, or logs from website clickstreams, and process it in near-real time.
1 Easily ingest live streaming data for an application using Apache Kafka cluster in Azure HDInsight.
2 Bring together all your structured data using Azure Data Factory to Azure Blob Storage.
3 Take advantage of Azure Databricks to clean, transform, and analyze the streaming data, and combine it with structured data from operational databases or data warehouses.
4Use scalable machine learning/deep learning techniques, to derive deeper insights from this data using Python, R or Scala, with inbuilt notebook experiences in Azure Databricks.
5 Leverage native connectors between Azure Databricks and Azure SQL Data Warehouse to access and move data at scale.
6 Build analytical dashboards and embedded reports on top of Azure Data Warehouse to share insights within your organization and use Azure Analysis Services to serve this data to thousands of users.
7 Power users take advantage of the inbuilt capabilities of Azure Databricks and Azure HDInsight to perform root cause determination and raw data analysis.
8 Take the insights from Azure Databricks to Cosmos DB to make them accessible through real time apps.
Unstructured data / real time streaming from web logs, weather, twitter trends, events.
Structured data (historical) from sales, marketing , company.
1 Easily ingest live streaming data for an application using Apache Kafka cluster in Azure HDInsight.
2 Bring together all your structured data using Azure Data Factory to Azure Blob Storage.
3 Take advantage of Azure Databricks to clean, transform, and analyze the streaming data, and combine it with structured data from operational databases or data warehouses.
4Use scalable machine learning/deep learning techniques, to derive deeper insights from this data using Python, R or Scala, with inbuilt notebook experiences in Azure Databricks.
5 Leverage native connectors between Azure Databricks and Azure SQL Data Warehouse to access and move data at scale.
6 Build analytical dashboards and embedded reports on top of Azure Data Warehouse to share insights within your organization and use Azure Analysis Services to serve this data to thousands of users.
7 Power users take advantage of the inbuilt capabilities of Azure Databricks and Azure HDInsight to perform root cause determination and raw data analysis.
8 Take the insights from Azure Databricks to Cosmos DB to make them accessible through real time apps.
Demo scenario:
Showcase how this all comes together in a real world scenario through a mobile application. We’ll talk about how the consumer is offered a highly personalized and engaging experience. This experience was made possible through the rich insights the marketing manager was able to act upon within his/her campaign. And finally, how all of this is made possible on Azure with the underlying solution delivered by the solution architect.
As a consumer: I open my application and I’m trying to buy shoes. Currently I see a generic experience (all shoes).
I log into the application…
I sign in as As Sunil
As soon as I sign in the application ensures I have security enabled through my sign in experience via pin code or fingerprint. This ensures your sign in experience as a consumer is easier. (What is the feature?)
Now this experience is more personalized to me…it has learned from my history that I prefer brighh colors like orange and green…so now I am more engaged with this application. I go ahead and click on the green shoe.
I love this shoe, I’ve probably visited this page a number of times. But the price tag is a bit high for me…and I’m still not over the edge to want to purchase this right away.
So I scroll down and look for more reviews on how people have reacted to this shoe and after seeing a 4 star rating, I’m a little bit more convinced.
But I still don’t know the price tag is a bit out of my budget.
But while I’m on this page…
A promotion pops up that is so relevant to me and what I have been browsing…I’ve been provided a discount that if I purchase this right now I’ll get a 20% off…and not only that, if I decide to pick it up at the store I get an additional 15% discount. This is an offer I cannot refuse. So I select the option to pick it up at the store.
I add this to the cart after selecting the discount and now it’s only $64. I will now check out.
Since we signed up for improved security, the app is able to make my experience so much easier by not having to reenter my details every time I check out. Instead I just use my fingerprint to auto update and I’m ready to check out.
When I check out it says my fingerprint is a match so I make the purchase
I’m provided a scan code that I need to take to the store to check out with.
When I go to the store, the store manager, is able to upsell to me because of the data they already have about my purchasing behavior and likes. For instance, when I’m at the store to pick up the shoes, the store manager can show me some shoelaces or tennis shorts I might be interested in to buy along with my new shoes.
Now le’s go through the experience of the online marketing manager for this shoe store. As a marketing manager I want to have as much data as possible about how customers are engaging with my product and the applications so that I am able to run the right campaigns and engage with my customers intelligently. Here’s a dashboard…
Here’s a dashboard that allows me to view my business end to end.
It provides me with sentiment by product category so I can better understand how my consumers are reacting to the different lines of business that I own, like mens shoes, sports accessories, women’s clothing, etc…all in one place.
Now I am able to see what consumers are clicking on so I can understand my most popular categories.
Not only that, I can see how consumers feel about my product portfolio by looking at what they are tweeting about and what is popular across the globe. I can use this data for product development as well as how I run my customized campaigns.
I can also right from my seat see the revenue that I have across the globe to understand where I need to run my campaigns. But I want to be able to see how my product categories are performing in terms of average revenue...
So I click on this dashboard
These are the different categories of my products and I see that this grey dot is performing well. Let’s see what it is.
This is mens casual shoes. Looks like the campaign I just ran for sunil and other customers is working well. But I see the red dot is not performing as well. Let’s look at what that category is.
Mens clothing. I’m wondering if I need to run a campaign. But I don’t think I have enough information to be able to run a campaign right away. So now I want to go into my campaigns dashboard.
This is a very handy tool that I have where from one single place I am able to see and analyze my campaign data and results from across the globe.
I can see my revenue cost and return on investment of the various campaigns I have run on the product categories.
I can also see up top my aggregated data of all of my global campaigns and how they are performing against my target.
I can also see the various types of channels that I am running campaigns through and the ROI on those channels. For instance, through emails, through social media, through events, etc…
But I am most interested in the intelligence built into this recommended campaign dashboard. Let’s zoom into this dashboard
Based on the data of the campaigns that I am running globally, combined with the data about how my consumers are engaging with the different product categories, how the popularity of these categories has been evolving, I have a recommended campaign dashboard all based on the rich data my solution is built upon. Here I see the ROI that I will expect if I run certain campaigns. My solution intelligently recommends that if I ran a campaign on mens clothing I will get a ROI of 82%. Not only that, it tells my what type of campaign I should be running to maximize this ROI, which is a 10% discount campaign. I think I have enough information to run this campaign. While I am sitting here I launch this campaign.
It ask me to confirm the campaign launch
And I click on OK.
Now le’s go how all of this magic was created in the back end.
Unstructured data / real time streaming from web logs, weather, twitter trends, events.
Structured data (historical) from sales, marketing , company.
1 Easily ingest live streaming data for an application using Apache Kafka cluster in Azure HDInsight.
2 Bring together all your structured data using Azure Data Factory to Azure Blob Storage.
3 Take advantage of Azure Databricks to clean, transform, and analyze the streaming data, and combine it with structured data from operational databases or data warehouses.
4Use scalable machine learning/deep learning techniques, to derive deeper insights from this data using Python, R or Scala, with inbuilt notebook experiences in Azure Databricks.
5 Leverage native connectors between Azure Databricks and Azure SQL Data Warehouse to access and move data at scale.
6 Build analytical dashboards and embedded reports on top of Azure Data Warehouse to share insights within your organization and use Azure Analysis Services to serve this data to thousands of users.
7 Power users take advantage of the inbuilt capabilities of Azure Databricks and Azure HDInsight to perform root cause determination and raw data analysis.
8 Take the insights from Azure Databricks to Cosmos DB to make them accessible through real time apps.
This is just one example of how Business Analytics and AI can help transform the business process for an organization. We have customers across every industry with similar stories for transformation:
- For example, Rockwell, a global industrial automation company, who cut develop time by 80% and costs up to 90% by providing full time pipeline visibility,
- or, QuarterSpot, a start up in the financial sector, who lowered loan defaults by 50%, leading to increased profitability