Anzeige
Anzeige

Más contenido relacionado

Presentaciones para ti(20)

Similar a Analytics in a Day Ft. Synapse Virtual Workshop(20)

Anzeige

Último(20)

Anzeige

Analytics in a Day Ft. Synapse Virtual Workshop

  1. CCG: Upcoming Workshops Introduction to Machine Learning Workshop | March 23rd | 9:00 AM – 11 AM EST • Designed to provide you with an overview of machine learning concepts, real world applications, and some user-friendly tools. Data Modernization in a Day | March 30th | 9:00 AM – 12:00 PM EST • This workshop will cover everything from whiteboarding migration strategies to hands-on experiences with data migration tools. Analytics in a Day Ft. Synapse Workshop | April 20th | 9:00 AM – 1:00 PM EST • Learn how to simplify and accelerate your journey towards the modern data warehouse. Data Governance Workshop with CCG+Profisee | May 4th | 9:00 AM – 12:00 PM EST • Learn how leveraging an MDM strategy within the context of Data Governance drives organizational alignment, ensures data quality, and accelerates Digital Transformation. Read more and register at ccganalytics.com/events Follow us on LinkedIn @CCGAnalytics to stay up to date on events
  2. Analytics in a day Cloud analytics in the age of self-service and data science
  3. Agenda Workshop Instruction & Discussion 9:00 – 9:30 Harness the power of analytics 9:30 – 9:45 Power BI And Azure Synapse: Unmatched Break for 15 min and Set Up Lab Hands-on lab 10:00 – 1:00 Hands-on labs using Azure Synapse and Power BI Times are approximate and will be fluid with the class
  4. Housekeeping Please message Sami with any questions, concerns or if you need assistance during this workshop. Please mute your line! We will be applying mute. This session will be recorded. If you do not want to be recorded, please disconnect at this time. Links: See chat window. Worksheet: See handouts. To make presentation larger, draw the bottom half of screen ‘up’.
  5. James McAuliffe, Cloud Solution Architect James McAuliffe is a Cloud Solution Architect with over 20 years of technology industry experience. During this journey into data and analytics, he’s held all of the traditional Business Intelligence Solution project roles, ranging from design and development to complete life cycle BI implementations. He is a Microsoft Preferred Partner Solutions expert and has worked with clients of all sizes, from local businesses to Fortune 500 companies. And I like old Italian cars. linkedin.com/in/jamesmcauliffesql/
  6. My Spare Time
  7. A premier Microsoft partner, CCG uses leading cloud platforms to develop solutions and provide analytics that help customers advance their digital strategies. 7 PARTNERSHIP SPOTLIGHT: MICROSOFT Certifications Gold Partner Independent System Vendor (ISV) and Co-Seller AI Inner Circle Partner Technologies Azure Data Services Azure Data Factory Azure Data Lake Store Azure Databricks Azure Cognitive Services Azure Machine Learning Azure Stream Analytics Azure Analysis Services Azure Synapse Analytics Power BI Platform
  8. Offerings Overview 8 Data and Analytics Strategy Advanced Analytics, Machine Learning, and AI Data Management and Data Governance Enterprise Business Intelligence Cloud Strategy, Migration, And Management
  9. Harness the power of analytics Section 1 All businesses are data businesses Section 2 The cloud for modern analytics Section 3 The paradox of analytics Section 4 The analytics continuum
  10. Section 1 All businesses are data businesses
  11. Today, all businesses are data businesses Data is the lifeblood of modern work
  12. All data businesses need to be analytic businesses Without analytics data is a cost center, not a resource
  13. Analytic businesses need to evolve data science Every business has opportunities to make analytics faster, easier, and more insightful
  14. Section 2 The cloud for modern analytics
  15. Modern businesses succeed in the cloud The cloud is the default environment for new technology initiatives
  16. Price, performance, and agility A cloud analytics platform is an economic breakthrough
  17. Structured, unstructured, and streaming data integrated in a single, scalable, environment A cloud analytics platform is the hub for all data models
  18. Section 3 The paradox of analytics
  19. Data Landscape – Volume and Pressure IDC Data Age 2025 - The Digitization of the World
  20. While data grows 400%, less than 30% gets analyzed 2025 2020 44ZB 175ZB
  21. * Harvard Business Review (2019), Understanding why analytics strategies fall short for some, but not for others Analytics & AI is the #1 investment for business leaders, however they struggle to maximize ROI 80% 55% ?
  22. More analytics solutions lead to more silos Data Technologies Skills
  23. Each new technology creates another siloed operation Big data Data integration Machine learning Business intelligence Data governance Security paradox
  24. Section 4 The analytics continuum
  25. Real-time BI Data integration Artificial intelligence Machine learning Data exploration Data warehousing
  26. Data exploration Real-time BI Data integration Artificial intelligence Machine learning Data warehousing
  27. ©Microsoft Corporation Azure The importance of data models BI models Power BI  Built and maintained by business users or BI developers  Use enterprise models, departmental data, and external sources  Focused on a single subject area, but often widely shared Machine learning models Azure Synapse Analytics  Built and maintained by data scientists  Mostly developed from raw sources in the data lake  Often experimental, needing a data engineer for production use Azure Synapse Analytics Enterprise models  Built and maintained by IT architects  Consolidated data from many systems  Centralized as an authoritative source for reporting and analysis
  28. ©Microsoft Corporation Azure If business users are tech-smart and data literate, why do they need enterprise models? Enterprise models in the self-service environment Consistency Some business processes can be built once and shared as a corporate standard Governance Certain data sets need complex security and privacy controls Efficiency No need to repeat design, preparing, and loading or securing Line-of-business sources Data ingestion & transformation Enterprise models Azure Synapse Analytics Power BI
  29. ©Microsoft Corporation Azure If enterprise models are so important, why do users need self- service BI models? BI models in the enterprise environment Flexibility Some data sets are temporary, external, or ad-hoc don’t need to be consolidated Efficiency Tech-smart business users have fresh and innovative ideas they need to explore with agility Ad-hoc, departmental and external sources Line-of-business sources Data ingestion & transformation Power BI Enterprise models Azure Synapse Analytics BI models
  30. ©Microsoft Corporation Azure What is the role of the data warehouse with data science? Data science models in the enterprise environment Integrating results with enterprise models Making the results of data science easily available for business functions Serving enterprise data for data scientists Helps ensure consistency across diverse analyses Power BI Azure Synapse Analytics Azure Databricks Enterprise models Azure Synapse Analytics Data science results
  31. Azure Synapse Analytics Limitless analytics service with unmatched time to insight
  32. The first unified, cloud native platform for converged analytics Azure Synapse is the only unified platform for analytics, blending big data, data warehousing, and data integration into a single cloud native service for end-to-end analytics at cloud scale.
  33. Powered by a new cloud native distributed SQL engine
  34. Flexible consumption models Serverless pay-per-query ideal for ad-hoc data lake exploration and transformation Dedicated clusters optimized mission-critical data warehouse workloads Serverless Dedicated
  35. ➢ Comprehensive security and compliance ➢ Streamlined data integration ➢ Flexible data warehousing ➢ Real-time operational analytics ➢ Integrated machine learning ➢ Power BI + Azure Synapse
  36. Real-time operational analytics Eliminate latency and accelerate decision making
  37. Integrating operational data with analytics systems
  38. Integrating operational data with analytics systems Azure Synapse Link
  39. Integrating operational data with analytics systems
  40. Enable real-time data analytics
  41. Real-time operational analytics One-click enablement in Azure Portal No data integration pipelines required Near-zero impact on operational systems Latency <90s at 99th percentile Azure Cosmos DB Analytical Store Column store optimized for analytical queries Transactional Store Row store optimized for transactional operations Azure Synapse Analytics Cloud-Native HTAP Azure Synapse Link
  42. Event Hubs IoT Hub T-SQL language Built-in streaming ingestion & analytics Streaming Ingestion Dedicated SQL Pool Synapse SQL IoT ingestion without aggregation Ingress up to 720 gigabytes/hour of raw events Analyze data in-flight using SQL language Azure Stream Analytics Join streaming data with other data assets in the data warehouse and data lake
  43. Power BI + Azure Synapse An unmatched combination
  44. Where do you find yourself on the curve? Hindsight Insight Foresight Value Difficulty What happened? Descriptive Analysis Why did it happen? Diagnostic Analysis What will happen? Predictive Analysis How can we make it happen? Prescriptive Analysis
  45. Where do you find yourself on the curve? Hindsight Insight Foresight Value Difficulty What happened? Descriptive Analysis Why did it happen? Diagnostic Analysis What will happen? Predictive Analysis How can we make it happen? Prescriptive Analysis BI
  46. BI + Analytics unlock the door to AI, machine learning, and real-time insights Hindsight Insight Foresight Value Difficulty What happened? Descriptive Analysis Why did it happen? Diagnostic Analysis What will happen? Predictive Analysis How can we make it happen? Prescriptive Analysis Analytics BI
  47. Unified experience to enrich data Automated ML for rapid development Seamless collaboration
  48. Build dashboard in Synapse Studio Code-free experience for development rich visualizations One-click publishing to for secure consumption across the enterprise
  49. Visualize and report Power BI Model & serve Azure Synapse Analytics CDM Folders Azure Data Lake Storage Respond instantly Enable instant response times with Power BI Aggregations on massive datasets when querying at the aggregated level Get granular with your data Queries at the granular level are sent to Azure Synapse Analytics with DirectQuery leveraging its industry-leading performance Save money with industry-leading performance Azure Synapse Analytics is up to 14x faster and 94% cheaper than other cloud providers View reports with a single pane of glass Skip the configuration when connecting to Power BI with integrated Power BI-authoring directly in the Azure Synapse Studio Accelerate business value with a powerful analytics platform
  50. Customers using Azure Synapse & Power BI today are transforming their business with purpose 27% Faster time to insights 271% Average ROI 26% Lower total cost of ownership 60% Increased customer satisfaction * Forrester, October 2019, “The Total Economic Impact of Microsoft Azure Analytics with Power BI”
  51. Why Power BI data models? • Power BI models are the way to effectively connect decision makers with the data warehouse • Semantic layer, providing intuitive browsing and exploration • Deliver high performance (with the assistance of cached aggregations) • Encapsulate complex calculation logic • Source for rich interactive reports, and dashboards © 2021 Microsoft. All rights reserved.
  52. Why Power BI data models? • Power BI data models can scale to query large data stores, such as a data warehouse • Typically, these models: • Are designed and developed by BI developers • Represent large data volumes • Support high report user concurrency • Consideration must be given to the Power BI capacity that will host the model © 2021 Microsoft. All rights reserved. Capacities are introduced later in this section
  53. Why Power BI data models? Architecture Reports Dashboards Azure Synapse Analytics © 2021 Microsoft. All rights reserved. Power BI service
  54. Power BI data modeling Model basics • Power BI is built upon the mature foundations of Analysis Services • Specifically, it is based on tabular modeling technologies • Models can source data from practically any data source • Cloud or on-premises • File, database, web page, or SaaS provider • When published to Power BI, a model is known as a dataset CRM © 2021 Microsoft. All rights reserved.
  55. Power BI data modeling Model basics • Models have tables, and tables have columns • Table are related for purposes of filter propagation • Often, dimensional modeling (star schema) is the optimal design approach for Power BI models • Dimension tables define business entities • Fact tables define business activity or observations • A star schema can comprise multiple stars, each based on a single fact table © 2021 Microsoft. All rights reserved.
  56. Power BI data modeling Model basics » Star schema * 1 Fact Dim Dim Dim Dim Dim © 2021 Microsoft. All rights reserved.
  57. Power BI data modeling Model basics » Star schema © 2021 Microsoft. All rights reserved.
  58. Power BI data modeling Model basics • Well designed tables and relationships are the model foundations • Power Query is the technology used to create model tables • Upon the foundations, modelers can: • Enhance column properties with default summarization, data categorization, sorting, or formatting • Add hierarchies to support drill down/up • Add calculations to create new tables, columns, or measures, which summarize model data • Organize model fields into display folders • Add roles to enforce row-level security (RLS) for different report user audiences © 2021 Microsoft. All rights reserved.
  59. Power BI data modeling Model basics » From source to visualization Source Azure Synapse Analytics Model Modeler view Report author view Visualization © 2021 Microsoft. All rights reserved.
  60. Power BI data modeling Storage modes • Each model table has a storage mode: • Import (Vertipaq) • DirectQuery • Dual—both Import and DirectQuery • Possible model frameworks: © 2021 Microsoft. All rights reserved. Table storage Model framework All tables are Import storage mode Import All tables are DirectQuery storage mode DirectQuery Model has Mixed storage mode Composite
  61. Power BI data modeling Model framework » Import • Import storage mode caches table data in compressed and optimized column stores • Generally, it provides the fastest query performance • But—requires periodic data refresh to keep data current © 2021 Microsoft. All rights reserved.
  62. Power BI data modeling Model framework » Import © 2021 Microsoft. All rights reserved.
  63. Power BI data modeling Model framework » Import • Develop an Import model when: • The entire model fits into available memory • Data latency between data refreshes can be tolerated • Power Query transformations are complex • Model calculations are complex • Because data warehouses represent large volumes of data, usually an Import model can only be considered when: • Caching high level summarizations • Caching only recent history © 2021 Microsoft. All rights reserved.
  64. Power BI data modeling Model framework » DirectQuery © 2021 Microsoft. All rights reserved.
  65. Power BI data modeling Model framework » DirectQuery • DirectQuery storage mode is an alternative to Import storage mode • When queried, a DirectQuery table uses native queries to retrieve data from an underlying data source • Generally, it allows: • Creating tables over large data volumes • Achieving near real-time reporting © 2021 Microsoft. All rights reserved.
  66. Power BI data modeling Model framework » DirectQuery • Develop a DirectQuery model when: • Data volume cannot fit into memory • Data latency must be low • Power Query transformations are simple • Model calculations are simple • This model mode is well-suited for data warehouses • The model can query all data • Reports display the latest data © 2021 Microsoft. All rights reserved.
  67. Understanding insights in the consumer goods market with Azure Synapse Analytics Challenge Solution
  68. Business intelligence Data integration Data management Machine learning and AI
  69. Accelerate time to solution Azure Open Data sets Pre-built samples to accelerate development • SQL Scripts • Notebooks • Data Pipelines
  70. Break
  71. Hands-on lab Build an end-to-end analytics solution with Azure Synapse & Power BI
  72. ©Microsoft Corporation Azure Lab Contents Overview
  73. Exercises Getting setup You will develop a Power BI model that connects to the Azure Synapse Analytics data warehouse You must use the lab Azure credentials to connect to Azure Synapse and publish to Power BI © 2021 Microsoft. All rights reserved.
  74. Power BI Exercises Exercise 05: Develop a Power BI Model 45 minutes Exercise 06: Optimize the Power BI Model 15 minutes Exercise 07: Create a Model Using DQ to Power BI 20 minutes Exercise 08: Author a Power BI Report (Optional) 45 minutes © 2021 Microsoft. All rights reserved.
  75. Azure Credits* for AIAD attendees Continue your learning after the workshop Starting Oct 18th, 2020, Analytics in a Day workshop (in-person or virtual) attendees get access to another Azure lab environment(Different from the lab environment used during workshop) to continue their learning and exploration for up to 6 hours after the workshop. 1.Attendees will get an invite email from noreply@cloudlabs.ai to activate the lab environment to explore further. Once attendee activate the lab environment, it’ll be valid for 6 hours duration and auto-delete after that. 2.When attendee click on launch lab from invite email to activate the lab environment, deployment could take around 45 minutes to get ready. 3.In case of any issue, attendees can reach out to CloudLabs Support team at cloudlabs- support@spektrasystems.com . *This is a limited time offer, subject to available funding. It’s available only to Analytics-in-a-Day (AID) attendees. Analytics Immersion Workshop (AIW) attendees don’t have access to this offer. Azure Environment Duration: 6 hours per pax. Timer will start after attendee activate the lab environment. Lab environment Activation Validity: 7 days from the date of workshop. Once activated, Lab environment will be valid for 6 hours duration and auto-delete after that.
  76. ©Microsoft Corporation Azure Lab Contents Overview
  77. ©Microsoft Corporation Azure Lab Contents Overview
  78. Data integration Code-free hybrid data integration
  79. Cloud native ETL/ELT 95+ connectors available Secure connectivity to on-premise data sources, other clouds, and SaaS applications Code-first and low/no code design interfaces Schedule and Event based triggering Code-free
  80. No/low-code data transformation Excel-like interface is familiar to users Transform data to desired shape completely visually Operationalize into pipelines Code-free data wrangling
  81. Real-time operational analytics No data integration pipelines required Near-zero impact on operational systems Latency <90s at 99th percentile
  82. Power BI Exercises Exercise 05: Develop a Power BI Model 45 minutes Exercise 06: Optimize the Power BI Model 15 minutes Exercise 07: Create a Model Using DQ to Power BI 20 minutes Exercise 08: Author a Power BI Report (Optional) 45 minutes © 2021 Microsoft. All rights reserved.
  83. Section 02 Deliver Discipline at the Core © 2021 Microsoft. All rights reserved. Analytics in a Day Azure Synapse + Power BI better together
  84. Section outline 02: Deliver Discipline at the Core • Why Power BI data models? • Power BI data modeling • Developing models © 2021 Microsoft. All rights reserved.
  85. Exercise 06 45 minutes Develop a Power BI Model You must use the lab Azure credentials to connect to Azure Synapse and publish to Power BI • Section 1: Create the Model • Section 2: Develop the Model • Section 3: Test the Model © 2021 Microsoft. All rights reserved.
  86. Get started today Create a free Azure account and get started with Azure Synapse Analytics: https://azure.microsoft.com/en-us/free/synapse-analytics/ Submit info for free proof of value package for modernizing SAP workloads: https://aka.ms/synapse-qlik Submit info for free proof of value package for migrating on-prem data warehouse: http://aka.ms/synapse-informatica Learn more: https://aka.ms/synapse
  87. Use the limited time offer to save up to 76 percent when migrating to Azure Synapse https://aka.ms/synapse-migration-offer
  88. Analytics in a Day Thank You! James McAuliffe jmcauliffe@ccganalytics.com https://www.linkedin.com/in/jamesmcauliffesql/ https://ccganalytics.com/
  89. © Copyright Microsoft Corporation. All rights reserved.
  90. Security and compliance Unified governance controls
  91. Category Feature Data Protection Data in transit Data encryption at rest Data discovery and classification Access Control Object level security (tables/views) Row level security Column level security Dynamic data masking Column level encryption Authentication SQL login Azure active directory Multi-factor authentication Network Security Managed virtual network Custom virtual network Firewall Azure ExpressRoute Azure Private Link Threat protection Threat detection Auditing Vulnerability assessment Isolation Dedicated metadata store Hosted in customer tenant Best-in-class security Customer & System Managed Keys All data encrypted by default Up to 3x levels of data encryption at rest Democratize data at scale with fine-grained ACL Proactive protection Comprehensive Compliance
  92. Eliminate network maintenance One-click enables automated management of virtual networks between cluster endpoints Synapse resources only ever interop with private endpoints No management of subnets or IP Ranges Prevents data exfiltration Compliance boundary
  93. More than just data security Native integration with Azure Purview Automatically discover and classify data assets End-to-end data lineage
  94. Data warehouse Scalable and secure analytics platform for SQL workloads
  95. Fully-managed elastic platform Elastic compute that can be easily optimized to different classes of workload All features available in a single tier Infinite cost effective PAYG storage
  96. SQL Editor Automatic code completion (Intellisense) Script collaboration within the Workspace Built-in visualizations Easily switch between clusters
  97. TPC-H and TPC-DS Leader Price/performance leadership relative to other cloud data warehouses “Polaris” is the only query engine to successfully complete TPC-H at 1PB scale https://aka.ms/synapse-dqp 11.5 7 62 9.5 28 2.5 30.5 48.5 99 22.5 9 5 11.5 6.5 2 5 27 99.5 18.5 21 95.5 8 Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18 Q19 Q20 Q21 Q22 TPC-H 1 petabyte execution times $47 $152 $564 $51 DW30000C 4X-Large BigQuery dc2.8xlarge 60N Test-H 30TB Price/performance @30TB ($ per query per hour) lower is better $153 $286 $309 $570 Azure Synapse Redshift Snowflake Enterprise BigQuery Flat Rate Test-DS 30TB Price/performance comparison (lower is better) * “GigaOm Analytics Field Test-H Benchmark Report” January 2019; “GigaOm Analytics Field Test-DS Benchmark Report” April 2019
  98. Only platform to complete TPC-H benchmark at 1 Petabyte Massive concurrency Global workload graph Workload aware query scheduling https://aka.ms/synapse-dqp
  99. Workload management Azure Synapse supports a more diverse set of workload management tools through workload importance, intra-cluster isolation, and elastic clusters. Scale in Scale out Workload Importance Workload Isolation Workload Group B 40% Elastic Cluster (Scale Up) 2000 cDWU
  100. Chooses the most secure cloud DW, Azure Synapse Analytics, to transform two business critical Teradata systems Challenge Solution
  101. Machine learning Empower everyone with predictive insights
  102. Democratize predictive power Synapse makes predictive analytics accessible to all Notebooks provides a code authoring experience for complex predictive models Automatic ML graphical interface provides a no- code experience for creating ML models Native integration with Azure Cognitive Search provides access to pre-built models All Code Low/no-Code Pre-built models
  103. Code-first ML model development PySpark, Scala, and C# languages supported Automatic code completion (Intellisense) Author multiple languages in a single notebook Analyze data from the data warehouse, data lake, and real-time operational data from one place
  104. Data + Languages Languages such as SQL, PySpark, Scala and C# in support of data science and data warehouse workloads The data lake supports and unlimited set of file formats including Parquet, ORC and Json as well as audio, image, and video formats Language Data
  105. All you need is data Fully automated feature exploration
  106. Code-free in Synapse Studio No-code creation on Machine Learning models Democratize ML to everyone since no data science domain knowledge required Support for ensemble models Supports classification, regression, and time-series forecasting
  107. Code-free in Synapse Studio No-code references to machine learning models Democratize ML to everyone since no data science domain knowledge required Easily embed in SQL stored procedures for transformation of Views for reporting
  108. SELECT d.*, p.Score FROM PREDICT(MODEL = @onnx_model, … In-engine ML scoring Machine learning models executed using SQL “In-engine” for performance and scalability No data leaves the platform for scoring No additional cost for scoring T-SQL Language Synapse SQL Model Data Predictions
  109. “Azure data services have been seamlessly integrated into existing infrastructure, which was especially helpful with respect to authentication and user access management.” New ways for optimizing Snow production and operational costs with Azure Synapse Challenge Solution
  110. Supply Chain Intelligence and Finance Analytics Migrate and Modernize Data from SAP ECC Legacy DW appliance Each motion has a customized free proof of value package
  111. Azure Synapse + Power BI + Qlik Data Integration Free Proof of Value for SAP ERP Free Deep Dive Session Free ½ Day Solution Architecture Workshop Free Software Subscriptions Free Technical and Subject Matter Expertise Azure Synapse Expert Support Azure Synapse Expert Support Azure Synapse Analytics (data warehousing) + Power BI (for duration of POV) Azure Synapse Analytics and Power BI Expertise (for duration of POV) SAP Real-time Data Integration Expert Support 1:1 Deep-Dive with Qlik Data Integration Team Qlik Data Integration for Data Warehouse Automation (for duration of POV) Qlik Data Integration and SAP Expertise (for duration of POV)
  112. Get started today with your free proof of value from Microsoft and Qlik https://aka.ms/synapse-qlik
  113. Azure Synapse + Informatica Free Proof of Value Free Hands-On Migration Workshop TCO Assessment Free Data Catalog & ETL Migration Free Code Conversion Global Black Belt Support On-premises Data Warehouse Code Conversion Azure Synapse Analytics (data warehousing) Subscription (for duration of PoV up to 30 days) Cloud Data Warehouse Modernization Workshop on Azure Customized Migration TCO Assessment Enterprise Data Catalog (EDC) Intelligent Cloud Services (IICS) (for duration of PoV up to 30 days) Free Azure Synapse Subscription
  114. Get started today with your free proof of value from Microsoft and Informatica http://aka.ms/synapse-informatica
Anzeige