Diese Präsentation wurde erfolgreich gemeldet.
Die SlideShare-Präsentation wird heruntergeladen. ×

A "M"ind Bending Experience. Power Query for Power BI and Beyond.

Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Anzeige
Wird geladen in …3
×

Hier ansehen

1 von 30 Anzeige

A "M"ind Bending Experience. Power Query for Power BI and Beyond.

Herunterladen, um offline zu lesen

Power Query is quickly becoming the number one skill your employees NEED TO HAVE working in a modern workplace. Join Alex Powers as he demystifies the Power Query M Language, writing your own custom solutions and how to approach the language in a way that makes sense to both technical and non-technical users.

Power Query is quickly becoming the number one skill your employees NEED TO HAVE working in a modern workplace. Join Alex Powers as he demystifies the Power Query M Language, writing your own custom solutions and how to approach the language in a way that makes sense to both technical and non-technical users.

Anzeige
Anzeige

Weitere Verwandte Inhalte

Diashows für Sie (20)

Ähnlich wie A "M"ind Bending Experience. Power Query for Power BI and Beyond. (20)

Anzeige

Aktuellste (20)

A "M"ind Bending Experience. Power Query for Power BI and Beyond.

  1. 1. Power Query M? (Informally known as "M") The Power Query Formula Language is a powerful query language optimized for building queries that mashup data. It's a functional, case sensitive language. For computer language theorists: Power Query is a mostly pure, higher-order, dynamically typed, partially lazy, functional language. You can create a variety of data mashup queries from simple to advanced scenarios. (Source: MSDN)
  2. 2. Power Query ?! (Data Connectivity and Preparation) Power Query enables business users to seamlessly access data stored in hundreds of data sources and reshape it to fit their needs, with an easy to use, engaging and no-code user experience. Data Connectors: (Source: Office Support) Access Database Hadoop File (HDFS) Microsoft Azure Table Storage PostgreSQL Database Text Active Directory IBM DB2 Database Microsoft Exchange SharePoint List CSV Excel Microsoft Azure HDInsight MySQL Database SQL Server Database Web Page Facebook Microsoft Azure Marketplace OData Feed Sybase Database XML Folder Microsoft Azure SQL Database Oracle Database Teradata Database And More…
  3. 3. POWER QUERY FOR POWER BI AND BEYOND. A “M”ind Bending Experience.
  4. 4. ALEX M POWERS Microsoft Certified Solutions Associate: BI Reporting (Excel, Power BI) Microsoft Technology Associate: Python (Introduction to Programming) Microsoft Office Expert: Excel 2013, Excel 2010 Microsoft Office Specialist: Access 2013, Excel 2010 Microsoft #HowWeExcel Contest Winner Co-Organizer of the St Louis Power BI User Group #STLPBIUG Millennial Whisperer and Trainer | It’s Not About The Cell LLC: itsnotaboutthecell.com Estimates that 95% of Cloud Storage is Cat Photos Enjoys Pineapple On His Pizza
  5. 5. MAKE IT FEEL FAMILIAR. BEFORE IT FEELS FOREIGN. Hello, World!
  6. 6. PRIMITIVE VALUES Type Example binary #binary("AQID") date #date(2018,7,10) time #time(18,30,0) datetime #datetime(2018,7,10,18,30,0) datetimezone #datetimezone(2018,7,10,18,30,0,0,6) duration #duration(1,0,0,0) logical true and false null null number 0, 1, -1, 1.5 text “hello, world”
  7. 7. STRUCTURED VALUES Type Example list {“foo”, “bar”, 10} record [a = 10, b = 20] table #table({“Animal”}, {{“Cat”},{“Dog”}})
  8. 8. M SYNTAX
  9. 9. let Source = “Hello, World!” in Source Power Query M’s let expression stores a set of named values called variables. Each variable has a unique name called an identifier. In the above let expression – the variable Source stores the primitive text value “Hello, World!”. The final expression is evaluated in the identifier titled Source.
  10. 10. let Source = "Hello, World", #"Something Else" = "Keepin' It", Val = 100, #"Struct List" = {Source, #"Something Else", Val}, #"List – Index" = #"Struct List"{2}, // 0 Based Index #"Numbered List - 0 to 100" = {0..#"List – Index"}, #"Table With Meta" = Table.FromList(#"Numbered List - 0 to 100", Splitter.SplitByNothing(), type table [Index = Int64.Type], null, ExtraValues.Error), #"Each Loop with ASCII Number" = Table.AddColumn(#"Table With Meta", "ASCII", each Character.FromNumber([Index]), type text), #"ASCII Column and Row" = #"Each Loop with ASCII Number"[ASCII]{97}, #"Alphabetical List - a to z" = {#"ASCII Column and Row".."z"} in #"Alphabetical List - a to z" In the above let expression – the variable’s enclosed in #” “, denote the use of a special character (space) or number for the variables identifier. Starting from the beginning of the expression we begin with primitive values, introduce lists { }, positional index operator {2}, list sequences for numbers {0..100}, for each list item additional column transformations, lookup and positional index operators [Column1]{2} and the use of alphabetical list sequences {“a”..”z”}. Full Solution: Familiar Feeling
  11. 11. To view the available Power Query M functions use the formula =#shared to view as structured records or =Record.ToTable(#shared) to cast the records into a structured table format. =#shared
  12. 12. WE CAN REBUILD IT. WE HAVE THE TECHNOLOGY. BETTER THAN IT WAS BEFORE. BETTER. STRONGER. FASTER. THE SIX MILLION DOLLAR QUERY.
  13. 13. let Source = Excel.Workbook(Web.Contents("https://www.ers.usda.gov/webdocs/DataFiles/48747/Unemployment.xls"), null, true), #"Unemployment Med HH Inc_Sheet" = Source{[Name="Unemployment Med HH Inc"]}[Data], // Extract the previous identifier's Column1 as a list. Find the position of the text "FIPStxt", its first occurence while ignoring case sensitivity #"Find Position" = Table.Skip( #"Unemployment Med HH Inc_Sheet", List.PositionOf( #"Unemployment Med HH Inc_Sheet"[Column1], "FIPStxt", 1, Comparer.OrdinalIgnoreCase ) ), #"Promoted Headers" = Table.PromoteHeaders(#"Find Position", [PromoteAllScalars=true]), /* Clean each item in Column Names. The use of a underscore avoids the hardcoding of individual column headers and instead transforms each item in the collection We used a Replacer to ReplaceText of underscores to spaces in our headers, this function is wrapped with a Text.Proper to standardize all column headers */ #"Clean Headers" = Table.TransformColumnNames( #"Promoted Headers", each Text.Proper( Replacer.ReplaceText( _ , "_", " ") ) ), #"Unpivoted Other Columns" = Table.UnpivotOtherColumns(#"Clean Headers", {"Fipstxt", "State", "Area Name"}, "Attribute", "Value"), // Rename the auto generated Attribute.2 Column Header to Year. This step avoids and unnnecessary Table.RenameColumns function #"Split Column by Delimiter" = Table.SplitColumn(#"Unpivoted Other Columns", "Attribute", Splitter.SplitTextByEachDelimiter({" "}, QuoteStyle.Csv, true), {"Attribute.1", "Year"}), #"Pivoted Column" = Table.Pivot(#"Split Column by Delimiter", List.Distinct(#"Split Column by Delimiter"[Attribute.1]), "Attribute.1", "Value", List.Count), #"Changed Type" = Table.TransformColumnTypes(#"Pivoted Column",{{"Fipstxt", type text}, {"State", type text}, {"Area Name", type text}, {"Year", Int64.Type}, {"Civilian Labor Force", Int64.Type}, {"Employed", Int64.Type}, {"Unemployed", Int64.Type}, {"Unemployment Rate", Int64.Type}, {"Median Household Income", Int64.Type}, {"Med Hh Income Percent Of State Total", Int64.Type}, {"Rural Urban Continuum Code", Int64.Type}, {"Urban Influence Code", Int64.Type}, {"Metro", Int64.Type}}) in #"Changed Type" In the above let expression – we introduce future proofing the header row indicator of "FIPStxt“ in our dataset and ignoring case sensitivity. Transforming our column names with an each loop we utilize the underscore character to iterate thru each item in our column headers collection. Full Solution: Six Million Dollar Query
  14. 14. UP IS DOWN. THE UPSIDE DOWN. DOWNISUP.
  15. 15. let Source = #table( type table [Vendor=Text.Type, State=Text.Type, Sales=Number.Type, #"Total Amount"=Number.Type], { {"Contoso","MO",150,4013.116648}, … } ), #"Filter: non null" = Table.SelectRows(Source, each ([State] <> null)), // Vendor ASC, Sales DESC #"Sort Columns" = Table.Sort(#"Filter: non null",{{"Vendor", Order.Ascending}, {"Sales", Order.Descending}}), // Largest Sales of Each #"Remove Duplicate Vendors" = Table.Distinct(#"Sort Columns", {"Vendor"}), #"Removed Other Columns" = Table.SelectColumns(Source,{"Vendor", "Sales", "Total Amount"}), #"Merged Queries" = Table.NestedJoin(#"Removed Other Columns",{"Vendor"},#"Remove Duplicate Vendors",{"Vendor"},"Clean States",JoinKind.Inner), #"Expanded Clean States" = Table.ExpandTableColumn(#"Merged Queries", "Clean States", {"State"}, {"State"}) in #"Expanded Clean States" In the above let expression – we debunk the notion that M is a top to bottom evaluation instead a series of parameters that can be recalled at various stages during the transformation. In the identifier #“Removed Other Columns” we reference the first identifier Source. In the identifier #”Merge Queries” we perform a self merge to begin with and edit the M formula’s second table reference to point to the #”Removed Duplicate Vendors” identifier. Full Solution: Down Is Up
  16. 16. ROADMAPS?... WHERE WE’RE GOING, WE DON’T NEED ROADMAPS. THE FUTURE IS OURS TO WRITE.
  17. 17. IF YOU WANT YOUR SERVICE TO BE A HIT. MAKE SURE IT WORKS WITH EXCEL. Oct-11 SQL Azure Labs Feb-13 Excel Add-In Jul-13 Excel (Natively) Dec-14 Power BI Apr-17 Azure Analysis Services Mar-18 Flow Nov-18 DataFlows Feb-19 SSIS Aug-17 PowerApps
  18. 18. FLOW
  19. 19. PowerApps
  20. 20. SSAS / AAS – 1400
  21. 21. Power BI Service
  22. 22. VISUAL STUDIO
  23. 23. RESOURCES Power Query Language Specification Guide https://docs.microsoft.com/en-us/powerquery-m/power-query-m-language-specification Collect, Combine, and Transform Data Using Power Query in Excel and Power BI https://www.amazon.com/Collect-Combine-Transform-Business-Skills/dp/1509307958 M Is for (Data) Monkey: A Guide to the M Language in Excel Power Query https://www.amazon.com/Data-Monkey-Guide-Language-Excel/dp/1615470344/ Excel 2016: Get & Transform Data (Power Query) https://www.lynda.com/Excel-tutorials/About-Get-Transform/608994/645967-4.html
  24. 24. CUSTOM CONNECTORS • Visual Studio Extensions • Power Query SDK • Auto Deploy • Resources • Getting Started with Data Connectors • A Deep Dive into the M Language: Creating a Custom Connector • MSDN: Connector Development • Example • OpenWeatherMap
  25. 25. THE FUTURE M-DBA (Highly Specialized in Writing Optimized M Syntax and Building Custom Connectors) Intellisense (DONE) Seamless Integration Across All Offerings (Working Toward a Unified Experience) SSIS and Azure Data Factory Integration (DONE) Microsoft Access (WHY NOT?) Excel for Mac (In Development) Excel Online (Crystal Ball Prediction) Version Control (GitHub) Standalone Product (WHY NOT?)
  26. 26. CLOSING REMARKS IT’S NOT ABOUT WHAT YOU KNOW. IT’S ABOUT WHAT YOU KNOW IS POSSIBLE.
  27. 27. IT’S NOT ABOUT THE CELL www.itsnotaboutthecell.com
  28. 28. QUESTIONS? YOU THINK I’M CRAZY?!
  29. 29. Connect Online @notaboutthecell /in/alexmpowers It’s Not About The Cell

×