This document provides an overview of NoSQL databases in Azure. It discusses 7 different database types - key-value, column family, document, graph and Hadoop. For each database type it provides information on what it is, examples of use cases, and how to query or model data. It encourages attendees to explore these databases and stresses that choosing the right database for the job is important.

1. 7 Databases
in 70 Minutes
Overview of NoSQL in Azure

2. Technical Architect at Microsoft
Primary focus on data solutions in the cloud
Lara Rubbelke
@sqlgal
www.linkedin.com/in/lararubbelke/

3. Karen has 20+ years of data and information architecture
experience on large, multi-project programs.
She is a frequent speaker on data modeling, data-driven
methodologies and pattern data models.
She wants you to love your data.
Karen López #TEAMDATA

4. The only reason for time is so that
everything doesn’t happen at once.
- Albert Einstein*
Session inspired by the book
Seven Databases in Seven Weeks

5. key concepts for
hybrid database
architectures
database /
datastore types
reasons to go
explore
Outcomes
We want you to leave here understanding:

6. This
is
NOT…
a deep dive on any technology
a comprehensive list
a roadmap discussion
What We Will Cover

7. What We’ll Cover
NoSQL
101
Comparison to relational
Not Only SQL (but really “Not SQL”)
Terminology
Categories What they are
Why you use them
When you use them
A little of how to use them
CAP
ACID
BASE
SCHEMA
Cloud
Scale

8. Distributed Systems and the CAP Theorem
AvailabilityConsistency
Partition Tolerant
Eric Brewer’s
CAP Theorem
and even better
CAP Twelve Years Later
Myth: Eric Brewer On Why Banks Are
BASE Not ACID - Availability Is Revenue

9. Basically Available
Soft State
Eventually Consistent
BASE ACID
Atomic
Consistent
Isolated
Durable
BASE - ACID

10. Polyglot
persistence
• Optimized for data
• Optimized for workload
Not all new
• EAV
• XML
• Architecture paradigm: OLAP/DW
and OLTP
The And

11. Polyschematic
Multiple schemas over
the same data
Schema on read, not
on write
Data integrity may be
managed elsewhere
The Why
* ALL DATA HAS STRUCTURE!
** EMBRACE DENORMALIZATION

12. Kinect Telemetry Retail Application
Reporting/Analysis
Hadoop Batch
Processing
Sensor Data
Column Family
Price Check
Key-Value
Product Catalog
Document Store
{ }

14. Data-Intensive Applications in
the Cloud Computing World
Activity Queue
Azure Storage
Google Analytics
Logs
Azure Storage
Email DBs
SQL Azure x 16
Username DBs
SQL Azure x 16
User Profiles
SQL Azure x 400
Activity Table
X 50 Partitions
Azure Storage
IIS Logs
Azure Storage
Data Analysis: Staging
Virtual Machine
Data
Warehouse
Reporting
Services
Activity Processors
Worker Roles x 2
Cache
Users and Friends Feed
Games and Leader Boards
Resources and References
Distributed Cache x 32
Cache Tasks
Worker Roles x 4
Back Office
Web Roles x 2
Background Tasks DB
Utility DB, Content
DB, Taxonomy DB
SQL Azure
Web Application
Web Roles x 180
Web Service/API
Web Roles x 2
Moderation
Service/Appliance
CRISP/3rd
Party

15. NoSQL, Not Only SQL
Relational Key Value
Column
Family
Document Hadoop Graph

16. …Lots of other sessions to learn about this….
Relational

17. Azure
Tables
Azure
Redis
Cache
Key-Value

18. Database
Key-Value: Sample Use
Table: PriceCompare
LocationID ProductBySellerID ProductProperties
123 013803204131 {Seller:“Camera Superstore”,
Price:425.99, PriceDate:2014-11-06,
SellerType:”Online”}
Row Key PropertiesPartition Key

19. • Low cost, scalable, highly available
and geo-redundant
• Flexible schema
• Fast reads and writes on single key
values or partitioned key values
• Log data and cache
Patterns/What Works Anti-Pattern/Danger
Anything that requires:
• Joins
• Custom sorting
• Non-key filters
Why Key-Value

20. // Create a table client.
CloudTableClient tableKinect = account.CreateCloudTableClient();
CloudTable tableKinectTelemetry =
tableKinect.GetTableReference(“pricecompare");
// Create a query for all entities.
IQueryable<DynamicTableEntity> query =
from q in tableKinectTelemetry.CreateQuery<DynamicTableEntity>()
where q.PartitionKey.Equals(123)
and q.RowKey.Equals(013803204131)
select q;
Azure Tables: LINQ Query

21. Introduction to Windows Azure Tables
Azure Redis Cache 101 on Channel9
Learn More: Azure Tables and Redis Cache

22. DocDB MongoDB BSON &
JSON
Databases,
Documents,
Collections
Document

23. Document: Persistence

24. Nested
Arrays
Keys & Values
Text, text, text….
Similar to XML patterns
Document Features

25. Document: Query
http://docs.mongodb.org/manual/core/read-operations-introduction/
DocDB
Mongo DB

26. • Variable Data Structures for same
type of entity
• Fast reads and writes on a complete
entity set
• Highly nested data stories
• Partially completed workflows
• You love JavaScript 
Patterns/What Works Anti-Pattern/Danger
Anything that requires:
• Joins
• Complex transactional needs
• Lots of aggregation
Why Document

27. Logs
Pre-aggregated data
Product Catalog
Shopping Cart
Travel Reservation
Document Use Cases

28. Azure DocumentDB .NET Code Samples
Azure DocumentDB 101 on Channel9
Azure DocumentDB 102 on Channel9
Build a web application with ASP.NET MVC using
DocumentDB
Learn More: Azure DocumentDB

29. Column Family

30. Sensor Data Analysis
Real-time Query
Web Indexer
Message Systems
Interactive Dashboards
Column Family Use Cases

31. Apache HBase Features
Random and Consistent Real-Time Read/Write
Automatic Sharding and Linear Scale
Billions of Rows and Millions of Columns

32. A map of maps….
With Tables
Column Families
Rows
Columns
Values
Column Family Stores

33. Don’tThinkAbout

34. ThinkAbout

35. ThinkAbout
Row Key
720 gender -> male age -> 62
721 gender -> male photo -> image
723 video -> stream
Person Table

36. sparse | persistent | distributed | sorted | multidimensional
Understanding BigTable
{
"trackingid" : 720,
"gender" : "male",
"age" : 62
}
Great Reference: Understanding HBase and Big Table

37. HBase: A map of maps…
{
"720" : {
"age" : "62",
"gender" : "male"
},
"721" : {
"age" : "40",
"gender" : "male",
"confidence" : "0.65"
},
"722" : {
"gender" : "female"
},
“723" : {
"age" : "12",
"gender" : "female",
"confidence" : "0.65"
},
…
}
Row Key
Sparse

38. HBase: Column Families
"720" :
{ “demographics” :
{ "age" : “62",
"gender" : “male“ },
“interactions” :
{ “devicestate” : “removed”,
“duration” : “100” }
},
"721" :
{ “demographics” :
{ "age" : “40",
"gender" : “male“ },
“interactions” :
{ “devicestate” : “replaced”,
“duration” : “50” }
}
…
Demographics
Interactions
Demographics
Interactions
Multidimensional

39. HBase: Physical View of a Sorted Map
Sort Order
Row Key
Column Name
Timestamp
Row Key Column Key Timestamp Value
720 demographics:age 1423234758774 62
720 demographics:gender 1423234758711 male
721 demographics:age 1423234758946 22
721 demographics:age 1423234758725 32
721 demographics:gender 1423234758950 female
telemetry
Cell
Uninterpreted Bytes
{row, column, version}

40. HBase: Query
And… HBase SDK for .NET

41. CREATE TABLE IF NOT EXISTS "kinecttelemetry"("k" VARCHAR
primary key, "age" VARCHAR, "gender" VARCHAR)
default_column_family='demographics';
Apache Phoenix: SQL Skin over HBase
Phoenix in 15 Minutes or Less

42. Get started using HBase with Hadoop in HDInsight
Analyze Real-Time Twitter Sentiment with HBase in
HDInsight
Learn More: HBase on Azure

43. Distributed Storage
(HDFS or Blob Storage)
Distributed Processing
(MapReduce)
Scripting
(Pig)
SQL-like Query
(HiveQL)
SQL-like Query
(Impala)
Resource Scheduling
(YARN)
Hadoop Zoo
Real-Time
(HBase)

44. Hadoop On Your Terms
Cloudera Selects Microsoft
Azure as a Preferred Cloud
Platform
Hortonworks Data Platform
is now Microsoft Azure
Certified
100% Apache Hadoop-based
Service in the Cloud
Microsoft Azure
HDInsight
Qubole Partners with
Microsoft Azure

45. It’s a text file…really
Hadoop: Persistence

46. CREATE EXTERNAL TABLE irs_data_20082(
state string,
zipcode string,
agi_class int,
n1 int,
mars2 int,
prep int,
n2 int,
numdep int,
ROW FORMAT DELIMITED FIELDS TERMINATED BY ','
STORED AS TEXTFILE
LOCATION
'wasb://$containerName@$storageAccountName.blob.co
re.windows.net/all/data/';
Create Table Query
select state, zipcode,
agi_class
from irs_Data_20082;
Hadoop Hive: External Table

47. • Batch processing
• Map…and reduce
• Lots of aggregation
• Multiple schemas on same data
• Fast
Patterns/What Works Anti-Pattern/Danger
Anything that requires:
• Joins
• Complex transactional needs
• Granular security requirements
• Not a relational database
replacement
• Not fast
Why Hadoop

48. http://azure.microsoft.com/en-
us/documentation/services/hdinsight/
http://vision.cloudera.com/cloudera-on-azure/
http://hortonworks.com/labs/microsoft/
Resource for Hadoop on Azure

49. Neo4j
Project Naiad (MSR
to Open Source)
Graph

50. CREATE Query
Graph Database
http://neo4j.com/docs/stable/cypherdoc-tv-shows.html

51. • Highly connected data
• Relationships make the data story
• Paths through data
• Finding shortest/longest path
Patterns/What Works Anti-Pattern/Danger
• Low connected data (e.g. Log data)
• Very high number of updates on a
regular basis.
Why Graph

52. FoaF
(Social Graph)
Market Basket Analysis
Forensics
Fraud Detection
Recommendations
Use Cases for Graph Databases

53. Free Graph Dabases E-Book
Project Naiad from Microsoft Research
Learn More: Graph Databases

54. It’s fun
Database technologies aren’t YES/NO decisions
It’s inexpensive to learn
It’s fast to spin up a learning environment
A data professional needs to knows more than one tool
Using the right tool for the right job is key
It’s fun
7 Reasons to Go Explore

55. MicrosoftAzure.com
• MSDN Subscription
Benefit
• Trial Accounts
Go Explore!

56. key concepts for
hybrid database
architectures
database /
datastore types
reasons to go
explore
Outcomes
We want you to leave here understanding: