Karen Lopez talks to data architects and data moders how they can best deliver value on modern data drive projects beyond relational database technologies. She covers NoSQL Databases and Datastores, which data stories they best fit and which ones they don't. She ends with 10 tips for adding more value to ployschematic database solutions.
3. Karen López
Karen has 20+ years of data and information architecture
experience on large, multi-project programs.
She is a frequent speaker on data modeling, data-driven
methodologies and pattern data models.
She wants you to love your data.
www.datamodel.com
5. Outcomes: Leave here understanding:
Why multi-
model
designs are
important
Database /
datastore
types
Multi-model
thinking
How to
future-proof
your data
arch career
How to
learn more
6. A Good Data Architect
Cost, Benefit,
Risk
Data
Protection
Business
Requirements
& models
Design
Models
Fit to needs
9. What do you mean,
SURVIVE?
• Continue to be involved in data-
related architectural decisions
• Data models wanted and appreciated
• Data model driven development still
a thing
• Valued for bringing business needs
and database opportunities together
• Team player
• Knowledgeable of database features
11. Hybrid
Purely relational (SQL)
databases don’t exist
any longer
• Columnstore features
• XML
• JSON
• Other NoSQL features
01
Applications make use
of multiple database
and datastore
technologies
02
Schemas in a variety of
places
03
12. Typical Characteristics of Traditional Apps
Real-time request/response
Synchronous
Always connected
Transactional
Scale-Up
Failover
21. Partition Key Row Key Value
Marketing 00001
Name Age
Bob 35
Marketing 00002
Name Age
Karen 42
Marketing Department
Name Count
MKTG 104
Sales 00010
Name Age
Sam 29
• Simple Map
• Semi-Structured
• De-normalized
Key-Value Databases
30. This means…
• SQL vs. NoSQL isn’t a thing any longer
• Other RDBMS vendors will be adding non-
relational functions and structures
• Data Architects who specialize in
relational-only modeling and design will
be overly specialized
• We don’t have a choice to ignore these
new database models
31. Most relational databases are adding NoSQL
support
Columnstore
Graph
XML
JSON
You’ve probably
even developed
your own as a
table
32. Hybrid is the future,
though
• Column Family + Key Value
• Relational + Graph + Columnstore
• Relational + Columnstore
• Graph + Column Family + Key
Value + Document
…all in the same engine. This is a
major change in how things worked
a decade ago.
33. Did you notice?
• Database as a Service from
Microsoft
• Supports (for now)
• Graph
• Key Value
• Column Family
• Document
• Globally distributed data
• Scalable
• Tunable Consistency
CosmosDB
34. But what does this all mean?
Data Modeling process impacts
Data Modeler impacts
35. Quotes from team
NoSQL
• No need for a data model
• Can’t be mapped to a data
model
• SQL is not flexible
• SQL Databases don’t scale
• SQL Databases die after about 1
GB size
• SQL Databases use old
technology
36. Quotes from team
SQL (relational)
• Eventual Consistency? WTH?
• What do you mean data quality
just doesn’t matter?
• My tools won’t work with these
databases
• It’s all a fad; There will be a new
fad tomorrow
• Bbbbbut…but..but DATA
QUALITY!!!!! INTEGRITY!!!
38. Data Models – Traditional Process
Conceptual
(Data)
Model
Logical
Data Model
Physical
Data
Model(s) OLTP
OLTP
OLTP OLTP
OLTP
MARTMART
OLTP
OLTP
OLTP
40. Schemas
• In the database as physical
structure
• In a separate location such as
• DSD
• Schema layer on top of the physical
structure
• In the application code
• In the data itself
41. New Process
Apply
Apply logical data model understandings to physical structure
• Using data modeling and/or tools
• Using a whiteboard
• Using code
Understand Understand data
Understand Understand underlying architecture
42. Traditional Data Modeler Involvement
Project Initiation
Architecture and
Infrastructure
Design
SW Requirements
Development
Deployment
43. Modern Data Modeler Involvement
Project Initiation
Architecture and
Infrastructure
Design
SW Requirements
Development
Deployment
44. What do we really mean by scale?
Rapidly add more compute & data power
Massively parallel processing
Cheap, commodity hardware, but lots of it
Optimized for
Query/Reads/Questions/Telling stories
45. Most NoSQL
• Was developed with scale in mind
• Has tunable consistency
• Scale out, versus scale up
• Uses distributed data (multiple
copies), globally
• Understanding workloads and
workload trends is important
This metadata isn’t often collected
and modeled by data architect
46. Where Data
Models Can
Help
• reverse engineer, where possible.
• one entity per tab spreadsheet
• some Normalized (master data +
combinations)
Create a
Physical Data
Model of Key
Data Sources
• definitions
• gotchas
• expected domains
• metadata
Model the
metadata
• Describe data consistently
• Don’t prescribe structure
• Naming standards
• Datatype-ish
Think
Differently
47. Think Differently
• Data Modeling tools have to catch up
• New tools are being developed
• Data Models can’t be prescriptions
all the time
• Naming standards over constraints
• Data Types are general, higher level
• Tunable data quality
49. Entity-Relationship
Modeling & IDEF1X
Is it enough?
Should we extend it?
Should we create new notations for each
type of Database?
Scrap and start all of it again?
52. Great Enterprise Data Modelers Characteristics
Patient
Zen balance / non-attachment
Get-er done
Collaboration not sentries/cops
Broad Business Domain
Broad, strategic view
Deep, project empathy
Respected
Good project experience
Good negotiators
52
53. So let’s summarize:
1. The more SQL-like features available for NoSQL
databases, the more likely a data modeling tool is to
support it.
2. Modeling tool vendors will support features that users
ask for cause them to win deals. This is not a bad thing.
3. Serious NoSQL vendors* understand that hybrid is the
enterprise data story. They want us to find a way.
4. Our data models have value, even if the NoSQL solution
doesn’t require a lot of constraints.
54. 10 Tips for Data Modelers
1. Learn about these methods – don’t avoid them
2. Get hands-on training. Get certified even
3. Learn the lingo
4. Use the lingo
5. Be able to describe data modeling and data
governance to the context of these database
technologies and their use cases
55. 10 Tips for Data Modelers
6. Bring data models (and other models) to the team
7. Be ahead of the curve on new NoSQL and SQL
features in your DBMSs
8. Understand the use cases for each type of database
technology
9. Let go, a little, when it makes sense
10. Enjoy the new database smell!
56. What to do
Learn
Get Hands-on
experience with
these new data
technologies
Talk to your tool
vendors
Bring models
you have
58. Making Sense of NoSQL clearly and
concisely explains the concepts,
features, benefits, potential, and
limitations of NoSQL technologies.
Using examples and use cases,
illustrations, and plain, jargon-free
writing, this guide shows how you
can effectively assemble a NoSQL
solution to replace or augment the
traditional RDBMS you have now.
60. This book is written for anyone who
is working with, or will be working
with MongoDB, including business
analysts, data modelers, database
administrators, developers, project
managers, and data scientists.