Data Modeling for Big Data & NoSQL Technologies with Karen Lopez

www.infoadvisors.com Aug 2014
1
Big Data, NoSQL &
Data Modeling
10 Tips for Data Modeling Success on Modern Data Projects
Karen Lopez, InfoAdvisors
www.datamodel.com
Karen López
Karen has 20+ years of data an information architecture
experience on large, multi-project programs.
She is a frequent speaker on data modeling, data-driven
methodologies and pattern data models.
She wants you to love your data.
©InfoAdvisors - infoadvisors.com
#TEAMDATA
Aug 2014

2
POLL: Who Are You?
©InfoAdvisors - infoadvisors.com Aug 2014
©InfoAdvisors - infoadvisors.com
POLL: NoSQL Much?
Aug 2014

3
“BIG DATA”
[x] Vs
“Data so big it’s awkward to work with”
Always capitalized Big Data
A confusing term because it defines what it IS NOT.
“NoSQL”
Scale
Not SQL?
Not Relational?
Not Only Relational?
A confusing term because it defines what it IS NOT.

4
Terminology
ACID
Atomic, Consistent, Isolated, Durable
BASE
Basically available, Soft state, Eventual consistency
Eventual consistency
Schemaless
Constraints / Have-to/ MUST / OBEY / Rigid / Inflexible
Relational Tables with
rows
same columns
with the same datatypes
with the same constraints
with the same domains
This is a
FEATURE
On purpose
With many benefits
Write-optimized
Trans-action-optimized
Data integrity
Data quality
Consistent

5
Data Models – Traditional Process
Conceptual
(Data)
Model
Logical
Data Model
MART MART
Physical
Data
Model(s) OLTP
OLTP
OLTP OLTP
OLTP
OLTP
OLTP
OLTP
Relational

6
Traditional Data Modeler Involvement
Project
Initiation
Architecture
and
Infrastructure
Design
SW
Requirements
Development
Deployment
The Big Data Story
Lots of data
Coming at us fast
Lots of variety in format & quality
We want all the data
Highly available
“It’s web scale”

7
What do we really mean by scale?
Bringing computing to the data
Massively parallel processing
Cheap, commodity hardware, but lots
of it
Optimized for
Query/Reads/Questions/Telling stories
Can we fit another buzzword in?
Cloud
• Enable on-demand scaling
• Pay as you go pricing
• Click to deploy
• Service licensing, not product licensing, if any
• Managed by others, not your data center

8
We’ve been down this road before…
Traditional
transactional
applications
Reporting-optimized
tables/structures
Data Warehouse
/ Dimensional
Modeling
ETL
EDW
Data
Mart
Data
Mart

9
Hadoop
ETL
EDW
Analytics
Mart
Data
Mart
NoSQL, Not Only SQL
Relational Graph
Columnar/Column
Family
Key Value
Document
Databases
Others

10
Graph Databases
Key Value Pair

11
Columnar
Sample Hive Statement
CREATE EXTERNAL TABLE TaxRebateUsage (
state string,
zipcode string,
agi_class int,
n1 int,
mars2 int,
prep int,
n2 int,
)
ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' STORED AS TEXTFILE

12
Sample JSON/MongoDB Notation
Sample FoundationDB Statement

13
Sample Cassandra Statement
Sample Vertica Statement

14
Sample Neo4j Statement
What else is different, now?
Maturing of high availability technologies
Maturing of ____ as a Service business models
RDBMS vendors adopting non-relational features
Open source software models
100s of Database options

15
The Big Data Big Lies
Schemaless
• Schema on
Read, not
Schema on Write
• Polyschematic
Big
• New data stories
• New
technologies
Importing Structures

16
10 Tips For Modeling in a Hybrid World
1. Models require a modeler
2. Data modeling tools are essential
3. There are many types of data models: know which ones
you need
4. Modeling does not have to happen at the same time in
every project. It should happen at the right time
5. Modeling is not just schema design. Think outside the
boxes and lines
10 Tips for Modeling in a Hybrid World
6. A data model is much more than a diagram
7. You will need training.
8. Team members may not understand modeling.
They will need training
9. NoSQL is not one thing. Learn many patterns
10.Modern data architectures are likely hybrid
solutions. You can’t just support one part.

17
Modern Data Modeler Involvement
Project
Initiation
Architecture
and
Infrastructure
Design
SW
Requirements
Development
Deployment
What does this mean for data modelers?
There will be jobs for traditional, ERD, relational modelers….
….just like there are still jobs of RPG and COBOL programmers
All data has a data story. Many data stories.
A good modeler is a an architect at heart – finding the right
solution for the data story.

18
Business Intelligence Journal
Look for September
Issue Article on Modern
Data Architectures
Thank You!
www.infoadvisors.com
www.datamodel.com
www.dataversity.net
#TEAMDATA

Data Modeling for Big Data & NoSQL Technologies with Karen Lopez

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Data Modeling for Big Data & NoSQL Technologies with Karen Lopez

Ähnlich wie Data Modeling for Big Data & NoSQL Technologies with Karen Lopez (20)

Mehr von Embarcadero Technologies

Mehr von Embarcadero Technologies (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Data Modeling for Big Data & NoSQL Technologies with Karen Lopez