SlideShare a Scribd company logo
1 of 15
Download to read offline
Hackolade Tutorial
Part 1 – What is a Data Model
Copyright © 2016-2023 Hackolade 1
What is a data model?
• An abstract representation of how elements of data are
organized, how they relate to each other, and how how they
relate to real-world concepts.
• An explicit representation of the structure of data.
• The purpose of data modeling is to support the development
of IT systems and to ensure consistency, compatibility, and
quality of data.
Copyright © 2016-2023 Hackolade 2
What is a data model?
• a data model describes a business.
• a blueprint for applications,
• a map helping evaluate design options
A data model helps to plan ahead, in order to minimize later rework.
Copyright © 2016-2023 Hackolade 3
ER diagrams help
understand data
models
They are a
graphical representation
of entities,
their characteristics, and
their relationships
Copyright © 2016-2023 Hackolade 4
Schemas are machine
readable contracts
between systems
• graphical representations
are for humans, not for
systems.
• models are useful after
the modeling
• models can generate
schema’s -- a contract
between 2 systems.
Copyright © 2016-2023 Hackolade 5
How to start a data model from scratch?
• it is common to start with "functional requirements”:
description of the service that the software must provide.
• data modeling is an iterative process.
• Don't model everything at once.
• start with the concepts of the core purpose of the application,
• dive into the details afterwards
• validate your design with all application stakeholders
Copyright © 2016-2023 Hackolade 6
Components of a data model
• a data modeler will try to identify: entities, their attributes, and the
relationships between the entities.
• Eg. associate nouns to entities, and verbs to indicate an attribute or a
relationship.
• For example: a customer, with first and last name, owns a vehicle. The vehicle
was registered on a given date and is manufactured by a make.
• Entities: customer, vehicle, and make
• Relationships: customer OWNS vehicle, and make MANUFACTURES vehicle
• Attributes: first name and last name are attributes of customer. Registration date is an
attribute of vehicle.
• In RDBMSs
• a physical table == an entity,
• columns == an attribute of that entity,
• rows == instances of that entity
• FK relationships link tables together.
• With NoSQL databases and denormalization, it's different
Copyright © 2016-2023 Hackolade 7
Different types of data models
• Traditionally, data modelers
have identified 3 different
levels of data modeling
• A conceptual data model is a
high-level representation of
the vocabulary and the data
without any details about how
the data is stored or
implemented.
• A logical data model provides
more details about the
business rules.
• A physical data model
specifies how the data is
physically stored in a database
or other storage system.
Copyright © 2016-2023 Hackolade 8
The role of Normalisation in Data Modeling
• Normalization is a technique
• to reduce data redundancy
• to ensure data consistency
• In a normalized database, data is stored in separate tables
• A table == a single entity or relationship,
• A column == a single attribute of the entity,
• A row == a unique instance of that entity.
• FK relationships link tables together.
• Logical Data Models were already normalised
• This complicates their use in NOSQL
• NOSQL leverages denormalization in function of the access pattern of the
application.
Copyright © 2016-2023 Hackolade 9
The match with Polyglot Persistence
• Polyglot persistence is a data storage architecture
• different types of data are stored in different types of databases, based on
the specific requirements of each type of data.
• Eg. use a relational database for transactional data, a graph database for social network
data, a document database for unstructured data, and a columnar database for analytics
data.
• Advantage
• best database for each type of data
• better performance, scalability, and flexibility
• can simplify data modeling / duplication / redundancy
• Disadvantage
• complexity in the application's data storage layer
• requires careful planning and coordination
Copyright © 2016-2023 Hackolade 10
The need for Polyglot Data Models
• Because
• NOSQL databases
• API based software development
need for heterogeneous data
models
• allow for denormalisation at the
conceptual/logical
• POLYGLOT DATA MODELS
• allow denormalization, if desired,
given access patterns;
• allow complex data types;
• generate schemas for a variety of
technologies, with automatic
mapping to the specific data types of
the respective target technologies.
Copyright © 2016-2023 Hackolade 11
The need for Domain-Driven Data Modeling
• DDDM will allow you to
• focus on your core ("domain and sub-domains")
• break down complex problems into smaller ones ("bounded context")
• use data modeling as a communication tool ("ubiquitous language")
• keep together what belongs together ("aggregates")
• reach a shared understanding between business and tech ("collaboration of
domain experts and developers")
• iterate and evolve ("continuous refinement")
Copyright © 2016-2023 Hackolade 12
Conceptual Logical Physical
Polyglot
Data Model
Target-specific
Data Model
Relational
data modeling
NoSQL/SQL/API/…
data modeling
Domain
Driven
Data
Modeling
Reading material
• Our articles on
• Polyglot data modeling
• Domain driven data modeling
• Metadata as code
• See Hackolade online documentation
• The Hackolade Blog
• Hackolade’s on social media: LinkedIn page, Twitter page
• Download Hackolade studio for free
Copyright © 2016-2023 Hackolade 14
Questions?
Answers!
Copyright © 2016-2023 Hackolade 15

More Related Content

What's hot

7. Key-Value Databases: In Depth
7. Key-Value Databases: In Depth7. Key-Value Databases: In Depth
7. Key-Value Databases: In DepthFabio Fumarola
 
From flat files to deconstructed database
From flat files to deconstructed databaseFrom flat files to deconstructed database
From flat files to deconstructed databaseJulien Le Dem
 
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the CloudAmazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the CloudNoritaka Sekiyama
 
Iceberg: a fast table format for S3
Iceberg: a fast table format for S3Iceberg: a fast table format for S3
Iceberg: a fast table format for S3DataWorks Summit
 
Migrating with Debezium
Migrating with DebeziumMigrating with Debezium
Migrating with DebeziumMike Fowler
 
Apache Spark Data Source V2 with Wenchen Fan and Gengliang Wang
Apache Spark Data Source V2 with Wenchen Fan and Gengliang WangApache Spark Data Source V2 with Wenchen Fan and Gengliang Wang
Apache Spark Data Source V2 with Wenchen Fan and Gengliang WangDatabricks
 
Introduction to Apache NiFi dws19 DWS - DC 2019
Introduction to Apache NiFi   dws19 DWS - DC 2019Introduction to Apache NiFi   dws19 DWS - DC 2019
Introduction to Apache NiFi dws19 DWS - DC 2019Timothy Spann
 
Hybrid MongoDB and RDBMS Applications
Hybrid MongoDB and RDBMS ApplicationsHybrid MongoDB and RDBMS Applications
Hybrid MongoDB and RDBMS ApplicationsSteven Francia
 
Consumer offset management in Kafka
Consumer offset management in KafkaConsumer offset management in Kafka
Consumer offset management in KafkaJoel Koshy
 
Building Data Lakes with Apache Airflow
Building Data Lakes with Apache AirflowBuilding Data Lakes with Apache Airflow
Building Data Lakes with Apache AirflowGary Stafford
 
Simplify CDC Pipeline with Spark Streaming SQL and Delta Lake
Simplify CDC Pipeline with Spark Streaming SQL and Delta LakeSimplify CDC Pipeline with Spark Streaming SQL and Delta Lake
Simplify CDC Pipeline with Spark Streaming SQL and Delta LakeDatabricks
 
Spark introduction and architecture
Spark introduction and architectureSpark introduction and architecture
Spark introduction and architectureSohil Jain
 
Parallelization of Structured Streaming Jobs Using Delta Lake
Parallelization of Structured Streaming Jobs Using Delta LakeParallelization of Structured Streaming Jobs Using Delta Lake
Parallelization of Structured Streaming Jobs Using Delta LakeDatabricks
 
The columnar roadmap: Apache Parquet and Apache Arrow
The columnar roadmap: Apache Parquet and Apache ArrowThe columnar roadmap: Apache Parquet and Apache Arrow
The columnar roadmap: Apache Parquet and Apache ArrowJulien Le Dem
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDBRavi Teja
 

What's hot (20)

7. Key-Value Databases: In Depth
7. Key-Value Databases: In Depth7. Key-Value Databases: In Depth
7. Key-Value Databases: In Depth
 
NoSQL
NoSQLNoSQL
NoSQL
 
From flat files to deconstructed database
From flat files to deconstructed databaseFrom flat files to deconstructed database
From flat files to deconstructed database
 
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the CloudAmazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
Amazon S3 Best Practice and Tuning for Hadoop/Spark in the Cloud
 
Iceberg: a fast table format for S3
Iceberg: a fast table format for S3Iceberg: a fast table format for S3
Iceberg: a fast table format for S3
 
Migrating with Debezium
Migrating with DebeziumMigrating with Debezium
Migrating with Debezium
 
Apache Spark Data Source V2 with Wenchen Fan and Gengliang Wang
Apache Spark Data Source V2 with Wenchen Fan and Gengliang WangApache Spark Data Source V2 with Wenchen Fan and Gengliang Wang
Apache Spark Data Source V2 with Wenchen Fan and Gengliang Wang
 
Introduction to Apache NiFi dws19 DWS - DC 2019
Introduction to Apache NiFi   dws19 DWS - DC 2019Introduction to Apache NiFi   dws19 DWS - DC 2019
Introduction to Apache NiFi dws19 DWS - DC 2019
 
Hybrid MongoDB and RDBMS Applications
Hybrid MongoDB and RDBMS ApplicationsHybrid MongoDB and RDBMS Applications
Hybrid MongoDB and RDBMS Applications
 
Key-Value NoSQL Database
Key-Value NoSQL DatabaseKey-Value NoSQL Database
Key-Value NoSQL Database
 
Consumer offset management in Kafka
Consumer offset management in KafkaConsumer offset management in Kafka
Consumer offset management in Kafka
 
Apache Spark Architecture
Apache Spark ArchitectureApache Spark Architecture
Apache Spark Architecture
 
Building Data Lakes with Apache Airflow
Building Data Lakes with Apache AirflowBuilding Data Lakes with Apache Airflow
Building Data Lakes with Apache Airflow
 
Simplify CDC Pipeline with Spark Streaming SQL and Delta Lake
Simplify CDC Pipeline with Spark Streaming SQL and Delta LakeSimplify CDC Pipeline with Spark Streaming SQL and Delta Lake
Simplify CDC Pipeline with Spark Streaming SQL and Delta Lake
 
Spark introduction and architecture
Spark introduction and architectureSpark introduction and architecture
Spark introduction and architecture
 
YARN Federation
YARN Federation YARN Federation
YARN Federation
 
Parallelization of Structured Streaming Jobs Using Delta Lake
Parallelization of Structured Streaming Jobs Using Delta LakeParallelization of Structured Streaming Jobs Using Delta Lake
Parallelization of Structured Streaming Jobs Using Delta Lake
 
The columnar roadmap: Apache Parquet and Apache Arrow
The columnar roadmap: Apache Parquet and Apache ArrowThe columnar roadmap: Apache Parquet and Apache Arrow
The columnar roadmap: Apache Parquet and Apache Arrow
 
Introduction to mongodb
Introduction to mongodbIntroduction to mongodb
Introduction to mongodb
 
Introduction to MongoDB
Introduction to MongoDBIntroduction to MongoDB
Introduction to MongoDB
 

Similar to Hackolade Tutorial - part 1 - What is a data model

Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...
Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...
Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...PascalDesmarets1
 
Tutorial Getting Started part 1 - Overview
Tutorial Getting Started part 1 - OverviewTutorial Getting Started part 1 - Overview
Tutorial Getting Started part 1 - OverviewPascalDesmarets1
 
DBMS data modeling.pptx
DBMS data modeling.pptxDBMS data modeling.pptx
DBMS data modeling.pptxMrwafaAbbas
 
Data modelling it's process and examples
Data modelling it's process and examplesData modelling it's process and examples
Data modelling it's process and examplesJayeshGadhave1
 
Chapter-2 Database System Concepts and Architecture
Chapter-2 Database System Concepts and ArchitectureChapter-2 Database System Concepts and Architecture
Chapter-2 Database System Concepts and ArchitectureKunal Anand
 
Types of data bases
Types of data basesTypes of data bases
Types of data basesJanu Jahnavi
 
INTRODUCTION OF DATA BASE
INTRODUCTION OF DATA BASEINTRODUCTION OF DATA BASE
INTRODUCTION OF DATA BASEAMUTHAG2
 
Fundamentals of Database ppt ch02
Fundamentals of Database ppt ch02Fundamentals of Database ppt ch02
Fundamentals of Database ppt ch02Jotham Gadot
 
Arches Getty Brownbag Talk
Arches Getty Brownbag TalkArches Getty Brownbag Talk
Arches Getty Brownbag Talkbenosteen
 
Relational Database explanation with detail.pdf
Relational Database explanation with detail.pdfRelational Database explanation with detail.pdf
Relational Database explanation with detail.pdf9wldv5h8n
 
Data Lake Acceleration vs. Data Virtualization - What’s the difference?
Data Lake Acceleration vs. Data Virtualization - What’s the difference?Data Lake Acceleration vs. Data Virtualization - What’s the difference?
Data Lake Acceleration vs. Data Virtualization - What’s the difference?Denodo
 
chapter5-220725172250-dc425eb2.pdf
chapter5-220725172250-dc425eb2.pdfchapter5-220725172250-dc425eb2.pdf
chapter5-220725172250-dc425eb2.pdfMahmoudSOLIMAN380726
 
Chapter 5: Data Development
Chapter 5: Data Development Chapter 5: Data Development
Chapter 5: Data Development Ahmed Alorage
 
NoSQL-Database-Concepts
NoSQL-Database-ConceptsNoSQL-Database-Concepts
NoSQL-Database-ConceptsBhaskar Gunda
 

Similar to Hackolade Tutorial - part 1 - What is a data model (20)

Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...
Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...
Hackolade Tutorial - part 3 - Query-driven data modeling based on access patt...
 
Tutorial Getting Started part 1 - Overview
Tutorial Getting Started part 1 - OverviewTutorial Getting Started part 1 - Overview
Tutorial Getting Started part 1 - Overview
 
DBMS data modeling.pptx
DBMS data modeling.pptxDBMS data modeling.pptx
DBMS data modeling.pptx
 
Data modelling it's process and examples
Data modelling it's process and examplesData modelling it's process and examples
Data modelling it's process and examples
 
Chapter-2 Database System Concepts and Architecture
Chapter-2 Database System Concepts and ArchitectureChapter-2 Database System Concepts and Architecture
Chapter-2 Database System Concepts and Architecture
 
(Dbms) class 1 & 2 (Presentation)
(Dbms) class 1 & 2 (Presentation)(Dbms) class 1 & 2 (Presentation)
(Dbms) class 1 & 2 (Presentation)
 
Types of data bases
Types of data basesTypes of data bases
Types of data bases
 
Database management system
Database management systemDatabase management system
Database management system
 
INTRODUCTION OF DATA BASE
INTRODUCTION OF DATA BASEINTRODUCTION OF DATA BASE
INTRODUCTION OF DATA BASE
 
Fundamentals of Database ppt ch02
Fundamentals of Database ppt ch02Fundamentals of Database ppt ch02
Fundamentals of Database ppt ch02
 
DBMS-Unit-1.pptx
DBMS-Unit-1.pptxDBMS-Unit-1.pptx
DBMS-Unit-1.pptx
 
Arches Getty Brownbag Talk
Arches Getty Brownbag TalkArches Getty Brownbag Talk
Arches Getty Brownbag Talk
 
Top 5-nosql
Top 5-nosqlTop 5-nosql
Top 5-nosql
 
Dbms
DbmsDbms
Dbms
 
Mis assignment (database)
Mis assignment (database)Mis assignment (database)
Mis assignment (database)
 
Relational Database explanation with detail.pdf
Relational Database explanation with detail.pdfRelational Database explanation with detail.pdf
Relational Database explanation with detail.pdf
 
Data Lake Acceleration vs. Data Virtualization - What’s the difference?
Data Lake Acceleration vs. Data Virtualization - What’s the difference?Data Lake Acceleration vs. Data Virtualization - What’s the difference?
Data Lake Acceleration vs. Data Virtualization - What’s the difference?
 
chapter5-220725172250-dc425eb2.pdf
chapter5-220725172250-dc425eb2.pdfchapter5-220725172250-dc425eb2.pdf
chapter5-220725172250-dc425eb2.pdf
 
Chapter 5: Data Development
Chapter 5: Data Development Chapter 5: Data Development
Chapter 5: Data Development
 
NoSQL-Database-Concepts
NoSQL-Database-ConceptsNoSQL-Database-Concepts
NoSQL-Database-Concepts
 

More from PascalDesmarets1

Tutorial Workgroup - Working with Forks
Tutorial Workgroup - Working with ForksTutorial Workgroup - Working with Forks
Tutorial Workgroup - Working with ForksPascalDesmarets1
 
Tutorial Advanced How-To - Oracle 23c Duality views
Tutorial Advanced How-To - Oracle 23c Duality viewsTutorial Advanced How-To - Oracle 23c Duality views
Tutorial Advanced How-To - Oracle 23c Duality viewsPascalDesmarets1
 
Tutorial Expert How-To - Docker-based automation
Tutorial Expert How-To - Docker-based automationTutorial Expert How-To - Docker-based automation
Tutorial Expert How-To - Docker-based automationPascalDesmarets1
 
Tutorial Getting Started part 4 - Domain-Driven Data Modeling
Tutorial Getting Started part 4 - Domain-Driven Data ModelingTutorial Getting Started part 4 - Domain-Driven Data Modeling
Tutorial Getting Started part 4 - Domain-Driven Data ModelingPascalDesmarets1
 
Tutorial Getting Started part 3 - Metadata-as-Code
Tutorial Getting Started part 3 - Metadata-as-CodeTutorial Getting Started part 3 - Metadata-as-Code
Tutorial Getting Started part 3 - Metadata-as-CodePascalDesmarets1
 
Tutorial Getting Started part 2 - Polyglot Data Modeling
Tutorial Getting Started part 2 - Polyglot Data ModelingTutorial Getting Started part 2 - Polyglot Data Modeling
Tutorial Getting Started part 2 - Polyglot Data ModelingPascalDesmarets1
 
Tutorial Expert How-To - Create a model for Avro schemas
Tutorial Expert How-To - Create a model for Avro schemasTutorial Expert How-To - Create a model for Avro schemas
Tutorial Expert How-To - Create a model for Avro schemasPascalDesmarets1
 
Tutorial Expert How-To - Verify Data Model
Tutorial Expert How-To - Verify Data ModelTutorial Expert How-To - Verify Data Model
Tutorial Expert How-To - Verify Data ModelPascalDesmarets1
 
Tutorial Workgroup - Model versioning and collaboration
Tutorial Workgroup - Model versioning and collaborationTutorial Workgroup - Model versioning and collaboration
Tutorial Workgroup - Model versioning and collaborationPascalDesmarets1
 
Tutorial Expert How-To - Naming Conventions
Tutorial Expert How-To - Naming ConventionsTutorial Expert How-To - Naming Conventions
Tutorial Expert How-To - Naming ConventionsPascalDesmarets1
 
Tutorial Expert How-To - Export-Import with Excel template
Tutorial Expert How-To - Export-Import with Excel templateTutorial Expert How-To - Export-Import with Excel template
Tutorial Expert How-To - Export-Import with Excel templatePascalDesmarets1
 
Tutorial Expert How-To - Compare and Merge
Tutorial Expert How-To - Compare and MergeTutorial Expert How-To - Compare and Merge
Tutorial Expert How-To - Compare and MergePascalDesmarets1
 
Tutorial Expert How-To - Custom properties
Tutorial Expert How-To - Custom propertiesTutorial Expert How-To - Custom properties
Tutorial Expert How-To - Custom propertiesPascalDesmarets1
 
Tutorial Expert How-To - Command Line Interface (CLI)
Tutorial Expert How-To - Command Line Interface (CLI)Tutorial Expert How-To - Command Line Interface (CLI)
Tutorial Expert How-To - Command Line Interface (CLI)PascalDesmarets1
 
Tutorial Expert How-To - Add reusable Definitions
Tutorial Expert How-To - Add reusable DefinitionsTutorial Expert How-To - Add reusable Definitions
Tutorial Expert How-To - Add reusable DefinitionsPascalDesmarets1
 
Hackolade Tutorial - part 12 - Create a REST API model
Hackolade Tutorial - part  12 - Create a REST API modelHackolade Tutorial - part  12 - Create a REST API model
Hackolade Tutorial - part 12 - Create a REST API modelPascalDesmarets1
 
Hackolade Tutorial - part 9 - Export or forward-engineer.pdf
Hackolade Tutorial - part 9 - Export or forward-engineer.pdfHackolade Tutorial - part 9 - Export or forward-engineer.pdf
Hackolade Tutorial - part 9 - Export or forward-engineer.pdfPascalDesmarets1
 
Hackolade Tutorial - part 8 - Import or reverse-engineer.pdf
Hackolade Tutorial - part 8 - Import or reverse-engineer.pdfHackolade Tutorial - part 8 - Import or reverse-engineer.pdf
Hackolade Tutorial - part 8 - Import or reverse-engineer.pdfPascalDesmarets1
 
Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdf
Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdfHackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdf
Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdfPascalDesmarets1
 
Hackolade Tutorial - part 4 - Create your first data model
Hackolade Tutorial - part 4 - Create your first data modelHackolade Tutorial - part 4 - Create your first data model
Hackolade Tutorial - part 4 - Create your first data modelPascalDesmarets1
 

More from PascalDesmarets1 (20)

Tutorial Workgroup - Working with Forks
Tutorial Workgroup - Working with ForksTutorial Workgroup - Working with Forks
Tutorial Workgroup - Working with Forks
 
Tutorial Advanced How-To - Oracle 23c Duality views
Tutorial Advanced How-To - Oracle 23c Duality viewsTutorial Advanced How-To - Oracle 23c Duality views
Tutorial Advanced How-To - Oracle 23c Duality views
 
Tutorial Expert How-To - Docker-based automation
Tutorial Expert How-To - Docker-based automationTutorial Expert How-To - Docker-based automation
Tutorial Expert How-To - Docker-based automation
 
Tutorial Getting Started part 4 - Domain-Driven Data Modeling
Tutorial Getting Started part 4 - Domain-Driven Data ModelingTutorial Getting Started part 4 - Domain-Driven Data Modeling
Tutorial Getting Started part 4 - Domain-Driven Data Modeling
 
Tutorial Getting Started part 3 - Metadata-as-Code
Tutorial Getting Started part 3 - Metadata-as-CodeTutorial Getting Started part 3 - Metadata-as-Code
Tutorial Getting Started part 3 - Metadata-as-Code
 
Tutorial Getting Started part 2 - Polyglot Data Modeling
Tutorial Getting Started part 2 - Polyglot Data ModelingTutorial Getting Started part 2 - Polyglot Data Modeling
Tutorial Getting Started part 2 - Polyglot Data Modeling
 
Tutorial Expert How-To - Create a model for Avro schemas
Tutorial Expert How-To - Create a model for Avro schemasTutorial Expert How-To - Create a model for Avro schemas
Tutorial Expert How-To - Create a model for Avro schemas
 
Tutorial Expert How-To - Verify Data Model
Tutorial Expert How-To - Verify Data ModelTutorial Expert How-To - Verify Data Model
Tutorial Expert How-To - Verify Data Model
 
Tutorial Workgroup - Model versioning and collaboration
Tutorial Workgroup - Model versioning and collaborationTutorial Workgroup - Model versioning and collaboration
Tutorial Workgroup - Model versioning and collaboration
 
Tutorial Expert How-To - Naming Conventions
Tutorial Expert How-To - Naming ConventionsTutorial Expert How-To - Naming Conventions
Tutorial Expert How-To - Naming Conventions
 
Tutorial Expert How-To - Export-Import with Excel template
Tutorial Expert How-To - Export-Import with Excel templateTutorial Expert How-To - Export-Import with Excel template
Tutorial Expert How-To - Export-Import with Excel template
 
Tutorial Expert How-To - Compare and Merge
Tutorial Expert How-To - Compare and MergeTutorial Expert How-To - Compare and Merge
Tutorial Expert How-To - Compare and Merge
 
Tutorial Expert How-To - Custom properties
Tutorial Expert How-To - Custom propertiesTutorial Expert How-To - Custom properties
Tutorial Expert How-To - Custom properties
 
Tutorial Expert How-To - Command Line Interface (CLI)
Tutorial Expert How-To - Command Line Interface (CLI)Tutorial Expert How-To - Command Line Interface (CLI)
Tutorial Expert How-To - Command Line Interface (CLI)
 
Tutorial Expert How-To - Add reusable Definitions
Tutorial Expert How-To - Add reusable DefinitionsTutorial Expert How-To - Add reusable Definitions
Tutorial Expert How-To - Add reusable Definitions
 
Hackolade Tutorial - part 12 - Create a REST API model
Hackolade Tutorial - part  12 - Create a REST API modelHackolade Tutorial - part  12 - Create a REST API model
Hackolade Tutorial - part 12 - Create a REST API model
 
Hackolade Tutorial - part 9 - Export or forward-engineer.pdf
Hackolade Tutorial - part 9 - Export or forward-engineer.pdfHackolade Tutorial - part 9 - Export or forward-engineer.pdf
Hackolade Tutorial - part 9 - Export or forward-engineer.pdf
 
Hackolade Tutorial - part 8 - Import or reverse-engineer.pdf
Hackolade Tutorial - part 8 - Import or reverse-engineer.pdfHackolade Tutorial - part 8 - Import or reverse-engineer.pdf
Hackolade Tutorial - part 8 - Import or reverse-engineer.pdf
 
Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdf
Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdfHackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdf
Hackolade Tutorial - part 6 - Add choice, conditional, pattern fields.pdf
 
Hackolade Tutorial - part 4 - Create your first data model
Hackolade Tutorial - part 4 - Create your first data modelHackolade Tutorial - part 4 - Create your first data model
Hackolade Tutorial - part 4 - Create your first data model
 

Recently uploaded

Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxRustici Software
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Jeffrey Haguewood
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...apidays
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...Martijn de Jong
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024The Digital Insurer
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfOverkill Security
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processorsdebabhi2
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)wesley chun
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024The Digital Insurer
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyKhushali Kathiriya
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...DianaGray10
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?Igalia
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu SubbuApidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbuapidays
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherRemote DBA Services
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...apidays
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024The Digital Insurer
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Zilliz
 

Recently uploaded (20)

Corporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptxCorporate and higher education May webinar.pptx
Corporate and higher education May webinar.pptx
 
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
Web Form Automation for Bonterra Impact Management (fka Social Solutions Apri...
 
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
Apidays New York 2024 - The Good, the Bad and the Governed by David O'Neill, ...
 
2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...2024: Domino Containers - The Next Step. News from the Domino Container commu...
2024: Domino Containers - The Next Step. News from the Domino Container commu...
 
Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024Manulife - Insurer Transformation Award 2024
Manulife - Insurer Transformation Award 2024
 
Ransomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdfRansomware_Q4_2023. The report. [EN].pdf
Ransomware_Q4_2023. The report. [EN].pdf
 
Exploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone ProcessorsExploring the Future Potential of AI-Enabled Smartphone Processors
Exploring the Future Potential of AI-Enabled Smartphone Processors
 
Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)Powerful Google developer tools for immediate impact! (2023-24 C)
Powerful Google developer tools for immediate impact! (2023-24 C)
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024AXA XL - Insurer Innovation Award Americas 2024
AXA XL - Insurer Innovation Award Americas 2024
 
Artificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : UncertaintyArtificial Intelligence Chap.5 : Uncertainty
Artificial Intelligence Chap.5 : Uncertainty
 
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
Connector Corner: Accelerate revenue generation using UiPath API-centric busi...
 
A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?A Year of the Servo Reboot: Where Are We Now?
A Year of the Servo Reboot: Where Are We Now?
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
+971581248768>> SAFE AND ORIGINAL ABORTION PILLS FOR SALE IN DUBAI AND ABUDHA...
 
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu SubbuApidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
Apidays Singapore 2024 - Modernizing Securities Finance by Madhu Subbu
 
Strategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a FresherStrategies for Landing an Oracle DBA Job as a Fresher
Strategies for Landing an Oracle DBA Job as a Fresher
 
Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...Apidays New York 2024 - The value of a flexible API Management solution for O...
Apidays New York 2024 - The value of a flexible API Management solution for O...
 
Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024Axa Assurance Maroc - Insurer Innovation Award 2024
Axa Assurance Maroc - Insurer Innovation Award 2024
 
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
Emergent Methods: Multi-lingual narrative tracking in the news - real-time ex...
 

Hackolade Tutorial - part 1 - What is a data model

  • 1. Hackolade Tutorial Part 1 – What is a Data Model Copyright © 2016-2023 Hackolade 1
  • 2. What is a data model? • An abstract representation of how elements of data are organized, how they relate to each other, and how how they relate to real-world concepts. • An explicit representation of the structure of data. • The purpose of data modeling is to support the development of IT systems and to ensure consistency, compatibility, and quality of data. Copyright © 2016-2023 Hackolade 2
  • 3. What is a data model? • a data model describes a business. • a blueprint for applications, • a map helping evaluate design options A data model helps to plan ahead, in order to minimize later rework. Copyright © 2016-2023 Hackolade 3
  • 4. ER diagrams help understand data models They are a graphical representation of entities, their characteristics, and their relationships Copyright © 2016-2023 Hackolade 4
  • 5. Schemas are machine readable contracts between systems • graphical representations are for humans, not for systems. • models are useful after the modeling • models can generate schema’s -- a contract between 2 systems. Copyright © 2016-2023 Hackolade 5
  • 6. How to start a data model from scratch? • it is common to start with "functional requirements”: description of the service that the software must provide. • data modeling is an iterative process. • Don't model everything at once. • start with the concepts of the core purpose of the application, • dive into the details afterwards • validate your design with all application stakeholders Copyright © 2016-2023 Hackolade 6
  • 7. Components of a data model • a data modeler will try to identify: entities, their attributes, and the relationships between the entities. • Eg. associate nouns to entities, and verbs to indicate an attribute or a relationship. • For example: a customer, with first and last name, owns a vehicle. The vehicle was registered on a given date and is manufactured by a make. • Entities: customer, vehicle, and make • Relationships: customer OWNS vehicle, and make MANUFACTURES vehicle • Attributes: first name and last name are attributes of customer. Registration date is an attribute of vehicle. • In RDBMSs • a physical table == an entity, • columns == an attribute of that entity, • rows == instances of that entity • FK relationships link tables together. • With NoSQL databases and denormalization, it's different Copyright © 2016-2023 Hackolade 7
  • 8. Different types of data models • Traditionally, data modelers have identified 3 different levels of data modeling • A conceptual data model is a high-level representation of the vocabulary and the data without any details about how the data is stored or implemented. • A logical data model provides more details about the business rules. • A physical data model specifies how the data is physically stored in a database or other storage system. Copyright © 2016-2023 Hackolade 8
  • 9. The role of Normalisation in Data Modeling • Normalization is a technique • to reduce data redundancy • to ensure data consistency • In a normalized database, data is stored in separate tables • A table == a single entity or relationship, • A column == a single attribute of the entity, • A row == a unique instance of that entity. • FK relationships link tables together. • Logical Data Models were already normalised • This complicates their use in NOSQL • NOSQL leverages denormalization in function of the access pattern of the application. Copyright © 2016-2023 Hackolade 9
  • 10. The match with Polyglot Persistence • Polyglot persistence is a data storage architecture • different types of data are stored in different types of databases, based on the specific requirements of each type of data. • Eg. use a relational database for transactional data, a graph database for social network data, a document database for unstructured data, and a columnar database for analytics data. • Advantage • best database for each type of data • better performance, scalability, and flexibility • can simplify data modeling / duplication / redundancy • Disadvantage • complexity in the application's data storage layer • requires careful planning and coordination Copyright © 2016-2023 Hackolade 10
  • 11. The need for Polyglot Data Models • Because • NOSQL databases • API based software development need for heterogeneous data models • allow for denormalisation at the conceptual/logical • POLYGLOT DATA MODELS • allow denormalization, if desired, given access patterns; • allow complex data types; • generate schemas for a variety of technologies, with automatic mapping to the specific data types of the respective target technologies. Copyright © 2016-2023 Hackolade 11
  • 12. The need for Domain-Driven Data Modeling • DDDM will allow you to • focus on your core ("domain and sub-domains") • break down complex problems into smaller ones ("bounded context") • use data modeling as a communication tool ("ubiquitous language") • keep together what belongs together ("aggregates") • reach a shared understanding between business and tech ("collaboration of domain experts and developers") • iterate and evolve ("continuous refinement") Copyright © 2016-2023 Hackolade 12
  • 13. Conceptual Logical Physical Polyglot Data Model Target-specific Data Model Relational data modeling NoSQL/SQL/API/… data modeling Domain Driven Data Modeling
  • 14. Reading material • Our articles on • Polyglot data modeling • Domain driven data modeling • Metadata as code • See Hackolade online documentation • The Hackolade Blog • Hackolade’s on social media: LinkedIn page, Twitter page • Download Hackolade studio for free Copyright © 2016-2023 Hackolade 14