1. BEXIS Tech Talk Series
#2: The Conceptual Model
Javad Chamanara
January 2016
2. Recall from the first talk
ā¢ Requirements
ā Data Lifecycle Management
ā Generic
ā Extensible
ā Portable
ā Scalable
2BEXIS Tech Talk #1: The Big Picture
3. Requirements -> DLM
ā¢ Flexible Data Structures
ā¢ Data Submission
ā¢ Validation
ā¢ Preserving
ā¢ Metadata Management
ā¢ Versioning
3BEXIS Tech Talk #1: The Big Picture
4. Conceptual Overview
BEXIS Tech Talk #1: The Big Picture 4
DataMetadata
Data StructureMetadata Structure Semantics Geo
Administration Security
Ā«useĀ»
Ā«useĀ»
Ā«useĀ» Ā«useĀ»
Ā«useĀ»
5. Extension Method
BEXIS Tech Talk #1: The Big Picture 5
SearchPublishing
CMLand Use
Reservation
Data
Submission
6. Data Package
BEXIS Tech Talk #1: The Big Picture 6
DataMetadata
Data StructureMetadata Structure Semantics Geo
Administration Security
Ā«useĀ»
Ā«useĀ»
Ā«useĀ» Ā«useĀ»
Ā«useĀ»
7. Datasets
ā¢ A set of tuples
ā¢ Data container for
observations,
measurements,
simulations, and other
supported forms of data
ā¢ has one Data Structure
(later)
BEXIS Tech Talk #1: The Big Picture 7
Dataset
8. Versions
ā¢ Each dataset can have
multiple versions.
ā¢ processing and
citations, independent
of the following
changes.
ā¢ Check-Out, Edit, Check-
In procedure
ā¢ Blocking check-outs
ā¢ Same user check-ins
BEXIS Tech Talk #1: The Big Picture 8
Dataset
Dataset Version
1
9. Tuples
ā¢ Tuple as a collection of Data
Cells containing the Data Items
ā¢ The data tuples belong to the
versions.
ā¢ Edits, deletion, additions are
preserved
ā¢ Previous versions are
reproducible
ā¢ Differential versioning
BEXIS Tech Talk #1: The Big Picture 9
Dataset
Dataset Version
Tuple
1
{Delta Association,
Only Structured Data}
10. Data Cells
ā¢ The value of an
observation, simulation,
etc
ā¢ Single vs Multiple Value
Cell
ā¢ Auxilliary Infomation
ā sampling time
ā result time
ā description
BEXIS Tech Talk #1: The Big Picture 10
Dataset Version
Tuple
DataValue
Data Cell
+Variable Values
11. Metadata
ā¢ Each version has its own
metadata
ā¢ Datasetās metadata is
the metadata of its
latest version
BEXIS Tech Talk #1: The Big Picture 11
Dataset Version
Metadata::Metadata
1
1
12. Satging
ā¢ Stages indicate quality,
status, state in
workflows, etc.
ā¢ Versions have stages
ā¢ The stage of the latest
version is the datasetās
current stage.
BEXIS Tech Talk #1: The Big Picture 12
Dataset
Dataset VersionDataset Stage
1
10..1
1
+Current Stage 1
13. Extensions: Amendments
ā¢ Special kinds of data
cells which can be
attached to specific
tuples
ā¢ Capturing exceptional/
occasional observations
ā¢ Diff. tuples may have
diff. amendments
BEXIS Tech Talk #1: The Big Picture 13
Dataset Version
Tuple
Amendment
14. Extensions: Extended Properties
ā¢ User defined, dataset
specific attribute whose
value applies to a single
column in a single
dataset
ā¢ Sampe usage :
ā Storing the error margin
of the instrument used
to measure the values of
a variable
BEXIS Tech Talk #1: The Big Picture 14
Dataset
Dataset Version
Extended Property
Value
1
15. Extensions: Views
ā¢ Subset of a dataset
obtained by selection or
projection
ā¢ Purpose
ā Further processing, sharing
or sampling
ā Security /Digital rights
management
ā¢ Spanning view
ā View across multiple
dataset using the same
Data Structure
BEXIS Tech Talk #1: The Big Picture 15
Dataset
View
Criteria
0..1
1
+Content Selection Criteria
0..1
1
+Variable Selection Criteria
0..1
17. Data Package All Together
BEXIS Tech Talk #1: The Big Picture 17
Dataset
Dataset Version
Dataset Stage
Tuple
DataValue
Data Cell
Extended Property
Extended Property
Value
Data Structure Content DescriptorView MetadataAmendment
+Variable Values1
1
1
1
10..1
1
+Current Stage
1
{Delta Association,
Only Structured Data}
1
1
{ordered}
{At least one for Unstructured}
0..*
0..1
18. Data Structure Package
BEXIS Tech Talk #1: The Big Picture 18
DataMetadata
Data StructureMetadata Structure Semantics Geo
Administration Security
Ā«useĀ»
Ā«useĀ»
Ā«useĀ» Ā«useĀ»
Ā«useĀ»
19. Data Structure
ā¢ Defines the organization
& meaning of the data
ā¢
BEXIS Tech Talk #1: The Big Picture 19
Data Structure
20. Types of Data Structures
ā¢ Structured data has a
header information and
is in tabular form
ā¢ Unstructured data can
be of any format
BEXIS Tech Talk #1: The Big Picture 20
Data Structure
Structured DataUnstructured Data
21. Tabular data headers
ā¢ Variables act as table
headers
ā¢ There are parameters
too, auxiliary data
about a variable
ā¢ Data attributes are the
shared/ reusable parts
of the variables
BEXIS Tech Talk #1: The Big Picture 21
Data Structure
Structured DataUnstructured Data
Data Container
Data Attribute
Base Usage
Variable
+Variables
1..*
22. Dataset and Data Structure
BEXIS Tech Talk #1: The Big Picture 22
Dataset
Data Structure
Structured DataUnstructured Data
Dataset Version
1
1
23. Data Container
ā¢ Factors out reusable
elements of variables (such
as UoM, data types, and
data validation rules)
ā¢ Re-use Data Containers in
different data structures
used in different datasets
ā¢ Automatic unit conversion
functionality
ā¢ Benefits
ā Cross dataset query
ā Easier data integration
ā Enhanced data discovery
BEXIS Tech Talk #1: The Big Picture 23
Data Attribute
Data Container
Metadata Attribute
{No Extended Property}
24. Data Container in Detial
BEXIS Tech Talk #1: The Big Picture 24
Data Container
Parameter VariableMetadata Attribute
{No Extended Property}
Data Container
Data Type Unit
0..1
+Applies To
1
Data Container
Data Type Unit
0..1
+Applies To
1
Data Container
Methodology
Aggregate Function
0..1
Data Container
Methodology
Aggregate Function
0..1
Data Container Constraint
Default Value
Domain Value
Validator
Data Container
Extended Property
Globalization InfoĀ«enumerationĀ»
Measurement Scale
0..1
1
{No Duplicate}
Data Container Constraint
Default Value
Domain Value
Validator
Data Container
Extended Property
Globalization InfoĀ«enumerationĀ»
Measurement Scale
0..1
1
{No Duplicate}
Data Container
Semantic
Description
Data Container
Semantic
Description
25. Data Cells to Hold Data
BEXIS Tech Talk #1: The Big Picture 25
DataValue
Data Cell
Dataset
Dataset Version Amendment
Tuple
Base Usage
Variable
Data Structure
Structured Data
Data Container
Data Attribute
1
+Variable Values
1
+Variables
1..*
26. Data Structure Package All
Together
BEXIS Tech Talk #1: The Big Picture 26
Data Attribute
Data Container
Aggregate Function
Globalization Info
Structured Data
Semantic
Description
Data::Dataset
Value Type
Container
Reference Type
Container
Domain Value
Ā«enumerationĀ»
Measurement Scale
Constraint
Extended PropertyUnstructured Data
Data Structure
View
Default Value
Validator
Methodology
Classifier
Unit Conversion Method
Data Type
ParameterVariable
Spanning View
Common::Base
Usage
1
{No Duplicate}
0..1
0..1
{Structure Type}
+Indexer
0..1
0..1
1
+Variables
1..*
{Duplicate Control}
+Parameters
+Usage Context
1
{Data Structure Only}
1
+Applies To
+To
1
+From
1
0..1
* *
0..1
0..1
27. Metadata Package
BEXIS Tech Talk #1: The Big Picture 27
DataMetadata
Data StructureMetadata Structure Semantics Geo
Administration Security
Ā«useĀ»
Ā«useĀ»
Ā«useĀ» Ā«useĀ»
Ā«useĀ»
28. The Metadata Entity
ā¢ Metadata is all the data
gathered about a
dataset
ā¢ Belongs to a version
ā¢ Is checked-out/in with
the version
Metadata
Dataset Version
1
1
29. Metadata Values
ā¢ Metadata consists of
values, which have
definitions.
ā¢ Definitions are captured
using the same
technique as Data Cells;
āData Containersā
BEXIS Tech Talk #1: The Big Picture 29
Metadata
Metadata Attribute
Value
Dataset Version
Metadata Simple
Atribute
1
1..*
11
1
30. Metadata has structure, too
ā¢ What attributes should
be present in a
metadata and how they
are arranaged
BEXIS Tech Talk #1: The Big Picture 30
Metadata
Dataset Version
MetadataStructure
11
1
31. Metadata Package All Together
Dataset
MetadataMetadata Attribute
Value
Dataset Version
Metadata Simple
Atribute
MetadataStructure
1
11..*
1
1
1 1
32. Metadata Structure Package
BEXIS Tech Talk #1: The Big Picture 32
DataMetadata
Data StructureMetadata Structure Semantics Geo
Administration Security
Ā«useĀ»
Ā«useĀ»
Ā«useĀ» Ā«useĀ»
Ā«useĀ»
33. Metadata Structure
ā¢ Metadata structure
determines what should
be captured by the
metadata
ā¢ Different datasets may
choose to have different
metadata structures
MetadataStructure
Metadata Package
Metadata Attribute
1..*
+Parent
+Children
34. Metadata Structure
ā¢ Structures are
hierarchical making it
possible to have any sub
tree as a structure
ā¢ Structures at any level
have a collection of
packages, which are a
bunch of attributes
ā¢ Attributes can be shared
among various packages,
the same for the
packages
MetadataStructure
Metadata Package
Metadata Attribute
1..*
+Parent
+Children
35. Simple & Compound Attributes
ā¢ Simple attributes are
metadata value
specifiers
ā¢ Compound attributes
are collection of simple
and compound
attributes to build
coarser grain attributes
BEXIS Tech Talk #1: The Big Picture 35
Metadata Attribute
Metadata::Metadata
Attribute Value
Metadata Compound
Attribute
Metadata Simple
Atribute
1
2..*
37. How a package is used in a
structure
ā¢ Packages may have
their
ā Roles, e.g., an EML Party
can be the author, the
owner, etc. of a dataset.
ā Cardinalities: Min/Max
occurrences
ā Optional/Mandatory
ā ā¦
BEXIS Tech Talk #1: The Big Picture 37
MetadataStructureMetadata PackageMetadata Attribute
Metadata::MetadataMetadata::Metadata
Attribute Value
Metadata Compound
Attribute
Metadata Simple
Atribute
Metdata Package
Usage
1
+Parent
+Children
11..*
1
2..*
1..*
38. MetadataStructureMetadata PackageMetadata Attribute
Metadata::MetadataMetadata::Metadata
Attribute Value
Metadata Compound
Attribute
Metadata Simple
Atribute
Metdata Attribute
Usage
1
+Parent
+Children
11..*
1
2..*
1..*
How an attribute is used in a
package
ā¢ Attributes may have their
ā Roles, e.g., a Date attribute
can be the project start
date, publication date, etc.
ā Cardinalities: Min/Max
occurrences
ā Optional/Mandatory
ā ā¦
ā¢ In their associated
packages
BEXIS Tech Talk #1: The Big Picture 38
39. MetadataStructureMetadata PackageMetadata Attribute
Metadata::MetadataMetadata::Metadata
Attribute Value
Metadata Compound
Attribute
Metadata Simple
Atribute
Metdata Compound
Usage
1
+Parent
+Children
11..*
1
2..*
1..*
How an attribute is used in a
package
ā¢ A compound attribute
may contain other
compound and/or
simple attributes. E.g.,
Person can have name
and Address
ā¢ At least two attributes
are needed
ā¢ Cardinality and Role
playing are available.
BEXIS Tech Talk #1: The Big Picture 39
40. Mapping to external metadata
formats
ā¢ Proper for metadata
format conversion
ā¢ Import/export from/to
standard metadata
schemas
BEXIS Tech Talk #1: The Big Picture 40
MetadataStructure
Metadata PackageMetadata Attribute
Mapping Info
1..*
+Parent
+Children
1
0..*
41. Metadata Structure Package All
Together
BEXIS Tech Talk #1: The Big Picture 41
MetadataStructureMetadata PackageMetadata Attribute
MetadataMetadata Attribute
Value
Mapping Info
Dataset Version
Dataset
Metadata Compound
Attribute
Metadata Simple
Atribute
Metdata Package
Usage
Data Container
Metdata Attribute
Usage
Metdata Compound
Usage
Base Usage
1
+Parent
+Children
11
11..*
1
10..*
1
{No Extended Property}
2..*
1..*
42. Administration Package
BEXIS Tech Talk #1: The Big Picture 42
DataMetadata
Data StructureMetadata Structure Semantics Geo
Administration Security
Ā«useĀ»
Ā«useĀ»
Ā«useĀ» Ā«useĀ»
Ā«useĀ»
43. The Party Entity
ā¢ Represents individuals,
institutes, projects,
consortiums, etc.
Party
Person Organization
55. Time!
ā¢ Parties have lifetime (start/end)
ā¢ Relationships have lifetime
ā¢ Relationshipās lifetime canāt exceed neither
partiesā lifetimes
56. Party Sub-package All Together
PartyPartyRelationship
PartyRelationshipType PartyTypePartyTypePair
Status
StatusType
Locator
CustomAttribute
CustomAttributeValue
Dataset Data Plan
1 1..*
+type 1
*
+type 1
*
1
+History
*
1
*
+AllowedSource
+AllowedTarget
+1st
+2nd
1
+CurrentStatus
1
1
*
1
*
*
1
*
*
*
1
0..1
*
*
57. Data Plan
ā¢ Data Plan
ā Enforces policies
ā Determines chosen data
structures available to
the data plan users
ā¢ Each party may have a
set of plans, e.g.,
Project A uses Plan P1
to enforce open access
policy
BEXIS Tech Talk #1: The Big Picture 57
Data Structure::Data
Structure
Data::Dataset
Data Plan
Accessibility Policy
Ownership PolicyPublishing Policy Access Policy
Party::Party
*
+Proposed Data Structures
1..*
1
0..1
*
*
**
58. Semantics Package
BEXIS Tech Talk #1: The Big Picture 58
DataMetadata
Data StructureMetadata Structure Semantics Geo
Administration Security
Ā«useĀ»
Ā«useĀ»
Ā«useĀ» Ā«useĀ»
Ā«useĀ»
59. Basic Definitions
ā¢ An ontology:
ā is a formal specification of a shared
conceptualization (Tom Gruber)
ā is the study of entities and their relations in an
area of concern
BEXIS Tech Talk #1: The Big Picture 59
60. The Purpose
ā¢ Semantic Annotation of Variables, Attributes,
metadata, and data to enhance:
ā Data discovery
ā Data integration
BEXIS Tech Talk #1: The Big Picture 60
61. Ontology
ā¢ Ontology is a collection
of relationships
between some terms
ā¢ Can be hierarchical to
build sub ontologies
BEXIS Tech Talk #1: The Big Picture 61
Ontology
TermRelationship
+Sub Ontologies
62. ā¢ Relationships follow this
pattern:
Subject -> Predicate-> Object
ā¢ Terms can be anything,
but controlled by the
āTermTypeā
BEXIS Tech Talk #1: The Big Picture 62
Ontology Term
TermRelationship
Ā«enumerationĀ»
Term Type
+Sub Ontologies
1
+Root
*1
*
+predicate
1
*
+object
1
*
+subject
1
63. Data Containers Get Annotated
ā¢ Data Containers may
get annotated as a unit,
an entity, or a
characteristic, but can
be relaxed
ā¢ Data and metadata
attributes inherit the
annotation feature
ā¢ Variables, data cells,
and metadata values
are included, too.
BEXIS Tech Talk #1: The Big Picture 63
Data Container
Semantic
Description
Ontology Term
TermRelationship
+Sub Ontologies
+Chracteristic
0..1
+Entity
0..1+Root
*1
*
+predicate
1
*
+object
1
*
+subject
1
+Unit
0..1
64. Semantic Package All Together
BEXIS Tech Talk #1: The Big Picture 64
Data Attribute
Data Container
Semantic
Description
Ontology Term
TermRelationship
Ā«enumerationĀ»
Term Type
Metadata Attribute
+Sub Ontologies
1
+Chracteristic
0..1
+Entity
0..1+Root
*1
*
+predicate
1
*
+object
1
*
+subject
1
+Unit
0..1
Its just the conceptual model;
implementation may differ
65. Security Package
BEXIS Tech Talk #1: The Big Picture 65
DataMetadata
Data StructureMetadata Structure Semantics Geo
Administration Security
Ā«useĀ»
Ā«useĀ»
Ā«useĀ» Ā«useĀ»
Ā«useĀ»
66. Package Responsibilities
ā¢ Authentication
ā Internal user base
ā Single Sign-On
ā¢ Authorization
ā Access to functionalities
ā Access to data objects
ā Attribute & Expression based Az
ā¢ Auditing
ā Who, what, when, on what, etc.
BEXIS Tech Talk #1: The Big Picture 66
67. Authorization
ā¢ Permission indicates that
whether a subject is
granted a right over an
object.
ā¢ Grant is effective only
during "From" and "To"
date/ times.
ā¢ No permission means
DENY, which can be also
determined by a global
security policy.
ā¢ The Az can be completely
turned OFF!
BEXIS Tech Talk #1: The Big Picture 67
Subject
Permission
Object
1
0..*
1
0..*
68. What is a Subject
ā¢ Can be user, a group, or
a role
ā¢ Various memberships in
place
ā¢ Effective subject is
derived from the
membership graph,
knowing the user
BEXIS Tech Talk #1: The Big Picture 68
Subject
User
Role
Group
Security Role
Security User
Membership
Ā«traceĀ»
Membership
Ā«traceĀ»
69. Permission
ā¢ A Right limits the
permission to a aspect,
e.g., Read, Create,
Execute, etc.
BEXIS Tech Talk #1: The Big Picture 69
Permission
Ā«enumeratio...
Right
0..*
+Operation 1..*
70. What can be an Object
ā¢ Actions are system
functionalities
ā¢ They can be hierarchical
ā¢ Permission on parents
propagates to children
ā¢ Data are the single or
partial entities, e.g.,
dataset, metadata,
view, etc.
BEXIS Tech Talk #1: The Big Picture 70
Object
Data
Action
*
0..1
71. Complex Authorization Rules
ā¢ A logical expression of:
ā Attributes of designated
data item
ā Operators
ā Precedence
ā¢ That is evaluated at
runtime to determine
whether the
āPermissionā is granted
BEXIS Tech Talk #1: The Big Picture 71
Permission
Expression
1
0..*
72. Security Package All Together
BEXIS Tech Talk #1: The Big Picture 72
Subject
User
Role
Group Permission
Expression
Ā«enumeratio...
Right
Object
Ā«enumeratio...
Action Type
Data
Action
{XOR}
Security Role
Security User
1
1 0..*
1
0..*
0..*
+Operation 1..*
Membership
Ā«traceĀ»
Membership
*
0..1
Ā«traceĀ»
10..*
73. Geographical Information
Package
BEXIS Tech Talk #1: The Big Picture 73
DataMetadata
Data StructureMetadata Structure Semantics Geo
Administration Security
Ā«useĀ»
Ā«useĀ»
Ā«useĀ» Ā«useĀ»
Ā«useĀ»
74. Geo Package
ā¢ Outsourced to a third party system that
provides:
ā Features
ā Geometry
ā Feature Attributes
ā Security Integration
ā API Access
ā¢ Querying
ā¢ Visualizing
BEXIS Tech Talk #1: The Big Picture 74
Geographic
Description
Geometry
Feature AttributeParty::Party
Feature
Feature Attribute
Value
1
1 0..1 1
1
75. References
ā¢ Journal paper
ā¢ Datasets paper
ā¢ Brazil Presentation
ā¢ CM Model URL
ā¢ Anything more
BEXIS Tech Talk #1: The Big Picture 75
76. Outlook
Whats next in the talk series?
ā¢ The Overal Architecture
ā¢ The Database Design
BEXIS Tech Talk #1: The Big Picture 76
Extended properties are custom properties with values specific to a dataset.
Amendments are sparse, occasional data items associated to a tuple
View are projected/selected areas of datasets
Red classes come from other packages
Variables are the usage of data attributes in various data structures. They can have their own labels, value optionality, etc.
Data Container is
Red classes come from other packages
The Locator class provides a way to fins/locate/ contact the party. Address, phone, GIS, etc.
Statuses keep change logs!
To avoid status jungle! Party Types are introduced.
Person and Organization are delete candidates! Customization is managed by the āCustom Fieldsā
Custom Attributes
āOrganizationā can āEmployā āPersonā
āProjectā can āEmployā āPersonā