This presentation is part of the course "184.742 Advanced Services Engineering" at The Vienna University of Technology, in Winter Semester 2012. Check the course at: http://www.infosys.tuwien.ac.at/teaching/courses/ase/

1. Advanced Services Engineering,
WS 2012, Lecture 6
Data marketplaces: models and concepts
Hong-Linh Truong
Distributed Systems Group,
Vienna University of Technology
truong@dsg.tuwien.ac.at
http://www.infosys.tuwien.ac.at/staff/truong
ASE WS 2012 1

2. Outline
 Data marketplaces
 Description models
 Exchange data agreement
 Data contract
ASE WS 2012 2

3. Recall – data service units in
clouds/internet
data data
data
Data service unit Data service unit Data service unit
data
People Things
Internet/Cloud
ASE WS 2012 3

4. Recall – data as a service
Data-as-a-Service – service models
Data publish/subcription Database-as-a-Service
middleware as a service (Structured/non-structured
querying systems)
Sensor-as-a-Service Storage-as-a-Service
(Basic storage functions)
deploy
Private/Public/Hybrid/Community Clouds
ASE WS 2012 4

5. Data marketplaces
 More than just DaaS
 DaaS focuses on data provisioning features
 Data marketplaces
 Multiple data providers and consumers
 Multiple DaaS
 Complex interactions among DaaS, data providers
and consumers
 Complex billing and pricing models
 Market dynamics
ASE WS 2012 5

6. Discussion time
WHAT ARE IMPORTANT ISSUES IN DATA
MARKETPLACES?
ASE WS 2012 6

7. Some important issues
DAAS DESCRIPTION MODEL
DATA AGREEMENT EXCHANGE
DATA CONTRACT
ASE WS 2012 7

8. Description Model for DaaS (1)
 State of the art:
 Providers have their own way to describe DaaS,
mainly in HTML
 Existing service description techniques are not
adequate in supporting description for DaaS
 Problems
 Service and data discovery cannot be done
automatically
 On-demand data integration, service integration, and
query optimization cannot be supported well.
 Service/data information and DaaS engineering
cannot be tied.
ASE WS 2012 8

9. Description Model for DaaS (2)
Which levels must be covered?
Here
Data resource
Data
items
Consumer
Data Data
Data items items
assets
Consumer
Data resource Data resource
Data resource Data resource
DaaS
ASE WS 2012 9

10. Description Model for DaaS – types
of information
Which types of information must be covered?
Quality of Ownership
data Price
License ....
Service
interface
Service
license
Quality of
service ....
ASE WS 2012 10

11. DEMOS – a description model for
Data-as-a-Service
Quang Hieu Vu, Tran Vu Pham, Hong
Linh Truong,, Schahram Dustdar,
Rasool Asal: DEMODS: A Description
Model for Data-as-a-Service. AINA
2012: 605-612
See prototype:
http://www.infosys.tuwien.ac.at/
prototype/SOD1/demods/
ASE WS 2012 11

12. Description model and data
marketplaces
ASE WS 2012 12

13. DEMODS – prototype (1)
ASE WS 2012 13

14. DEMODS – prototype (2)
Check: http://demodsmanagement.appspot.com/
ASE WS 2012 14

15. Discussion time
WHICH TYPES OF DAAS INFORMATION
ARE DYNAMIC? AND THEIR IMPACT ON
DESCRIPTION MODELS?
ASE WS 2012 15

16. Exchange data agreement (1)
Consumer DaaS provider Data
provider
DaaS
Consumer
DaaS
DaaS
Sensor
How they interact w.r.t. data concerns?
How their data agreements look like?
ASE WS 2012 16

17. Exchange data agreement (2)
 Lack of models and protocols for data
agreement in data marketplaces
 Constraints for data usage are not clear
 Inadequate data/service description → hindering data
selection and integration
 Existing techniques are not adequate for
dynamic data agreement exchange in data
marketplaces
Need generic exchange models suitable for different
ways of data provisioning in data marketplaces
ASE WS 2012 17

18. Data Agreement Exchange as a
Service (DAES)
 Metamodel for data agreement exchange
 Techniques for enriching and associating data
assets with agreement terms
 Interaction models for data agreement exchange
Hong Linh Truong, Schahram Dustdar, Joachim Götze, Tino Fleuren, Paul Müller, Salah-Eddine Tbahriti, Michael Mrissa,
Chirine Ghedira: Exchanging Data Agreements in the DaaS Model. APSCC 2011: 153-160
ASE WS 2012 18

19. Metamodel for data agreements
 Different
category of
agreements
 Licensing,
privacy, quality
of data
 Extensions
 Languages
 Different types
of agreements
 Different
specifications
ASE WS 2012 19

20. Associating data with data
agreements
 Solutions
 (a) directly inserting agreements into data assets
 (b) providing two-step access to agreements and data
assets
 (c) linking data agreements to the description of DaaS
 (d) linking data agreements to the message sent by
DaaS
ASE WS 2012 20

21. Possible interaction models for data
enriched with data agreements
ASE WS 2012 21

22. DAES – conceptual architecture
 Jersey, JAX-RS Restful WS Weblogic
 Using URIs to identify agreements
ASE WS 2012 22

23. DAES – managed information
 Specific applications: agreement creation, agreement validation,
agreement compatibility analysis, agreement management
 Implementation: Jersey, JAX-RS Restful WS Weblogic
ASE WS 2012 23

24. Illustrating examples – insert
agreement into data asset
 A pay-per-use consumer uses dataAPI of DaaS
search for data
 The consumer pays the use APIs
 Each call can return different types of data
Example with
People Search in
Infochimps
But a strong consequence
for data service engineering
techniques: dealing with
elastic requirements!
ASE WS 2012 24

25. Illustrating examples – link
agreements to geospatial data
 Domain-specific DaaS: different agreements for different data requests
 Vector data of geographic features via Web-Feature-Service (WFS)
 Terrain elevation data via Web-Coverage Services (WCS)
ASE WS 2012 25

26. Illustrating examples – link
agreements to geospatial data
Consumers can interpret and
reason if the data can be
used for specific purposes
ASE WS 2012 26

27. Illustrative examples – develop an
app for policy compliance (1)
ASE WS 2012 27

28. Illustrative examples – develop an
app for policy compliance (2)
Configuration
Results
ASE WS 2012 28

29. Discussion time
HOW NEAR-REALTIME DATA IMPACTS ON
DATA AGREEMENT EXCHANGE?
ASE WS 2012 29

30. Data contract
How to specific data contract?
Data resource
Data
items
Consumer
Data Data
Data items items
assets
Consumer
Data resource Data resource
Data resource Data resource
DaaS
ASE WS 2012 30

31. Data contracts
 Give a clear information about data usage
 Have a remedy against the consumer where the
circumstances are such that the acts complained
of do not
 Limit the liability of data providers in case of
failure of the provided data;
 Specify information on data delivery,
acceptance, and payment
ASE WS 2012 31

32. Data contracts
 Well-researched contracts for services but not
for DaaS and data marketplaces
 But service APIs != data APIs =! data assets
 Several open questions
 Right to use data? Quality of data in the data
agreement? Search based on data contract? Etc.
➔ Require extensible models
➔ Capture contractual terms for data contracts
➔ Support (semi-)automatic data service/data selection
techniques.
Hong-Linh Truong, Marco Comerio, Flavio De Paoli, G.R. Gangadharan, Schahram Dustdar, "Data Contracts for
Cloud-based Data Marketplaces ", International Journal of Computational Science and Engineering, 2012 Vol.7, No.4,
pp.280 - 295
ASE WS 2012 32

33. Study of main data contract terms
 Data rights
 Derivation, Collection, Reproduction, Attribution
 Quality of Data (QoD)
 Not mentioned, Not clear how to establish QoD metrics
 Regulatory Compliance
 Sarbanes-Oxley, EU data protection directive, etc.
 Pricing model
 Different models, pricing for data APIs and for data assets
 Control and Relationship
 Evolution terms, support terms, limitation of liability, etc
Most information is in human-readable form
ASE WS 2012 33

34. Data contract study
ASE WS 2012 34

35. Developing data contracts in cloud-
based data marketplaces
 Follow community-based approach for data
contract
 Propose generic structures to represent data
contract terms and abstract data contracts
 Develop frameworks for data contract applications
 Incorporate data contracts into data-as-a-service
description
 Develop data contract applications
ASE WS 2012 35

36. Community view on data contract
development
 Community users can develop:
 Term categories, term names, values, and units
 Rules for data contracts
 Common contract and contract fragments
Community users =!
novice users
ASE WS 2012 36

37. Representing data contract terms
 Contract term: (termName,termValue)
 Term name: common terms or user-specific terms
 Term value: a single value, a set, or a range
ASE WS 2012 37

38. Structuring abstract data contracts
Concrete data contracts generates
can be in RDF, XML or
JSON
Use Identifiers and
Tags for identifying
and searches
ASE WS 2012 38

39. Development of contract
applications
 Main applications:
 Data contract compatibility evaluation, data contract
composition
 Some common steps
 Extract DCTermType in TermCategoryType
 Extact comprable terms from all contracts,
- e.g., dataRight: Derivation, Composition and Reproduction
 Use evaluation rules associated with DCTermType
from from rule repositories
 Execute rules by passing comparable terms to rules
 Aggregate results
ASE WS 2012 39

40. Prototype
 RDF for representing term categories, term
names, term values, units
 Allegro Graph for storing contract knowledge
ASE WS 2012 40

41. Illustrating examples
 A large sustainability monitoring data platform
shows how green buildings are
 Real-time total and per capita of CO2 emission
of monitored building
 Open government data about CO2 per capita at
national level
 We created contracts from
 Open Data Commons Attribution License
 Open Government License
ASE WS 2012 41

42. Existing
common
knowledge
about Open
Data
Commons
ASE WS 2012 42

43. Step 2: provide OpenBuildingCO2
OpenBuildingCO2 by OpenGov for
modifying quality of government data
data and data right
Data contract for green building data
ASE WS 2012 43

44. Experiments – composing data
contract terms
ASE WS 2012 44

45. Discussion time
CAN WE AUTOMATICALLY GENERATE
DATA CONTRACTS FOR NEAR-REALTIME
DATA?
ASE WS 2012 45

46. Exercises
 Read mentioned papers
 Examine existing data marketplaces and write
DEMODS-based specification for some of them
 Develop some specific data contracts for open
government data
 Work on some algorithms for checking data
contract compatiblity
ASE WS 2012 46

47. Thanks for
your attention
Hong-Linh Truong
Distributed Systems Group
Vienna University of Technology
truong@dsg.tuwien.ac.at
http://www.infosys.tuwien.ac.at/staff/truong
ASE WS 2012 47