SlideShare ist ein Scribd-Unternehmen logo
1 von 20
Efficiently Maintaining Distributed Model-
 Based Views on Real-Time Data Streams




     Alexandru Arion, Hoyoung Jeung, Karl Aberer
                     EPFL, 2011
Data networks


Local: low power connected devices transmit to base stations.

Large scale: base stations transmit over large distances using existing
communication infrastructure.
Relevance


Large numbers of sensor networks are already being
interconnected and share huge amount of streaming data.

Example: SwissEx (http://www.swiss-experiment.ch)
Related work
S. Shah, et all., “An efficient and resilient approach to filtering and disseminating
streaming data,” in VLDB, 2003, pp. 57–68.
Y. Zhou, et all., “Disseminating streaming data in a dynamic environment: an
adaptive and cost-based approach,” The VLDB Journal, vol. 17, no. 6, pp. 1465–
1483, 2008.
D. J. Abadi, et all., “The design of the Borealis stream processing engine,” in
CIDR, 2005, pp. 277–289.
M. Balazinska, et all., “Load management and high availability in the medusa
distributed stream processing system,” in SIGMOD, 2004, pp. 929–930.
P. Pietzuch, et all., “Network-aware operator placement for stream-
processing systems,” in ICDE, 2006, p. 49.
The framework
Key features (1)


Feature 1: reduces communication costs (does not
require any data transfer of actual streams)


Feature 2: any type of queries can be processed (all
data required for query processing is available to
consumer nodes)
Key features (2)


Feature 3: any type of model can be employed
(serves any application)


Feature 4: systematic solution that can guarantee
user-specified accuracy requirements for model-
based views.
Algorithms (1)


Coded model update:
● predetermines parameter values
● encodes them with bitmaps
● updates models efficiently sending only bitmaps
Algorithms (2)


Coded inter-variable model:
● uses correlation information
● reduces data redundancy
Framework properties

Accuracy requirements solution:
 ● The producer node generates a model-driven value when a new raw
   reading is streamed, and checks whether the difference between the
   raw value and the model-driven value stays within the error bound.
 


    ● If the difference does not exceed the error bound, no communication is
      required between the two nodes, and the consumer node generates
      values for their model-based views.
 


    ● Otherwise, the producer node reconstructs its model, so that the
      model-driven value generated from the reconstructed model does not
      exceed the error bound from the current raw reading. Next, the
      producer node updates the models at consumer nodes by sending
      new parameter values of the reconstructed model.
Coded Model Update
Coded Inter-variable Model
Coded Inter-variable Model (2)
Experiments (1)
Experiments (2)
Experiments (3)
Further related work
A. Deshpande and S. Madden, “MauveDB: supporting model-based user views in
database systems,” in SIGMOD, 2006
Y. Ahmad, O. Papaemmanouil, U. C¾ etintemel, and J. Rogers, “Simultaneous
equation systems for query processing on continuous-time data streams,” in ICDE,
2008
A. Thiagarajan and S. Madden, “Querying continuous functions in a database
system,” in SIGMOD, 2008
A. Deligiannakis, Y. Kotidis, and N. Roussopoulos, "Compressing historical
information in sensor networks,” in SIGMOD, 2004
H. Chen, J. Li, and P. Mohapatra, “RACE: time series compression with
rate adaptivity and error bound for sensor networks,” 2004
S. Gandhi, S. Nath, S. Suri, and J. Liu, “Gamps: Compressing multi
sensor data by grouping and amplitude scaling,” in SIGMOD, 2009
Conclusions


● Generic framework

● Arbitrary numerical models

● Coded model update

● Coded inter-variable model

Weitere Àhnliche Inhalte

Andere mochten auch

Mystartingpointworkbook
MystartingpointworkbookMystartingpointworkbook
Mystartingpointworkbookkelseysadlerx
 
"Data Driven World" - Microsoft, Didier Ongena
"Data Driven World" - Microsoft, Didier Ongena"Data Driven World" - Microsoft, Didier Ongena
"Data Driven World" - Microsoft, Didier OngenaCristal Events
 
freeseminar
freeseminarfreeseminar
freeseminarFaye Zasada
 
Bittarget digital marketing-campaign
Bittarget digital marketing-campaignBittarget digital marketing-campaign
Bittarget digital marketing-campaignbittarget17
 
àž„àžłàž­àž˜àžŽàžšàžČàžąàžŁàžČàžąàž§àžŽàžŠàžČ
àž„àžłàž­àž˜àžŽàžšàžČàžąàžŁàžČàžąàž§àžŽàžŠàžČàž„àžłàž­àž˜àžŽàžšàžČàžąàžŁàžČàžąàž§àžŽàžŠàžČ
àž„àžłàž­àž˜àžŽàžšàžČàžąàžŁàžČàžąàž§àžŽàžŠàžČPrae Samart
 
PresentaciĂłn1 examenfinal alicia perez
PresentaciĂłn1 examenfinal alicia perezPresentaciĂłn1 examenfinal alicia perez
PresentaciĂłn1 examenfinal alicia perezAliciaPerezRuizDiaz
 
Introduction into VIRTUAL RETAIL. by ELSE Corp- Milan, Italy
Introduction into VIRTUAL RETAIL. by ELSE Corp- Milan, ItalyIntroduction into VIRTUAL RETAIL. by ELSE Corp- Milan, Italy
Introduction into VIRTUAL RETAIL. by ELSE Corp- Milan, ItalyAndrey Golub
 
Security and Privacy in the current e-mobility charging infrastructure
Security and Privacy in the current e-mobility charging infrastructureSecurity and Privacy in the current e-mobility charging infrastructure
Security and Privacy in the current e-mobility charging infrastructureAchim Friedland
 
ICIC 2016: Business Intelligence at the Service of Leading Edge Innovation
ICIC 2016: Business Intelligence at the Service of Leading Edge InnovationICIC 2016: Business Intelligence at the Service of Leading Edge Innovation
ICIC 2016: Business Intelligence at the Service of Leading Edge InnovationDr. Haxel Consult
 
ICIC 2016: New Product Introductions FIZ Karlsruhe / STN
ICIC 2016: New Product Introductions FIZ Karlsruhe / STNICIC 2016: New Product Introductions FIZ Karlsruhe / STN
ICIC 2016: New Product Introductions FIZ Karlsruhe / STNDr. Haxel Consult
 
Eee3420 lecture08 rev2011
Eee3420 lecture08 rev2011Eee3420 lecture08 rev2011
Eee3420 lecture08 rev2011benson215
 
Omron ladder programming
Omron ladder programmingOmron ladder programming
Omron ladder programmingSuzaini Supingat
 
API and App Ecosystems - Build The Best: a deep dive
API and App Ecosystems - Build The Best: a deep diveAPI and App Ecosystems - Build The Best: a deep dive
API and App Ecosystems - Build The Best: a deep diveCisco DevNet
 

Andere mochten auch (17)

Mystartingpointworkbook
MystartingpointworkbookMystartingpointworkbook
Mystartingpointworkbook
 
"Data Driven World" - Microsoft, Didier Ongena
"Data Driven World" - Microsoft, Didier Ongena"Data Driven World" - Microsoft, Didier Ongena
"Data Driven World" - Microsoft, Didier Ongena
 
freeseminar
freeseminarfreeseminar
freeseminar
 
PlanetData Management Overview
PlanetData Management OverviewPlanetData Management Overview
PlanetData Management Overview
 
respostas 02
respostas 02respostas 02
respostas 02
 
Bittarget digital marketing-campaign
Bittarget digital marketing-campaignBittarget digital marketing-campaign
Bittarget digital marketing-campaign
 
àž„àžłàž­àž˜àžŽàžšàžČàžąàžŁàžČàžąàž§àžŽàžŠàžČ
àž„àžłàž­àž˜àžŽàžšàžČàžąàžŁàžČàžąàž§àžŽàžŠàžČàž„àžłàž­àž˜àžŽàžšàžČàžąàžŁàžČàžąàž§àžŽàžŠàžČ
àž„àžłàž­àž˜àžŽàžšàžČàžąàžŁàžČàžąàž§àžŽàžŠàžČ
 
PresentaciĂłn1 examenfinal alicia perez
PresentaciĂłn1 examenfinal alicia perezPresentaciĂłn1 examenfinal alicia perez
PresentaciĂłn1 examenfinal alicia perez
 
Trabajo diseño final luis
Trabajo diseño final luisTrabajo diseño final luis
Trabajo diseño final luis
 
Week 6
Week 6Week 6
Week 6
 
Introduction into VIRTUAL RETAIL. by ELSE Corp- Milan, Italy
Introduction into VIRTUAL RETAIL. by ELSE Corp- Milan, ItalyIntroduction into VIRTUAL RETAIL. by ELSE Corp- Milan, Italy
Introduction into VIRTUAL RETAIL. by ELSE Corp- Milan, Italy
 
Security and Privacy in the current e-mobility charging infrastructure
Security and Privacy in the current e-mobility charging infrastructureSecurity and Privacy in the current e-mobility charging infrastructure
Security and Privacy in the current e-mobility charging infrastructure
 
ICIC 2016: Business Intelligence at the Service of Leading Edge Innovation
ICIC 2016: Business Intelligence at the Service of Leading Edge InnovationICIC 2016: Business Intelligence at the Service of Leading Edge Innovation
ICIC 2016: Business Intelligence at the Service of Leading Edge Innovation
 
ICIC 2016: New Product Introductions FIZ Karlsruhe / STN
ICIC 2016: New Product Introductions FIZ Karlsruhe / STNICIC 2016: New Product Introductions FIZ Karlsruhe / STN
ICIC 2016: New Product Introductions FIZ Karlsruhe / STN
 
Eee3420 lecture08 rev2011
Eee3420 lecture08 rev2011Eee3420 lecture08 rev2011
Eee3420 lecture08 rev2011
 
Omron ladder programming
Omron ladder programmingOmron ladder programming
Omron ladder programming
 
API and App Ecosystems - Build The Best: a deep dive
API and App Ecosystems - Build The Best: a deep diveAPI and App Ecosystems - Build The Best: a deep dive
API and App Ecosystems - Build The Best: a deep dive
 

Ähnlich wie Efficiently Maintaining Distributed Model-Based Views on Real-Time Data Streams

Application-Aware Big Data Deduplication in Cloud Environment
Application-Aware Big Data Deduplication in Cloud EnvironmentApplication-Aware Big Data Deduplication in Cloud Environment
Application-Aware Big Data Deduplication in Cloud EnvironmentSafayet Hossain
 
Implementation of Automation for the Seamless Identification of Fault in Mode...
Implementation of Automation for the Seamless Identification of Fault in Mode...Implementation of Automation for the Seamless Identification of Fault in Mode...
Implementation of Automation for the Seamless Identification of Fault in Mode...ijtsrd
 
Evaluation of Different Machine.pptx
Evaluation of Different Machine.pptxEvaluation of Different Machine.pptx
Evaluation of Different Machine.pptxtariqqureshi33
 
Introduction_PPT.pptx
Introduction_PPT.pptxIntroduction_PPT.pptx
Introduction_PPT.pptxtariqqureshi33
 
Aplications for machine learning in IoT
Aplications for machine learning in IoTAplications for machine learning in IoT
Aplications for machine learning in IoTYashesh Shroff
 
Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...
Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...
Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...Luigi Vanfretti
 
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...OtĂĄvio Carvalho
 
Poster (1)
Poster (1)Poster (1)
Poster (1)Daniel Osei
 
2. visualization in data mining
2. visualization in data mining2. visualization in data mining
2. visualization in data miningAzad public school
 
Sensor Network to monitor Atmosphere for Green House and Agriculture Sciences
Sensor Network to monitor Atmosphere for Green House and Agriculture SciencesSensor Network to monitor Atmosphere for Green House and Agriculture Sciences
Sensor Network to monitor Atmosphere for Green House and Agriculture SciencesKarthik Sharma
 
Ncct Ieee Software Abstract Collection Volume 1 50+ Abst
Ncct   Ieee Software Abstract Collection Volume 1   50+ AbstNcct   Ieee Software Abstract Collection Volume 1   50+ Abst
Ncct Ieee Software Abstract Collection Volume 1 50+ Abstncct
 
JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...
JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...
JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...IEEEGLOBALSOFTTECHNOLOGIES
 
Harvesting aware energy management for time-critical wireless sensor networks
Harvesting aware energy management for time-critical wireless sensor networksHarvesting aware energy management for time-critical wireless sensor networks
Harvesting aware energy management for time-critical wireless sensor networksIEEEFINALYEARPROJECTS
 
Reliable and Efficient Data Acquisition in Wireless Sensor Network
Reliable and Efficient Data Acquisition in Wireless Sensor NetworkReliable and Efficient Data Acquisition in Wireless Sensor Network
Reliable and Efficient Data Acquisition in Wireless Sensor NetworkIJMTST Journal
 
Big data analytics
Big data analyticsBig data analytics
Big data analyticsnitesh saxena
 
Energy Efficient Optimal Paths Using PDORP-LC
Energy Efficient Optimal Paths Using PDORP-LCEnergy Efficient Optimal Paths Using PDORP-LC
Energy Efficient Optimal Paths Using PDORP-LCpaperpublications3
 
Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...
Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...
Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...IJERA Editor
 
Scalable Interconnection Network Models for Rapid Performance Prediction of H...
Scalable Interconnection Network Models for Rapid Performance Prediction of H...Scalable Interconnection Network Models for Rapid Performance Prediction of H...
Scalable Interconnection Network Models for Rapid Performance Prediction of H...Jason Liu
 
IEEE Mobile computing Title and Abstract 2016
IEEE Mobile computing Title and Abstract 2016 IEEE Mobile computing Title and Abstract 2016
IEEE Mobile computing Title and Abstract 2016 tsysglobalsolutions
 

Ähnlich wie Efficiently Maintaining Distributed Model-Based Views on Real-Time Data Streams (20)

Application-Aware Big Data Deduplication in Cloud Environment
Application-Aware Big Data Deduplication in Cloud EnvironmentApplication-Aware Big Data Deduplication in Cloud Environment
Application-Aware Big Data Deduplication in Cloud Environment
 
Implementation of Automation for the Seamless Identification of Fault in Mode...
Implementation of Automation for the Seamless Identification of Fault in Mode...Implementation of Automation for the Seamless Identification of Fault in Mode...
Implementation of Automation for the Seamless Identification of Fault in Mode...
 
Evaluation of Different Machine.pptx
Evaluation of Different Machine.pptxEvaluation of Different Machine.pptx
Evaluation of Different Machine.pptx
 
Introduction_PPT.pptx
Introduction_PPT.pptxIntroduction_PPT.pptx
Introduction_PPT.pptx
 
Aplications for machine learning in IoT
Aplications for machine learning in IoTAplications for machine learning in IoT
Aplications for machine learning in IoT
 
Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...
Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...
Model-Simulation-and-Measurement-Based Systems Engineering of Power System Sy...
 
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
Distributed Near Real-Time Processing of Sensor Network Data Flows for Smart ...
 
Poster (1)
Poster (1)Poster (1)
Poster (1)
 
2. visualization in data mining
2. visualization in data mining2. visualization in data mining
2. visualization in data mining
 
Sensor Network to monitor Atmosphere for Green House and Agriculture Sciences
Sensor Network to monitor Atmosphere for Green House and Agriculture SciencesSensor Network to monitor Atmosphere for Green House and Agriculture Sciences
Sensor Network to monitor Atmosphere for Green House and Agriculture Sciences
 
Ncct Ieee Software Abstract Collection Volume 1 50+ Abst
Ncct   Ieee Software Abstract Collection Volume 1   50+ AbstNcct   Ieee Software Abstract Collection Volume 1   50+ Abst
Ncct Ieee Software Abstract Collection Volume 1 50+ Abst
 
JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...
JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...
JAVA 2013 IEEE NETWORKING PROJECT Harvesting aware energy management for time...
 
Harvesting aware energy management for time-critical wireless sensor networks
Harvesting aware energy management for time-critical wireless sensor networksHarvesting aware energy management for time-critical wireless sensor networks
Harvesting aware energy management for time-critical wireless sensor networks
 
cv_Md_Ariful_Islam
cv_Md_Ariful_Islamcv_Md_Ariful_Islam
cv_Md_Ariful_Islam
 
Reliable and Efficient Data Acquisition in Wireless Sensor Network
Reliable and Efficient Data Acquisition in Wireless Sensor NetworkReliable and Efficient Data Acquisition in Wireless Sensor Network
Reliable and Efficient Data Acquisition in Wireless Sensor Network
 
Big data analytics
Big data analyticsBig data analytics
Big data analytics
 
Energy Efficient Optimal Paths Using PDORP-LC
Energy Efficient Optimal Paths Using PDORP-LCEnergy Efficient Optimal Paths Using PDORP-LC
Energy Efficient Optimal Paths Using PDORP-LC
 
Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...
Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...
Intrusion Detection and Countermeasure in Virtual Network Systems Using NICE ...
 
Scalable Interconnection Network Models for Rapid Performance Prediction of H...
Scalable Interconnection Network Models for Rapid Performance Prediction of H...Scalable Interconnection Network Models for Rapid Performance Prediction of H...
Scalable Interconnection Network Models for Rapid Performance Prediction of H...
 
IEEE Mobile computing Title and Abstract 2016
IEEE Mobile computing Title and Abstract 2016 IEEE Mobile computing Title and Abstract 2016
IEEE Mobile computing Title and Abstract 2016
 

Mehr von PlanetData Network of Excellence

A Contextualized Knowledge Repository for Open Data about Trentino
A Contextualized Knowledge Repository for Open Data about TrentinoA Contextualized Knowledge Repository for Open Data about Trentino
A Contextualized Knowledge Repository for Open Data about TrentinoPlanetData Network of Excellence
 
On Leveraging Crowdsourcing Techniques for Schema Matching Networks
On Leveraging Crowdsourcing Techniques for Schema Matching NetworksOn Leveraging Crowdsourcing Techniques for Schema Matching Networks
On Leveraging Crowdsourcing Techniques for Schema Matching NetworksPlanetData Network of Excellence
 
Towards Enabling Probabilistic Databases for Participatory Sensing
Towards Enabling Probabilistic Databases for Participatory SensingTowards Enabling Probabilistic Databases for Participatory Sensing
Towards Enabling Probabilistic Databases for Participatory SensingPlanetData Network of Excellence
 
Pay-as-you-go Reconciliation in Schema Matching Networks
Pay-as-you-go Reconciliation in Schema Matching NetworksPay-as-you-go Reconciliation in Schema Matching Networks
Pay-as-you-go Reconciliation in Schema Matching NetworksPlanetData Network of Excellence
 
Demo: tablet-based visualisation of transport data in Madrid using SPARQLstream
Demo: tablet-based visualisation of transport data in Madrid using SPARQLstreamDemo: tablet-based visualisation of transport data in Madrid using SPARQLstream
Demo: tablet-based visualisation of transport data in Madrid using SPARQLstreamPlanetData Network of Excellence
 
On the need for a W3C community group on RDF Stream Processing
On the need for a W3C community group on RDF Stream ProcessingOn the need for a W3C community group on RDF Stream Processing
On the need for a W3C community group on RDF Stream ProcessingPlanetData Network of Excellence
 
Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...
Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...
Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...PlanetData Network of Excellence
 
Linking Smart Cities Datasets with Human Computation: the case of UrbanMatch
Linking Smart Cities Datasets with Human Computation: the case of UrbanMatchLinking Smart Cities Datasets with Human Computation: the case of UrbanMatch
Linking Smart Cities Datasets with Human Computation: the case of UrbanMatchPlanetData Network of Excellence
 
SciQL, Bridging the Gap between Science and Relational DBMS
SciQL, Bridging the Gap between Science and Relational DBMSSciQL, Bridging the Gap between Science and Relational DBMS
SciQL, Bridging the Gap between Science and Relational DBMSPlanetData Network of Excellence
 
Scalable Nonmonotonic Reasoning over RDF Data Using MapReduce
Scalable Nonmonotonic Reasoning over RDF Data Using MapReduceScalable Nonmonotonic Reasoning over RDF Data Using MapReduce
Scalable Nonmonotonic Reasoning over RDF Data Using MapReducePlanetData Network of Excellence
 
Evolution of Workflow Provenance Information in the Presence of Custom Infere...
Evolution of Workflow Provenance Information in the Presence of Custom Infere...Evolution of Workflow Provenance Information in the Presence of Custom Infere...
Evolution of Workflow Provenance Information in the Presence of Custom Infere...PlanetData Network of Excellence
 
Abstract Access Control Model for Dynamic RDF Datasets
Abstract Access Control Model for Dynamic RDF DatasetsAbstract Access Control Model for Dynamic RDF Datasets
Abstract Access Control Model for Dynamic RDF DatasetsPlanetData Network of Excellence
 
Towards Parallel Nonmonotonic Reasoning with Billions of Facts
Towards Parallel Nonmonotonic Reasoning with Billions of FactsTowards Parallel Nonmonotonic Reasoning with Billions of Facts
Towards Parallel Nonmonotonic Reasoning with Billions of FactsPlanetData Network of Excellence
 
Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...
Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...
Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...PlanetData Network of Excellence
 

Mehr von PlanetData Network of Excellence (20)

Dl2014 slides
Dl2014 slidesDl2014 slides
Dl2014 slides
 
A Contextualized Knowledge Repository for Open Data about Trentino
A Contextualized Knowledge Repository for Open Data about TrentinoA Contextualized Knowledge Repository for Open Data about Trentino
A Contextualized Knowledge Repository for Open Data about Trentino
 
On Leveraging Crowdsourcing Techniques for Schema Matching Networks
On Leveraging Crowdsourcing Techniques for Schema Matching NetworksOn Leveraging Crowdsourcing Techniques for Schema Matching Networks
On Leveraging Crowdsourcing Techniques for Schema Matching Networks
 
Towards Enabling Probabilistic Databases for Participatory Sensing
Towards Enabling Probabilistic Databases for Participatory SensingTowards Enabling Probabilistic Databases for Participatory Sensing
Towards Enabling Probabilistic Databases for Participatory Sensing
 
Privacy-Preserving Schema Reuse
Privacy-Preserving Schema ReusePrivacy-Preserving Schema Reuse
Privacy-Preserving Schema Reuse
 
Pay-as-you-go Reconciliation in Schema Matching Networks
Pay-as-you-go Reconciliation in Schema Matching NetworksPay-as-you-go Reconciliation in Schema Matching Networks
Pay-as-you-go Reconciliation in Schema Matching Networks
 
Demo: tablet-based visualisation of transport data in Madrid using SPARQLstream
Demo: tablet-based visualisation of transport data in Madrid using SPARQLstreamDemo: tablet-based visualisation of transport data in Madrid using SPARQLstream
Demo: tablet-based visualisation of transport data in Madrid using SPARQLstream
 
On the need for a W3C community group on RDF Stream Processing
On the need for a W3C community group on RDF Stream ProcessingOn the need for a W3C community group on RDF Stream Processing
On the need for a W3C community group on RDF Stream Processing
 
Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...
Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...
Urbanopoly: Collection and Quality Assessment of Geo-spatial Linked Data via ...
 
Linking Smart Cities Datasets with Human Computation: the case of UrbanMatch
Linking Smart Cities Datasets with Human Computation: the case of UrbanMatchLinking Smart Cities Datasets with Human Computation: the case of UrbanMatch
Linking Smart Cities Datasets with Human Computation: the case of UrbanMatch
 
SciQL, Bridging the Gap between Science and Relational DBMS
SciQL, Bridging the Gap between Science and Relational DBMSSciQL, Bridging the Gap between Science and Relational DBMS
SciQL, Bridging the Gap between Science and Relational DBMS
 
CLODA: A Crowdsourced Linked Open Data Architecture
CLODA: A Crowdsourced Linked Open Data ArchitectureCLODA: A Crowdsourced Linked Open Data Architecture
CLODA: A Crowdsourced Linked Open Data Architecture
 
Scalable Nonmonotonic Reasoning over RDF Data Using MapReduce
Scalable Nonmonotonic Reasoning over RDF Data Using MapReduceScalable Nonmonotonic Reasoning over RDF Data Using MapReduce
Scalable Nonmonotonic Reasoning over RDF Data Using MapReduce
 
Data and Knowledge Evolution
Data and Knowledge Evolution  Data and Knowledge Evolution
Data and Knowledge Evolution
 
Evolution of Workflow Provenance Information in the Presence of Custom Infere...
Evolution of Workflow Provenance Information in the Presence of Custom Infere...Evolution of Workflow Provenance Information in the Presence of Custom Infere...
Evolution of Workflow Provenance Information in the Presence of Custom Infere...
 
Access Control for RDF graphs using Abstract Models
Access Control for RDF graphs using Abstract ModelsAccess Control for RDF graphs using Abstract Models
Access Control for RDF graphs using Abstract Models
 
Arrays in Databases, the next frontier?
Arrays in Databases, the next frontier?Arrays in Databases, the next frontier?
Arrays in Databases, the next frontier?
 
Abstract Access Control Model for Dynamic RDF Datasets
Abstract Access Control Model for Dynamic RDF DatasetsAbstract Access Control Model for Dynamic RDF Datasets
Abstract Access Control Model for Dynamic RDF Datasets
 
Towards Parallel Nonmonotonic Reasoning with Billions of Facts
Towards Parallel Nonmonotonic Reasoning with Billions of FactsTowards Parallel Nonmonotonic Reasoning with Billions of Facts
Towards Parallel Nonmonotonic Reasoning with Billions of Facts
 
Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...
Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...
Automation in Cytomics: A Modern RDBMS Based Platform for Image Analysis and ...
 

KĂŒrzlich hochgeladen

Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slidespraypatel2
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersThousandEyes
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking MenDelhi Call girls
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityPrincipled Technologies
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Drew Madelung
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfEnterprise Knowledge
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsEnterprise Knowledge
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024Rafal Los
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountPuma Security, LLC
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxOnBoard
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...shyamraj55
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerThousandEyes
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024Results
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationMichael W. Hawkins
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking MenDelhi Call girls
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘RTylerCroy
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptxHampshireHUG
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonAnna Loughnan Colquhoun
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slidevu2urc
 

KĂŒrzlich hochgeladen (20)

Slack Application Development 101 Slides
Slack Application Development 101 SlidesSlack Application Development 101 Slides
Slack Application Development 101 Slides
 
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for PartnersEnhancing Worker Digital Experience: A Hands-on Workshop for Partners
Enhancing Worker Digital Experience: A Hands-on Workshop for Partners
 
08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men08448380779 Call Girls In Civil Lines Women Seeking Men
08448380779 Call Girls In Civil Lines Women Seeking Men
 
Boost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivityBoost PC performance: How more available memory can improve productivity
Boost PC performance: How more available memory can improve productivity
 
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
Strategies for Unlocking Knowledge Management in Microsoft 365 in the Copilot...
 
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdfThe Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
The Role of Taxonomy and Ontology in Semantic Layers - Heather Hedden.pdf
 
IAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI SolutionsIAC 2024 - IA Fast Track to Search Focused AI Solutions
IAC 2024 - IA Fast Track to Search Focused AI Solutions
 
The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024The 7 Things I Know About Cyber Security After 25 Years | April 2024
The 7 Things I Know About Cyber Security After 25 Years | April 2024
 
Breaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path MountBreaking the Kubernetes Kill Chain: Host Path Mount
Breaking the Kubernetes Kill Chain: Host Path Mount
 
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
Neo4j - How KGs are shaping the future of Generative AI at AWS Summit London ...
 
Maximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptxMaximizing Board Effectiveness 2024 Webinar.pptx
Maximizing Board Effectiveness 2024 Webinar.pptx
 
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
Automating Business Process via MuleSoft Composer | Bangalore MuleSoft Meetup...
 
How to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected WorkerHow to Troubleshoot Apps for the Modern Connected Worker
How to Troubleshoot Apps for the Modern Connected Worker
 
A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024A Call to Action for Generative AI in 2024
A Call to Action for Generative AI in 2024
 
GenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day PresentationGenCyber Cyber Security Day Presentation
GenCyber Cyber Security Day Presentation
 
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
08448380779 Call Girls In Diplomatic Enclave Women Seeking Men
 
🐬 The future of MySQL is Postgres 🐘
🐬  The future of MySQL is Postgres   🐘🐬  The future of MySQL is Postgres   🐘
🐬 The future of MySQL is Postgres 🐘
 
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
04-2024-HHUG-Sales-and-Marketing-Alignment.pptx
 
Data Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt RobisonData Cloud, More than a CDP by Matt Robison
Data Cloud, More than a CDP by Matt Robison
 
Histor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slideHistor y of HAM Radio presentation slide
Histor y of HAM Radio presentation slide
 

Efficiently Maintaining Distributed Model-Based Views on Real-Time Data Streams

  • 1. Efficiently Maintaining Distributed Model- Based Views on Real-Time Data Streams Alexandru Arion, Hoyoung Jeung, Karl Aberer EPFL, 2011
  • 2. Data networks Local: low power connected devices transmit to base stations. Large scale: base stations transmit over large distances using existing communication infrastructure.
  • 3. Relevance Large numbers of sensor networks are already being interconnected and share huge amount of streaming data. Example: SwissEx (http://www.swiss-experiment.ch)
  • 4. Related work S. Shah, et all., “An efficient and resilient approach to filtering and disseminating streaming data,” in VLDB, 2003, pp. 57–68. Y. Zhou, et all., “Disseminating streaming data in a dynamic environment: an adaptive and cost-based approach,” The VLDB Journal, vol. 17, no. 6, pp. 1465– 1483, 2008. D. J. Abadi, et all., “The design of the Borealis stream processing engine,” in CIDR, 2005, pp. 277–289. M. Balazinska, et all., “Load management and high availability in the medusa distributed stream processing system,” in SIGMOD, 2004, pp. 929–930. P. Pietzuch, et all., “Network-aware operator placement for stream- processing systems,” in ICDE, 2006, p. 49.
  • 6. Key features (1) Feature 1: reduces communication costs (does not require any data transfer of actual streams) Feature 2: any type of queries can be processed (all data required for query processing is available to consumer nodes)
  • 7. Key features (2) Feature 3: any type of model can be employed (serves any application) Feature 4: systematic solution that can guarantee user-specified accuracy requirements for model- based views.
  • 8. Algorithms (1) Coded model update: ● predetermines parameter values ● encodes them with bitmaps ● updates models efficiently sending only bitmaps
  • 9. Algorithms (2) Coded inter-variable model: ● uses correlation information ● reduces data redundancy
  • 10. Framework properties Accuracy requirements solution: ● The producer node generates a model-driven value when a new raw reading is streamed, and checks whether the difference between the raw value and the model-driven value stays within the error bound.
  • 11.   ● If the difference does not exceed the error bound, no communication is required between the two nodes, and the consumer node generates values for their model-based views.
  • 12.   ● Otherwise, the producer node reconstructs its model, so that the model-driven value generated from the reconstructed model does not exceed the error bound from the current raw reading. Next, the producer node updates the models at consumer nodes by sending new parameter values of the reconstructed model.
  • 19. Further related work A. Deshpande and S. Madden, “MauveDB: supporting model-based user views in database systems,” in SIGMOD, 2006 Y. Ahmad, O. Papaemmanouil, U. Cž etintemel, and J. Rogers, “Simultaneous equation systems for query processing on continuous-time data streams,” in ICDE, 2008 A. Thiagarajan and S. Madden, “Querying continuous functions in a database system,” in SIGMOD, 2008 A. Deligiannakis, Y. Kotidis, and N. Roussopoulos, "Compressing historical information in sensor networks,” in SIGMOD, 2004 H. Chen, J. Li, and P. Mohapatra, “RACE: time series compression with rate adaptivity and error bound for sensor networks,” 2004 S. Gandhi, S. Nath, S. Suri, and J. Liu, “Gamps: Compressing multi sensor data by grouping and amplitude scaling,” in SIGMOD, 2009
  • 20. Conclusions ● Generic framework ● Arbitrary numerical models ● Coded model update ● Coded inter-variable model