BigDataEurope aims to maximize the societal value of big data by addressing challenges in health, food security, energy, transport, climate action, and secure societies. It develops a big data aggregator platform to integrate diverse data sources and apply analytics to help solve societal problems. The platform follows the Lambda architecture to handle both batch and real-time processing while retaining semantic meaning from data. Current activities include workshops and interest groups in specific societal challenge domains.

1. BIG DATA EUROPE
Integrating Big Data, Software & Communities
for Addressing Europe’s Societal Challenges

2. negative conotation
Big Data has often a
24-sept.-15www.big-data-europe.eu

3. Big Data in Marketing
24-sept.-15www.big-data-europe.eu

4. Big Data in Intelligence
24-sept.-15www.big-data-europe.eu

5. BigDataEurope aims to help
maximizing the societal value of Big
Data
 Health, demographic change and wellbeing;
 Food security, sustainable agriculture and forestry,
marine and maritime and inland water research, and the
Bioeconomy;
 Secure, clean and efficient energy;
 Smart, green and integrated transport;
 Climate action, environment, resource efficiency and
raw materials;
 Europe in a changing world - inclusive, innovative and
reflective societies;
 Secure societies - protecting freedom and security of
Europe and its citizens.
24-sept.-15www.big-data-europe.eu

6. The three Big Data „V“ –
Variety is often neglected
Quelle: Gesellschaft für Informatik

7. © Fraunhofer-Allianz Big Data 7
Proactive Maintenance at Rolls Royce
New Business Model integrating Sensor Data & Big Data
Analytics
Dr. Dirk Hecker
Condition Monitoring, Proactive maintenance, „Power-by-the-hour“,
as-a-service Business Model – payment modell by flight hours
Quelle: www.springboeck.ch/SR_Technics.htm
© Mark Hillary | Flickr

8. © Fraunhofer-Allianz Big Data 8
The rolling Smartphone
New Business Models for the Automotive Industry with Data
Value Chains
Dr. Dirk Hecker
Windshield wiper as rain sensors for micro wether prognosis
Automotive industry can become data provider for other industries
Quelle:GTÜ
Quelle:www.farming-simulator.com

9. © Fraunhofer-Allianz Big Data 9
Predictive Analytics
Dr. Dirk Hecker
From Business Intelligence to Big Data Analytics
Business Intelligence Monitoring Predictive Analytics
What happened
before?
What happens now?
What will happen
soon?
What should
happen?
Prescriptive Analytics
„the last Mile“
“prescriptive analytics suggests decision options on how to
take advantage of a future opportunity”
Quelle: BMW Quelle: www.7-forum.com Quelle: BMW Quelle: Volvo

10. BigDataEurope Rationale
 Show societal value of Big Data
 Lower barrrier for using big data technologies
o Required effort and resources
o Limited data science skills
o Lack of Generic Architectures, components
 Help establishing cross-
lingual/organizational/domain Data Value
Chains
o Multiple Data Sources
o Required: Integration, Harmonisation
24-sept.-15www.big-data-europe.eu

11. BigDataEurope: Objectives
24-sept.-15www.big-data-europe.eu
COORDINATION
Stakeholder
Engagement
(Requirements
Elicitation)
SUPPORT
Design, Realise,
Evaluate
Big Data Aggregator
Platform
Create and Manage
Societal Big Data
Interest Groups
Cloud-deployment
ready
Big Data Aggregator
Platform
CSA
Measure
s
Results

12. Orthogonal Dimensions of Big Data
Ecosystems
Generic Big Data Enabling Technologies
Data Value Chain
Data Generation
& Acquisition
Data Analysis &
Processing
Data Storage &
Curation
Data
Visualization &
Usage
Data-driven
Services
SocietalChallenges
DomainSpecificDataAssets&Technology
Healthcare
Food Security
Energy
Intelligent Transport
Climate & Environment
Inclusive & Reflective Societies
Secure Societies

13. Stakeholder Engagement
Cycle

14. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
Open-Source Technologies for Big Data Apps (small selection :-)
14

15. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
Big Data - Technologies
Volume
VelocityVariety
Storm
15

16. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
Groups of Technologies
Big Data
Technolog
ies
Data Storage
Technologies
Data
Processin
g
Workflow
Coordinati
on
Querying/
Processin
g
Search
Data
Export/
Import
Data
Analysis
Statistics
Text
Mining
16

17. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
Big Data Requirements
Analysis of historical dta
 Millions of entries
 Varying analysis quesitions
 Years of input data
 => Big Data Batch Processing
Interactive analysis by online queries
 Thousands of users online
 Extremely fast response time
 Super high availability
 => Big Data Databases
Analysis of actual data with low latency in
"real-time"
 React to newest trends
 Low-Latency change detection
 Real-time online monitoring
 => Big Data Stream Processing
But how to put it together ?
17

18. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
The „traditional“ Hadoop Ecosystem + NoSQL
components
a Big Data Management System
ZooKeeper
askaban
Kafka
cassandra
voldemort
MongoDB
CouchDB
elastic search
solr
lucene

19. © Fraunhofer-Institut für Intelligente Analyse- und Informationssysteme IAIS
Batch
Function
Speed
Function
Data Storage
pages with
postings
Batch View
Realtime
View
messagepassing
message passing
Application
Horizontal Scalability in the Lambda Architecture
19
> volume
> users
> users, volume
> velocity> volume, velocity

20. Blueprint of the Data Aggregator
Platform
 Follows typical Lambda Architecture
 Integrated on top of existing Big Data distribution
 + Semantic Layer (Retaining Semantics using LD
Batch Layer
Speed Layer
Data Storage
Real-time data &
Transactions …
Batch View
Real-time
View
messagepassing
message passing
Applications & Showcases
Real-time dashboards
Domain-specific BDE apps
Big Data Analytics
In-stream Mining
BDEPlatform&
Intelligence
Input data
Stream
Spatial
Social
Statistical
Temporal
Transactiona
l
Imagery

21. BDE Platform based on BigTop
Packaging Smoke testing Virtualization
Package RPMs and DEBs,
so that you can manage
and maintain your own
cluster.
Integrated smoke testing
framework
Vagrant recipes, raw
images, and docker recipes
for deploying BigData
infrastructures from zero.
24-sept.-15www.big-data-europe.eu
+ Semantic Layer - Retaining Semantics using
Linked Data

22. Data Aggregator Platform
Challenges
 Ingest semantic (RDF) and non-semantic
(CSV, JSON, XML, …) data
o Integrate various mapping techniques (R2RML,
CSV on the Web, JSON-LD)
 preserve semantics, provenance and
metadata in Big Data processing chains
o Preserve URI/IRIs
o Preserve triples
 Exploit semantics for aggregations
24-sept.-15www.big-data-europe.eu

23. Current Activities – Year#1
 2015 BDE Societal Workshops (7) Planned
o Schedule on Website
 7 W3C Interest Groups set up: Please Join!
o SC1: HEALTH https://www.w3.org/community/bde-health/join
o SC2: FOOD & AGRICULTURE https://www.w3.org/community/bde-food/
o SC3: ENERGY https://www.w3.org/community/bde-energy/
o SC4: TRANSPORT https://www.w3.org/community/bde-transport/
o SC5: CLIMATE & ENVIRONMENT https://www.w3.org/community/bde-climate/
o SC6: SOCIETIES https://www.w3.org/community/bde-societies/
o SC7: SECURITY https://www.w3.org/community/bde-secure-societies/
www.big-data-europe.eu

24. BDE Partners

25. Sören Auer
Big Data Europe Coordinator
Fraunhofer IAIS & University of Bonn
auer@cs.uni-bonn.de
Thanks
24-sept.-15www.big-data-europe.eu

26. Energy/Climate Example:
Greenshifting
24-sept.-15www.big-data-europe.eu