BDE review: Scope and Opportunities (Prof. Maria-Ester Vidal, Franhofer IAIS) at the BigDataEurope Workshop, Amsterdam, Novermber 2017

1. BIG DATA EUROPE PROJECT
SCOPE AND OPPORTUNITIES
BDE CONSORTIUM
SC3 Final Workshop28.11.2017

2. Stakeholder Engagement & Activities
M12M6 M18 M24 M30
Coordination & Support Action 2015-17

3. BDE Partners

4. Where we started…
25-janv.-18www.big-data-europe.eu

5. Importance of Volume Importance of Velocity
Stakeholder Requirements (I)

6. Importance of Variety Efficiency of Data Infrastructures
Stakeholder Requirements (II)

7. Societal Data Value Chain Requirements

8. Major Results
25-janv.-18www.big-data-europe.eu

9. Watch:
big-data-europe.eu/webinar/
7 Societal Communities
 Bringing Communities closer
o Understanding common challenges & needs
o Defining Generic Big Data Architecture
 Community Activities
o Yearly workshops (1 per SC)
o Hangouts (SC- specific), Webinars (technical, general)
 Identifying 7 Pilots, showcase value of BD Platform
25-janv.-18www.big-data-europe.eu

10. BDE Integrator Platform Released!
 Stacks Open Source solutions (Free)
 Dockerization
 Facilitates integration and deployment
 Plug-and-play BD Platform
 Key BDE additions
o Support layer: integrated UI
o Semantification layer
 Final Release:
o November 2017
25-janv.-18www.big-data-europe.eu

11. 7 Integrator Platform Instances
25-janv.-18www.big-data-europe.eu
www.big-data-europe.eu/pilot/
SC1: Health
SC2: Food&
Agriculture
SC4: Transport
SC7: Security
SC6: Social Sciences
SC5: Climate
SC3: Energy

12. Moving Forward…
25-janv.-18www.big-data-europe.eu

13. Big Data Platform
 Open-Source Community
o Maintenance of BD components remains independent
o Commitment from core BDE consortium team
o Maintenance driven by ‘users’
 Users include follow-up projects:
o BigDataOcean, BETTER, NextGEOSS, Special, iASiS
 Planned series of ‘Semantics in Big Data’ workshop
 Standardisation efforts within BDVA
25-janv.-18www.big-data-europe.eu

14. Data Protection & Security Engineering &
DevOps
Standards
Data Processing Architectures
Data Visualisation and User Interaction
Data Analytics
Data Management
Infrastructure Cloud, Communication(5G), HPC, IoT/CPS
BigDataPriorityTechAreas Sectors, Applications, Cross-cutting functions
Builds on
ForecastingSANSA
Ontario
Positioning BigDataEurope into BDVA Ref. Model
Resource
Management
Layer (Swarm)
Zeppelin
Support Layer
Init Daemon
GUIs
Monitor
App Layer
Forecasting Satellite Image Analysis
Platform Layer
Spark Flink Semantic Layer
Ontario SANSA Semagrow
Kafka
Real-time Stream Monitoring
...
...
Resource Management Layer (Swarm)
Hardware Layer
Premises Cloud (AWS, GCE, MS Azure, …)
Data Layer
Hadoop NOSQL Store CassandraElasticsearch ...RDF Store

15. Questions & Contacts
www.big-data-europe.eu
25-janv.-18
#BigDataEurope
BIG DATA INTEGRATOR
www.github.com/big-data-europe
PILOT DESCRIPTIONS
https://www.big-data-europe.eu/pilot/
PROJECT COORDINATION (Fraunhofer IAIS)
Prof. Sören Auer, auer © cs.uni-bonn · de
Dr. Simon Scerri, scerri © cs.uni-bonn · de
Prof. Maria-Esther Vidal, maria.vidal © tib · eu
EIS Department/Group,
Fraunhofer IAIS & CS Department Uni-Bonn, Bonn, Germany
Input: Requirements,
Technical, Standardisation
Leads Fraunhofer
Big Data Alliance