ISWC 2014 - Dandelion: from raw data to dataGEMs for developers

•

0 gefällt mir•1,290 views

This is the presentation showed during ISWC 2014 at Riva del Garda. The session was titled "Developers Workshop", and the focus was on how you solved practical problems for Linked Data. We presented dandelion platform and our data curation workflow, and the overall idea of dataGEM APIs.

Technologie

Dandelion: from raw data
to dataGEMs for
developers
Stefano Parmesan
Tatiana Tarasova
Ugo Scaiella
Michele Barbera

A bit of context
• SpazioDati s.r.l.
• Italian startup: Pisa & Trento
• Members of the DBpedia Association
• Manage the italian DBpedia

Goal
• Close the gap between getting the data and
using it
• Build a Knowledge Graph as-a-service:
• Make it querable
• Make it stable, make it scale
• Support different access levels

How?
• Phase #1: PUT the data in
• Data normalization
• Entity deduplication
• Phase #2: GET the data out
• Slices

How?
Data Normalisation Entity Deduplication Data Storage Data Access
Sample
Raw Data
Reconciliation Services
Source 1
Source N
Azkaban Silk
Framework Titan Graph dandelion.eu
Linked Data
Slices
dataGEM

Why…
• … slices?
• SQL-like APIs
• Common knowledge, linked data
• … a graph at all?
• Traversals
• Data is centralized
• Different sources, different access levels

Why…
• … titan/gremlin?
• Scalable
• Richer (multi-prop, undef-depth queries)
• OpenSource
• ElasticSearch powered

And now what?
• Still a prototype:
• Private beta access to slices (demo)
• English and italian DBpedia
• Corporate private data

Future?
• Phase #1b: PUT the data in
• Scalable entity deduplication
• Phase #2b: GET the data out
• API for graph traversal
• Text analysis tools (dataTXT)
• Customizations

RDF mappings
<http://data.spaziodati.eu/resource/ef4a83008d4ffd9e85c1fcf6dfe59c8758226cdb> a code:ISTATAdministrativeDivision ;
sd:childOf <http://data.spaziodati.eu/resource/7b7d45857f1372e1205bcfc87c19b2b2db2e0f59> ;
sd:code "001001" ;
sd:acheneID "ef4a83008d4ffd9e85c1fcf6dfe59c8758226cdb" ;
code:cadastralCode "A074" ;
sd:label "Agliè" ;
code:elevation "315"^^xsd:int ;
code:isCoastal "false"^^xsd:boolean ;
code:isMountainous "false"^^xsd:boolean ;
sd:level "60"^^xsd:int .
!
_:node194hhq904x1 rdf:subject <http://data.spaziodati.eu/resource/ef4a83008d4ffd9e85c1fcf6dfe59c8758226cdb> ;
rdf:predicate code:population ;
rdf:object "2574"^^xsd:int ;
sd:acheneID "31e4104e62168ffc4c3d6d278ecc775effff6ebc" ;
metaprop:validSince "2001-10-21"^^xsd:date .
!
_:node194hhq904x2 rdf:subject <http://data.spaziodati.eu/resource/ef4a83008d4ffd9e85c1fcf6dfe59c8758226cdb> ;
rdf:predicate code:population ;
rdf:object "2644"^^xsd:int ;
sd:acheneID "f38e87252cc5614faeec4abbeedd6315f5d00e9f" ;
metaprop:validSince "2011-10-09"^^xsd:date .

Graph structure
Provenance nodes
Type nodes
Bristle node
Achene node

Traversing
• v.as(‘x’).out(‘sd:childOf’)
.loop(‘x’){ cur ->
cur.outE(‘sd:childOf’).hasNext()
}.path()

Weitere ähnliche Inhalte

Was ist angesagt?

PID services - understandability and findability of data

EOSC-hub project

How to get your data into Sindice and Google with sitemap4rdf

Richard Cyganiak

ESWC 2017 Tutorial Knowledge Graphs

Peter Haase

TXDHC OpenRefine Training

Liz Grumbach

30° Nexa Lunch Seminar - Linked Data Platform vs real world

Diego Valerio Camarda

Talk di Juan Arevalo & Marco Giacomassi | Drupal Day Roma 2011 The Open Data movement is now moving a step forward, many governments, institutions and business have recently started the process of making information available to citizens and customers. Data is now seen as a powerful instrument to increase transparency in public administration and business on policies. About 80% of this information has a spatial component that is not entirely exploited yet. A range of open source solutions are now available to address this challenge, in this session we will explore their potential and possible applications. The so-called “data deluge” is here.. but we can build good umbrellas. Please come to learn more about it!

Drupal Day 2011 - Thinking spatially with your open data

DrupalDay

With the tons of bits of data around enterprises and the challenge to turn these data into knowledge, meaning is arguably in the systems of the best database holder. Turning data pieces into actionable knowledge and data-driven decisions takes a good and reliable database. The RDF database is one such solution. It captures and analyzes large volumes of diverse data while at the same time is able to manage and retrieve each and every connection these data ever get to enter in. In our latest slides, you will find out why we believe RDF graph databases work wonders with serving information needs and handling the growing amounts of diverse data every organization faces today.

It Don’t Mean a Thing If It Ain’t Got Semantics

Ontotext

(Enterprise) Linked Data Platform a new standard to manage LOD

Diego Valerio Camarda

Informal presentation about RES

Christophe Guéret

Querying the Wikidata Knowledge Graph

Ioan Toma

ODI Summit 2016 - Linked Open Data at Springer Nature

Michele Pasin

Presented by Rachel Tillay and Mike Waugh, LSU Libraries Is your data running loose in your library? OpenRefine is a tool that can help libraries more easily view, analyze, clean, and match large data sets. It is particularly useful for digital projects, statistics, or user data. This presentation will describe how OpenRefine is different from spreadsheets, datasheets, and programming. It will also include demonstrations of some of the most useful functions in OpenRefine, compatible tools, and solutions it provides to would-be data wranglers. Examples will include real-life problems that LSU Libraries has encountered in its cataloging and digital projects.

Data Wrangling with Open Refine

LOUIS Libraries

Linking Open, Big Data Using Semantic Web Technologies - An Introduction

Ronald Ashri

Linked Open (meta)Data

AIMS (Agricultural Information Management Standards)

Keynote session - LOD2014 W3C event

Diego Valerio Camarda

Iterative data discovery and transformation with open refine

Martin Magdinier

Using the Semantic Web Stack to Make Big Data Smarter

Matheus Mota

Maximising (Re)Usability of Library metadata using Linked Data

Asuncion Gomez-Perez

If you are looking for solutions that allow you not only to manage all of your data (structured, semi-structured and unstructured) but to also make the most out of them, using a common language is critical. Adding Semantic Technology to data integration is the glue that holds together all your enterprise data and their relationships in a meaningful way. Learn how you can quickly design data processing jobs and integrate massive amounts of data and see what semantic integration can do for your data and your business. www.ontotext.com

The Bounties of Semantic Data Integration for the Enterprise

Ontotext

Fairhair.ai – alan turing institute june '17 (public)

Giorgio Orsi

Was ist angesagt? (20)

PID services - understandability and findability of data

How to get your data into Sindice and Google with sitemap4rdf

ESWC 2017 Tutorial Knowledge Graphs

TXDHC OpenRefine Training

30° Nexa Lunch Seminar - Linked Data Platform vs real world

Drupal Day 2011 - Thinking spatially with your open data

It Don’t Mean a Thing If It Ain’t Got Semantics

(Enterprise) Linked Data Platform a new standard to manage LOD

Informal presentation about RES

Querying the Wikidata Knowledge Graph

ODI Summit 2016 - Linked Open Data at Springer Nature

Data Wrangling with Open Refine

Linking Open, Big Data Using Semantic Web Technologies - An Introduction

Linked Open (meta)Data

Keynote session - LOD2014 W3C event

Iterative data discovery and transformation with open refine

Using the Semantic Web Stack to Make Big Data Smarter

Maximising (Re)Usability of Library metadata using Linked Data

The Bounties of Semantic Data Integration for the Enterprise

Fairhair.ai – alan turing institute june '17 (public)

Ähnlich wie ISWC 2014 - Dandelion: from raw data to dataGEMs for developers

http://www.bigdataspain.org/2014/conference/state-of-play-data-science-on-hadoop-in-2015-keynote Machine Learning is not new. Big Machine Learning is qualitatively different: More data beats algorithm improvement, scale trumps noise and sample size effects, can brute-force manual tasks. Session presented at Big Data Spain 2014 Conference 18th Nov 2014 Kinépolis Madrid http://www.bigdataspain.org Event promoted by: http://www.paradigmatecnologico.com Slides: https://speakerdeck.com/bigdataspain/state-of-play-data-science-on-hadoop-in-2015-by-sean-owen-at-big-data-spain-2014

State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...

Big Data Spain

Watch here: https://bit.ly/3cZGCxr For their machine learning and data science projects to be successful, data scientists need access to all of the enterprise data delivered through their myriad of data models. However, gaining access to all data, integrated into a central repository has been a challenge. Often 80% of the project time is spent on these tasks. But, a virtual layer can help the data scientist speed up some of the most tedious tasks, like data exploration and analysis. At the same time, it also integrates well with the data science ecosystem. There is no need to change tools and learn new languages. The data virtualization platform helps data scientists offload these data integration tasks, allowing them to focus on advanced analytics. In this session, you will learn how data virtualization: - Provides all of the enterprise data, in real-time, and without replication - Enables data scientists to create and share multiple logical models using simple drag and drop - Provides a catalog of all business definitions, lineage, and relationships

How Data Virtualization Adds Value to Your Data Science Stack

Denodo

Large scale computing

Bhupesh Bansal

Searching Chinese Patents Presentation at Enterprise Data World

OpenSource Connections

WhereHows: Taming Metadata for 150K Datasets Over 9 Data Platforms

Mars Lan

Intro to Big Data - Spark

Sofian Hadiwijaya

Дмитрий Попович "How to build a data warehouse?"

Fwdays

Big Data Analytics: Finding diamonds in the rough with Azure

Christos Charmatzis

D.3.1: State of the Art - Linked Data and Digital Preservation

PRELIDA Project

As part of the final BETTER Hackathon, project partners prepared 4 hackathon exercises. Fraunhofer IAIS organised this exercise in conjunction with external partner MKLab ITI-CERTH (EOPEN project). This step-by-step exercise featured the setup of local Docker images on Linux OS featuring Dcoker Compose and (pre-installed) Python, SANSA, Hadoop, Apache Spark and Apache Zeppelin. It featured semantic transformation and and the use of SANSA (Scalable Semantic Analytics Stack - http://sansa-stack.net/) libraries on a sample of tweets ahead of geo-clustering. Project website (Hackathon information): https://www.ec-better.eu/pages/2nd-hackathon Github repository: https://github.com/ec-better/hackathon-2020-semanticgeoclustering

Better Hackathon 2020 - Fraunhofer IAIS - Semantic geo-clustering with SANSA

PRBETTER

Metadata & brokering - a modern approach #2

Daniele Bailo

Watch the full session: Denodo DataFest 2016 sessions: https://goo.gl/Bvmvc9 Data prep and data blending are terms that have come to prominence over the last year or two. On the surface, they appear to offer functionality similar to data virtualization…but there are important differences! In this session, you will learn: • How data virtualization complements or contrasts technologies such as data prep and data blending • Pros and cons of functionality provided by data prep, data catalog and data blending tools • When and how to use these different technologies to be most effective This session is part of the Denodo DataFest 2016 event. You can also watch more Denodo DataFest sessions on demand here: https://goo.gl/VXb6M6

Denodo DataFest 2016: Comparing and Contrasting Data Virtualization With Data...

Denodo

Watch full webinar here: https://bit.ly/3r4wEVw During this session, Denodo CTO Alberto Pan will discuss how a logical data fabric and the associated technologies of machine learning, artificial intelligence and data virtualization is the right approach to assist organizations to unify their data. He will discuss how a Logical Data Fabric reduces time to value hence increasing the overall business value of your data assets.

Product Keynote: Advancing Denodo’s Logical Data Fabric with AI and Advanced ...

Denodo

Watch full webinar here: https://buff.ly/309CZ1Y Advanced data science techniques, like machine learning, have proven an extremely useful tool to derive valuable insights from existing data. Platforms like Spark, and complex libraries for R, Python and Scala put advanced techniques at the fingertips of the data scientists. However, these data scientists spent most of their time looking for the right data and massaging it into a usable format. Data virtualization offers a new alternative to address these issues in a more efficient and agile way. Attend this webinar and learn: *How data virtualization can accelerate data acquisition and massaging, providing the data scientist with a powerful tool to complement their practice *How popular tools from the data science ecosystem: Spark, Python, Zeppelin, Jupyter, etc. integrate with Denodo *How you can use the Denodo Platform with large data volumes in an efficient way *About the success McCormick has had as a result of seasoning the Machine Learning and Blockchain Landscape with data virtualization

Minimizing the Complexities of Machine Learning with Data Virtualization

Denodo

Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...

Open Analytics

Open Data Summit Presentation by Joe Olsen

Christopher Whitaker

Bringing Deep Learning into production

Paolo Platter

Given at Data Day Seattle 2015. Bitly generates over 9 billion clicks on shortened links a month, as well as over 100 million unique link shortens. Analyzing data of this scale is not without its challenges. At Bitly, we have started adopting Apache Spark as a way to process our data. In this talk, I’ll elaborate on how I use Spark as part of my data science workflow. I’ll cover how Spark fits into our existing architecture, the kind of problems I’m solving with Spark, and the benefits and challenges of using Spark for large-scale data science.

Data Science at Scale: Using Apache Spark for Data Science at Bitly

Sarah Guido

Ravi_Shrivas_CV

Ravi Shrivas

Spark Summit EU talk by Bas Geerdink

Spark Summit

Ähnlich wie ISWC 2014 - Dandelion: from raw data to dataGEMs for developers (20)

State of Play. Data Science on Hadoop in 2015 by SEAN OWEN at Big Data Spain ...

How Data Virtualization Adds Value to Your Data Science Stack

Large scale computing

Searching Chinese Patents Presentation at Enterprise Data World

WhereHows: Taming Metadata for 150K Datasets Over 9 Data Platforms

Intro to Big Data - Spark

Дмитрий Попович "How to build a data warehouse?"

Big Data Analytics: Finding diamonds in the rough with Azure

D.3.1: State of the Art - Linked Data and Digital Preservation

Better Hackathon 2020 - Fraunhofer IAIS - Semantic geo-clustering with SANSA

Metadata & brokering - a modern approach #2

Denodo DataFest 2016: Comparing and Contrasting Data Virtualization With Data...

Product Keynote: Advancing Denodo’s Logical Data Fabric with AI and Advanced ...

Minimizing the Complexities of Machine Learning with Data Virtualization

Social Media, Cloud Computing, Machine Learning, Open Source, and Big Data An...

Open Data Summit Presentation by Joe Olsen

Bringing Deep Learning into production

Data Science at Scale: Using Apache Spark for Data Science at Bitly

Ravi_Shrivas_CV

Spark Summit EU talk by Bas Geerdink

Mehr von SpazioDati

Dandelion API e Atoka: due strumenti utili al Data Journalism

SpazioDati

SpazioDati presents Dandelion dataTXT - SenTaClAus project - final meeting

SpazioDati

SpazioDati presents dataTXT - SenTaClAus project - final meeting

SpazioDati

Opening “Big Data Challenge” data: some insights on our role in the story

SpazioDati

News Fact-checking: One Practical Application of Linked Statistics

SpazioDati

SpazioDati presents dataTXT - SenTaClAus project - 2nd open day

SpazioDati

Find the specific Wikipedia page you’re looking for, using Wikisearch API

SpazioDati

Dandelion API and mobile payment: food for thoughts for H-ACK PAYMENT

SpazioDati

Cerved Group scommette sull'analisi semantica made in Italy

SpazioDati

LinkedStat: making ISTAT data more valuable

SpazioDati

Smart Open Data Kickoff - Madrid - Linked

SpazioDati

Linked STAT per l'evento datalab con ISTAT alla Smart City Exhibition 2013

SpazioDati

Introducing JSONpedia

SpazioDati

Pubblicare Linked Open Data, lezione 1

SpazioDati

Mehr von SpazioDati (14)