Diese Präsentation wurde erfolgreich gemeldet.
Wir verwenden Ihre LinkedIn Profilangaben und Informationen zu Ihren Aktivitäten, um Anzeigen zu personalisieren und Ihnen relevantere Inhalte anzuzeigen. Sie können Ihre Anzeigeneinstellungen jederzeit ändern.
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.
When RDF Alone Is Not Enough
Stephen Buxton, MarkLogic
stephe...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 2
The Path
 Triples: What is Semantics?
 Documents an...
WHAT IS SEMANTICS?
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 4
Data is stored in triples, expressed as: Subject : Pr...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 5
Triples Come in Different Formats
John livesIn London...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 6
Enter Semantics…
John livesIn IsIn EnglandLondon
Trip...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 7
Semantics Is: A New Way to Organize Data
<http://exam...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 8
Triple Store
Just a Triple Store is good when you wan...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 9
<Institution><Author>
Biology
<category>
Neuro-Biolog...
WHAT IS MARKLOGIC?
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 11
Hierarchical Era
―For your application data!"
 Appl...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 13
Triples, Documents, Data: Architecture
STORAGE LAYER...
10.3
2.82
1.48 1.39
0.73 0.55
0.19 0.17 0.13 0.13
0
2
4
6
8
10
12
Triple Store Rankings (DB-engines, March 2015)
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 16
Where Triples Come From
 The World at Large
– Linke...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 17
Where Triples Come From
 The World at Large
– Linke...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 18
Where Triples Come From
 The World at Large
– Linke...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 19
Where Triples Come From
 The World at Large
– Linke...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 20
Why MarkLogic?
 MarkLogic is an Enterprise Triple S...
TRIPLES, DOCUMENTS, DATA
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 22
Triples + Documents + Data: Complementary
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 23
Triples + Documents + Data: Complementary
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 24
Triples + Documents + Data: Complementary
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 25
Triples + Documents + Data: Intertwingled
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 26
Triples + Documents + Data: Intertwingled
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 27
Triples + Documents + Data: Intertwingled
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 28
Data Documents Triples
RDF
Enterprise Features
HA/DR...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 29
Benefits of a Triple Store
Data Documents Triples
Ju...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 30
Benefits of a Document Store
Data Documents Triples
...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 31
Benefits of a Document Store and Triple Store Combin...
SOME USE CASES
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 33
Leading Organizations Using MarkLogic Semantics
 In...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 34
Leading Organizations Using MarkLogic Semantics
 In...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 35
Link different terms that
mean the same or similar
t...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 36
APA – Sophisticated analysis for academic publishing...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 37
Leading Organizations Using MarkLogic Semantics
 In...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 38
BBC – Dynamic Semantic Publishing
For the 2012 Olymp...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 39
Leading Organizations Using MarkLogic Semantics
 In...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 40
Entertainment
Company
Entertainment Company – Semant...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 41
Talent
Kristen Wiig
Acted in
Episode 4
Anne Hathaway...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 42
Leading Organizations Using MarkLogic Semantics
 In...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 43
Agriculture Company – Semantics intelligence for R&D...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 44
Leading Organizations Using MarkLogic Semantics
 In...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 45
Energy Trader – Intelligent regulatory compliance
EN...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 46
Leading Organizations Using MarkLogic Semantics
 In...
OBJECT-BASED INTELLIGENCE
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 48
Object-based Intelligence – The world around us, in ...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 49
The Path
 What is Semantics?
 What is MarkLogic?
...
Getting Started
Read
http://info.marklogic.com
/semantics-summer
Learn
marklogic.com/training
Watch
mlwonline.marklogic.com
UNDER THE COVERS
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 52
A Look Under the Covers
 Some worked examples combi...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 53
Complementary: Better Search, Better Answers
 Featu...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 54
Complementary: Better Search, Better Answers
 Featu...
/* find the document related to the team with nickname "El Verde" */
var sem = require("/MarkLogic/semantics.xqy");
var us...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 56
Complementary: graph contains documents
 Feature: t...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 57
Complementary: graph contains documents
 Feature: t...
/* find the documents related to the player David Villa */
var userInput = "David Villa"
var bindings = { "playerNameInput...
/* iterate over the valueIterator (from sem.sparql) and build an array of values
(for display by the search app) */
var do...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 60
Intertwingled[1]: Triples annotated in a generalized...
var sparql = 'select ?name ?p ?value 
where { 
?name ?p ?value 
FILTER ( (?p=<http://example.com/earnings>) || 
?p=<http:/...
© COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 62
Intertwingled[2]: Triples embedded in document, data...
var wordText = "hat-trick" ;
// find (injury to Knee OR Hamstring) AND "hat-trick" AND at least 4 goals
var ctsQuery =
cts...
QUESTIONS, ANSWERS
Nächste SlideShare
Wird geladen in …5
×

Stephen Buxton: When RDF alone is not enough - triples, documents, and data in combination

803 Aufrufe

Veröffentlicht am

http://2015.semantics.cc/stephen-buxton

Veröffentlicht in: Daten & Analysen
  • Als Erste(r) kommentieren

  • Gehören Sie zu den Ersten, denen das gefällt!

Stephen Buxton: When RDF alone is not enough - triples, documents, and data in combination

  1. 1. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED. When RDF Alone Is Not Enough Stephen Buxton, MarkLogic stephen.buxton@marklogic.com Triples, Documents, and Data in Combination
  2. 2. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 2 The Path  Triples: What is Semantics?  Documents and Data: What is MarkLogic?  Triples and Documents, Triples and Data  Some Use Cases  Under the Covers (if there's time)
  3. 3. WHAT IS SEMANTICS?
  4. 4. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 4 Data is stored in triples, expressed as: Subject : Predicate : Object John Smith : livesIn : London London : isIn : England Query with SPARQL, gives us simple lookup .. and more! Find people who live in (a place that's in) England Semantics Is: A New Way to Organize Data RDF triples John livesIn IsIn EnglandLondon
  5. 5. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 5 Triples Come in Different Formats John livesIn London <sem:triple> <sem:subject> http://xmlns.com/foaf/0.1/name/"John"</sem:subject> <sem:predicate> http://example.org/livesIn</sem:predicate> <sem:object datatype="http://www.w3.org/2001/XMLSchema#string">"London"</sem:object> </sem:triple> { "triple" : { "subject": "http://xmlns.com/foaf/0.1/name" "John", "predicate": "http://example.org/livesIn", "object": { "value": "London", "datatype": "xs:string" } } <http://dbpedia.org/resource/John> <http://dbpedia.org/ontology/LivesIn> <http://dbpedia.org/resource/London> . Turtle JSON XML 3 IRI’s 2 IRI’s, 1 string
  6. 6. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 6 Enter Semantics… John livesIn IsIn EnglandLondon Triples Subject :Predicate :Object Semantics is a simple and elegant way to model data as facts and relationships. Semantics uses a data model called RDF that you query with SPARQL.
  7. 7. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 7 Semantics Is: A New Way to Organize Data <http://example.org/dir/js> <http://xmlns.com/foaf/0.1/firstname> "John". <http://example.org/dir/js> <http://xmlns.com/foaf/0.1/lastname> "Smith". Example of RDF SELECT ?person ?place WHERE { ?person <http://example.org/LivesIn> ?place . ?place <http://example.org/IsIn> "England" . } Example of SPARQL
  8. 8. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 8 Triple Store Just a Triple Store is good when you want to …  look up facts – model atomic facts, relationships – reference data  explore a graph – model relationships/links  combine sources – triples are easy to share, easy to combine  update some triples – easy to insert/delete/update a single fact – easy to insert/delete/update any part of the ontology (facts about the data)  use the magic of inference – simpler data modeling, data integration
  9. 9. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 9 <Institution><Author> Biology <category> Neuro-Biology <category> London <place> New York <place> Works at Located inLives inLived in Studies type of Neurology <category>Related To Specializes In <Institution> Funded By Specializes In  Reference Data  Metadata  Provenance  Modeling facts, relationships, links Triple Store as Graph More on Semantics
  10. 10. WHAT IS MARKLOGIC?
  11. 11. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 11 Hierarchical Era ―For your application data!"  Application- and hardware-specific We Are The New Generation Database Relational Era “For all your structured data!”  Normalized, tabular model  Application-independent query  User control Any Structure Era “For all your data!”  Schema-agnostic  Massive scale  Search and query  Analytics  Application services  Faster time-to-results
  12. 12. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 13 Triples, Documents, Data: Architecture STORAGE LAYER Scalability and Elasticity ACID Transactions Triple Store INTERFACE LAYER mlcpJSON, XML, RDF, Geo, Binaries REST API Graph / SPARQL QUERY LAYER JS XQuery SPARQL JavaScript XQuery SPARQLSQL INDEXES / CACHE Universal Index Geospatial Index Triple Index Triple Cache Automated Failover Reverse Index
  13. 13. 10.3 2.82 1.48 1.39 0.73 0.55 0.19 0.17 0.13 0.13 0 2 4 6 8 10 12 Triple Store Rankings (DB-engines, March 2015)
  14. 14. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 16 Where Triples Come From  The World at Large – Linked Open Data – DBpedia – GeoNames
  15. 15. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 17 Where Triples Come From  The World at Large – Linked Open Data – DBpedia – GeoNames  Facts from your domain – Proprietary company data – An industry-wide ontology such as FIBO http://www.omg.org/spec/EDMC-FIBO/BE/1.0/Beta1/PDF/
  16. 16. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 18 Where Triples Come From  The World at Large – Linked Open Data – DBpedia – GeoNames  Facts from your domain – Proprietary company data – An industry-wide ontology such as FIBO  Facts from documents – Document metadata (author, publish date, source, etc.) – Entities and events in free-flowing text
  17. 17. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 19 Where Triples Come From  The World at Large – Linked Open Data – DBpedia – GeoNames  Facts from your domain – Proprietary company data – An industry-wide ontology such as FIBO  Facts from documents – Document metadata – Facts in free-flowing text  Facts about data – Data metadata – Semantics of data Data Metadata  Provenance, source, security, bitemporal The Semantics of Data  Data Integration: I added a new dataset where old:customer <is the same as> new:ID  Better queries: I know the semantics of ownership Acme just acquired Pinky, who owns Perky so … Acme owns Perky  Simpler, more accurate data modeling: No need to (try to) represent all relationships explicitly
  18. 18. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 20 Why MarkLogic?  MarkLogic is an Enterprise Triple Store – Robust – Horizontally scalable – billions of triples per box – HA/DR features such as backup/restore, replication, automatic failover – Government-grade security  Triples + Documents + Data – Complementary – Intertwingled
  19. 19. TRIPLES, DOCUMENTS, DATA
  20. 20. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 22 Triples + Documents + Data: Complementary
  21. 21. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 23 Triples + Documents + Data: Complementary
  22. 22. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 24 Triples + Documents + Data: Complementary
  23. 23. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 25 Triples + Documents + Data: Intertwingled
  24. 24. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 26 Triples + Documents + Data: Intertwingled
  25. 25. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 27 Triples + Documents + Data: Intertwingled
  26. 26. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 28 Data Documents Triples RDF Enterprise Features HA/DR, SECURITY, ACID TRANSACTIONS, SCALABILITY & ELASTICITY JSON, XML Flexible Data Model Search & Query BUILT-IN FULL-TEXT SEARCH
  27. 27. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 29 Benefits of a Triple Store Data Documents Triples Just A Triple Store:  Store and query hundreds of billions of facts and relationships  More context for your data  Graph visualizations  Reliance on a common standard  Ability to infer new information
  28. 28. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 30 Benefits of a Document Store Data Documents Triples Just A Document Store:  Easily store heterogeneous data (transactional data, records, free-text)  Schema-agnostic for modeling freedom and avoiding ETL  Search flexibility and specificity  Fast app development
  29. 29. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 31 Benefits of a Document Store and Triple Store Combined Data Documents Triples All the benefits of each, plus:  Docs can contain triples, Triples can annotate docs, Graphs can contain docs – Faster data integration using semantics as the glue – Ideal model for reference data, metadata, provenance – Ability to run really powerful queries  Massive speed and scale  Simplicity of a single unified platform  Enterprise features (security, HA/DR, ACID transactions,…)
  30. 30. SOME USE CASES
  31. 31. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 33 Leading Organizations Using MarkLogic Semantics  Intelligent Search  Dynamic Semantic Publishing  Semantic Metadata Hub  Complex Data Integration  Compliance  Object-based Intelligence Entertainment Company Agriculture Company
  32. 32. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 34 Leading Organizations Using MarkLogic Semantics  Intelligent Search  Dynamic Semantic Publishing  Semantic Metadata Hub  Complex Data Integration  Compliance  Object-based Intelligence Entertainment Company Agriculture Company
  33. 33. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 35 Link different terms that mean the same or similar things 1 Compositional hierarchy to relate each part to the whole (―partonomy‖) 2 Engine Engine cooling Conditioner compressor gasket oil pan gasket 196,000+ Unique Vehicles … Vocabulary 1 Vocabulary 2 Vocabulary 3 Vocabulary 4 Searchable Knowledge Graph Mitchell1 – Knowledge graph for car parts
  34. 34. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 36 APA – Sophisticated analysis for academic publishing 2. Doing Sophisticated Data Analysis1. Defining Relationships in the Data Leveraging semantic data for efficient big data analytics (e.g. who cited APA, who cited those citations, and so-on) Designing an ontology (vocabulary) to manage the structure and relationships of content Author Subject Is an expert in University Went to school at Is sponsored by Company
  35. 35. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 37 Leading Organizations Using MarkLogic Semantics  Intelligent Search  Dynamic Semantic Publishing  Semantic Metadata Hub  Complex Data Integration  Compliance  Object-based Intelligence Entertainment Company Agriculture Company
  36. 36. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 38 BBC – Dynamic Semantic Publishing For the 2012 Olympics, semantics helped the BBC manage content for over 10,000 web pages with real-time updates— without hiring additional support 1. Diego Costa plays for Chelsea 2. Chelsea is in the Premier league 3. Diego plays in the Premier league Semantic Inference
  37. 37. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 39 Leading Organizations Using MarkLogic Semantics  Intelligent Search  Dynamic Semantic Publishing  Semantic Metadata Hub  Complex Data Integration  Compliance  Object-based Intelligence Entertainment Company Agriculture Company
  38. 38. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 40 Entertainment Company Entertainment Company – Semantic metadata hub Ontology Mgmt, Semantic Enrichment RDF Triple Store, Search and Query Metadata HubAssets Downstream Systems RDF Outputs Title HD Master Dates Production Date Editing Date Release Date International Date is Asset Title Character Film Series Animated Actress City Data Model Using Documents + Data + Triples Search App
  39. 39. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 41 Talent Kristen Wiig Acted in Episode 4 Anne Hathaway and Killers Part of Played Character Maharelle Sister Season 34 Segment The Lawrence Welk Show Aired on Date 10/4/08 Era Acted in Includes Part of NBC’s SNL – Intelligent (and hilarious) content delivery
  40. 40. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 42 Leading Organizations Using MarkLogic Semantics  Intelligent Search  Dynamic Semantic Publishing  Semantic Metadata Hub  Complex Data Integration  Compliance  Object-based Intelligence Entertainment Company Agriculture Company
  41. 41. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 43 Agriculture Company – Semantics intelligence for R&D Data and Research (90+ data sources) Search App Classification, Publishing, Ontology Mgmt, Semantic Enrichment RDF Triple Store, Search and Query, Indexing Semantics Intelligence Platform What is the corn yield and the underlying soil type for this set of states? Corn yield data- (state_50yr_mean_corn yld.xlsx) Geospatial boundaries SSURGO soil type data
  42. 42. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 44 Leading Organizations Using MarkLogic Semantics  Intelligent Search  Dynamic Semantic Publishing  Semantic Metadata Hub  Complex Data Integration  Compliance  Object-based Intelligence Entertainment Company Agriculture Company
  43. 43. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 45 Energy Trader – Intelligent regulatory compliance ENERGY TRADER Real-time alerts Automated reports Built-in search Trading data, market data, weather data, trade communications Trade Surveillance Platform … Trader 1: just got back, whats up Trader 2: thinking about pushing the close today if you want in Trader 1: cool, ttyl … Trader 2 ―Pushing the close‖ sameAs Trade conducted friendOf Trader 1 ―Banging the close‖ Alert
  44. 44. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 46 Leading Organizations Using MarkLogic Semantics  Intelligent Search  Dynamic Semantic Publishing  Semantic Metadata Hub  Complex Data Integration  Compliance  Object-based Intelligence Entertainment Company Agriculture Company
  45. 45. OBJECT-BASED INTELLIGENCE
  46. 46. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 48 Object-based Intelligence – The world around us, in context
  47. 47. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 49 The Path  What is Semantics?  What is MarkLogic?  Semantics and Documents, Semantics and Data  Some Use Cases  Under the Covers
  48. 48. Getting Started Read http://info.marklogic.com /semantics-summer Learn marklogic.com/training Watch mlwonline.marklogic.com
  49. 49. UNDER THE COVERS
  50. 50. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 52 A Look Under the Covers  Some worked examples combining documents, data, triples  Code is in server-side JavaScript – XQuery is another option  Examples are based on the tutorial at https://github.com/grechaw/semantics-tutorial – Last presented at the NoSQL Now! conference http://nosql2015.dataversity.net/ – Questions? Ask stephen.buxton@marklogic.com
  51. 51. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 53 Complementary: Better Search, Better Answers  Feature: call SPARQL from server-side XQuery or JavaScript  Benefit: expand search terms using SPARQL – Look up synonyms, related terms/entities, nicknames, city-country-region, etc.  Example: – user types in "La Verde" (a nickname for the Mexico national soccer team) – SPARQL expands the term to "Mexico" and searches for a document looks up players for "recommendation"; looks up Mexico flag + games in current championship + previous scores, to add to results
  52. 52. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 54 Complementary: Better Search, Better Answers  Feature: call SPARQL from server-side XQuery or JavaScript  Benefit: expand search terms using SPARQL – Look up synonyms, related terms/entities, nicknames, city-country-region, etc.  Example: – user types in "La Verde" (a nickname for the Mexico national soccer team) – SPARQL expands the term to "Mexico"; looks up players for "recommendation"; looks up Mexico flag, games in current championship, previous scores to add to results
  53. 53. /* find the document related to the team with nickname "El Verde" */ var sem = require("/MarkLogic/semantics.xqy"); var userInput = "La Verde" ; var bindings = { "nicknameInput": userInput } ; var team = sem.sparql("n prefix dbo: <http://dbpedia.org/ontology/> prefix foaf: <http://xmlns.com/foaf/0.1/> select ?teamName where{ ?team a dbo:SportsTeam . ?team foaf:nick ?nicknameInput . ?team foaf:name ?teamName . }", bindings ) /* convert the valueIterator to an array, and grab the team name */ var teamName = team.toArray()[0].teamName ; /* show the document with this team name (country name) in the id element */ cts.search( cts.jsonPropertyWordQuery( "id", teamName ) )
  54. 54. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 56 Complementary: graph contains documents  Feature: the subject or object of a triple can be a document or data in the database  Benefit: query using SPARQL, return a document or data as a result – Look up synonyms, related terms/entities, provenance, ownership, etc. – Return a document or data  Example: – user types in "David Villa" (a player name) – SPARQL finds the player and his team; returns the player and team document or data from the graph
  55. 55. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 57 Complementary: graph contains documents  Feature: the subject or object of a triple can be a document or data in the database  Benefit: query using SPARQL, return a document or data as a result – Look up synonyms, related terms/entities, provenance, ownership, etc. – Return a document or data  Example: – user types in "David Villa" (a player name) – SPARQL finds the player and his team; returns the player and team document or data from the graph
  56. 56. /* find the documents related to the player David Villa */ var userInput = "David Villa" var bindings = { "playerNameInput": userInput } ; var page = sem.sparql(' prefix dbo: <http://dbpedia.org/ontology/> prefix dbp: <http://dbpedia.org/property/> prefix foaf: <http://xmlns.com/foaf/0.1/> prefix mlpred: <http://marklogic.com/semantics/predicates/> select ?playerDocURI ?teamDocURI where { # establish this players IRI ?playerIRI a dbo:SoccerPlayer ; foaf:name ?playerNameInput . # find the document describing this player ?playerIRI mlpred:hasDoc ?playerDocURI . # find the document describing this players national team ?playerIRI a dbo:SoccerPlayer ; dbp:nationalteam ?natTeamIRI ; foaf:name ?playerNameInput . ?natTeamIRI mlpred:hasDoc ?teamDocURI . # find other related documents … }', bindings )
  57. 57. /* iterate over the valueIterator (from sem.sparql) and build an array of values (for display by the search app) */ var docsArray = [] ; for (var p of page) { docsArray.push( p.playerDocURI ) ; docsArray.push( p.teamDocURI ) ; } ; fn.doc( docsArray )
  58. 58. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 60 Intertwingled[1]: Triples annotated in a generalized way  Feature: triple storage can be annotated by XML or JSON metadata Triples metadata can be added in a completely generalized way  Benefit: query the triples with SPARQL, restrict by the context of the document Find facts, but only where the metadata matches some criteria – Provenance; dates; bitemporal; security; etc.  Example: – Show me the earnings and earnings–rank of every sportsperson, but only where the facts are from a reliable source, where we have at least 70% confidence, and they were published this year
  59. 59. var sparql = 'select ?name ?p ?value where { ?name ?p ?value FILTER ( (?p=<http://example.com/earnings>) || ?p=<http://example.com/earningsRank>) ) } order by ?name ' /* I'm only interested in a reliable source, where we have more than 70% confidence, published after Jan 2015 */ var publication = ["forbes on-line", "WSJ", "Bloomberg"] var date = xs.date("2015-01-01") var confidence = 70 var ctsQuery = cts.andQuery( [ cts.elementValueQuery( xs.QName("publication"), publication ), cts.elementRangeQuery( xs.QName("reported-date"), ">", date ), cts.elementRangeQuery( xs.QName("confidence"), ">", confidence ) ] ) /* run a SPARQL query, restricted by a cts query (a document/metadata query). */ var result = sem.sparql( sparql, [], [], ctsQuery ) result
  60. 60. © COPYRIGHT 2015 MARKLOGIC CORPORATION. ALL RIGHTS RESERVED.SLIDE: 62 Intertwingled[2]: Triples embedded in document, data  Feature: Triples can be embedded in an XML or JSON document The triple index stores the DocID  Benefit: query the triples with SPARQL, restrict by the context of the rest of the document Find facts, but only where they appear in some context; find the document where those facts appear – Show me all the people that John met: but only where that fact was found in a police report; within the last 6 months; that mentions a place within 5 miles of a training camp; and the interview notes mention an explosive device.  Example: – Show me the injuries that occurred in high-scoring games in the 2010 World Cup where the text mentions a hat-trick – Now show me the match report – did the injuries affect the match? – You can only abstract some structured information from a document!
  61. 61. var wordText = "hat-trick" ; // find (injury to Knee OR Hamstring) AND "hat-trick" AND at least 4 goals var ctsQuery = cts.andQuery( [ cts.orQuery( [ cts.tripleRangeQuery( [], sem.iri( "http://example.com/hasInjury" ), "Knee", "=" ), cts.tripleRangeQuery( [], sem.iri( "http://example.com/hasInjury" ), "Hamstring", "=" )] ), cts.wordQuery( wordText ), cts.elementRangeQuery( xs.QName("goals"), ">=", 4) ]) // Find an embedded triple, and return the document var doc = cts.search( ctsQuery ) doc
  62. 62. QUESTIONS, ANSWERS

×