1.
May (updated) 2010 Product Stack

2.
Enterprise Approach

3.
Enterprise Approach <ul><li>Semantic Enterprise based on semantic Web , linked data </li></ul><ul><li>Leverage existing assets </li></ul><ul><ul><li>Data, records and instances </li></ul></ul><ul><ul><li>Taxonomies, structure and schema </li></ul></ul><ul><li>Layer semantics on to existing systems </li></ul><ul><li>Develop incrementally </li></ul><ul><li>Add sophistication, scope over time </li></ul><ul><li>Keep risks low </li></ul><ul><li>Integrate with public and Web data (“open world”) </li></ul>

4.
Linked Data <ul><li>“ Linked Data is a set of best practices for publishing and deploying instance and class data using the RDF data model, naming the data objects using uniform resource identifiers (URIs), thereby exposing the data for access via the HTTP protocol, while emphasizing data interconnections, interrelationships and context useful to both humans and machine agents.” </li></ul>

5.
Layers and Current Products

6.
Current Products <ul><li>the pivotal product; Web services middleware that provides distributed data access and federation </li></ul><ul><li>Drupal-based structured data linkage to structWSF </li></ul><ul><li>spreadsheet, JSON and XML authoring and conversion framework </li></ul><ul><li>reference set of linking subjects and basis for domain vocabularies </li></ul><ul><li>an ontology- and entity-driven information extraction and tagging system </li></ul>

7.
Fit of Current Products within Layers

8.
Existing Assets Layer

9.
Existing Assets <ul><li>These are the materials that need to be federated, made interoperable, and given a common semantics </li></ul><ul><ul><ul><ul><ul><li>structured data / databases </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>semi-structure data (XML, Web pages) </li></ul></ul></ul></ul></ul><ul><ul><ul><ul><ul><li>unstructured data (text) </li></ul></ul></ul></ul></ul>

10.
Preserving Existing Assets <ul><li>Relational databases (RDBMs) </li></ul><ul><li>Distributed structured assets </li></ul><ul><ul><li>spreadsheets </li></ul></ul><ul><ul><li>lightweight datastores </li></ul></ul><ul><li>Web pages and Web sites </li></ul><ul><li>Existing documents and text </li></ul><ul><li>Web databases and APIs </li></ul><ul><li>Other databases (RDF, OO, etc.) </li></ul>

11.
Access/Conversion Layer

12.
Conversion <ul><li>Provides in-place access to existing information </li></ul><ul><li>Translates existing formats and structures to RDF </li></ul><ul><li>Extracts structured information from unstructured text </li></ul><ul><li>Aids creation of interoperable datasets </li></ul><ul><li>Geared almost entirely to records , instances or entities (that is, basic data) </li></ul>

13.
Conversion Methods <ul><li>Relational DBs: RDB2RDF </li></ul><ul><li>RDFizers </li></ul><ul><li>Information Extraction </li></ul><ul><li>New Dataset Authoring </li></ul><ul><li>Direct Use (already in RDF) </li></ul>

14.
Relational DB Conversion <ul><li>Simple mappings of instance records to RDF </li></ul><ul><li>Methodologies well proven if kept to the instance level </li></ul><ul><li>RDB schema inform the interoperable layer (“ontologies”) </li></ul><ul><li>Relational datastores left in place </li></ul><ul><li>Record data obtained via access layer (structWSF) </li></ul>

15.
RDFizers <ul><li>General serialization or data format conversions to RDF </li></ul><ul><li>Mostly applied to: </li></ul><ul><ul><li>Standard data formats and data structs </li></ul></ul><ul><ul><li>Web content </li></ul></ul><ul><ul><li>APIs </li></ul></ul><ul><ul><li>Some legacy content </li></ul></ul><ul><li>Sometimes some minor ontology or schema mapping </li></ul><ul><li>Embodies all conversion steps to linked data </li></ul><ul><li>We have access to more than 100+ existing formats </li></ul>

16.
RDFizers – Listing 1 <ul><li>URN handlers (in addition to IRI and URI): </li></ul><ul><li>DOI </li></ul><ul><li>LSID </li></ul><ul><li>OAI </li></ul><ul><li>RDF </li></ul><ul><li>Serialization formats: </li></ul><ul><ul><li>irON </li></ul></ul><ul><ul><li>N3 </li></ul></ul><ul><ul><li>RDF/XML </li></ul></ul><ul><ul><li>Turtle </li></ul></ul><ul><li>Languages and ontologies: </li></ul><ul><ul><li>AB Meta </li></ul></ul><ul><ul><li>Annotea </li></ul></ul><ul><ul><li>APML </li></ul></ul><ul><ul><li>AtomOWL </li></ul></ul><ul><ul><li>Bibliographic Ontology </li></ul></ul><ul><ul><li>Creative Commons </li></ul></ul><ul><ul><li>EXIF </li></ul></ul><ul><ul><li>FOAF </li></ul></ul><ul><ul><li>GeoNames </li></ul></ul><ul><ul><li>GoodRelations </li></ul></ul><ul><ul><li>Java </li></ul></ul><ul><ul><li>Javadoc </li></ul></ul><ul><ul><li>MARC/MODS </li></ul></ul><ul><ul><li>Meta Standards </li></ul></ul><ul><ul><li>Music Ontology </li></ul></ul><ul><ul><li>Natural Language </li></ul></ul><ul><ul><li>Open Archives Initiative Protocol for Metadata Harvesting (OAI-PMH) </li></ul></ul><ul><ul><li>Open Geospatial </li></ul></ul><ul><ul><li>OWL </li></ul></ul><ul><ul><li>SIOC </li></ul></ul><ul><ul><li>SIOCT </li></ul></ul><ul><ul><li>SKOS </li></ul></ul><ul><ul><li>UMBEL </li></ul></ul><ul><ul><li>vCard </li></ul></ul><ul><ul><li>XML </li></ul></ul><ul><ul><li>Others </li></ul></ul><ul><li>(X)HTML pages </li></ul><ul><li>Embedded Microformats and GRDDL * (see note below): </li></ul><ul><ul><li>DC </li></ul></ul><ul><ul><li>eRDF </li></ul></ul><ul><ul><li>geoURL </li></ul></ul><ul><ul><li>Google Base </li></ul></ul><ul><ul><li>hAudio </li></ul></ul><ul><ul><li>hCalendar </li></ul></ul><ul><ul><li>hCard </li></ul></ul><ul><ul><li>hListing </li></ul></ul><ul><ul><li>hResume </li></ul></ul><ul><ul><li>hReview </li></ul></ul><ul><ul><li>HR-XML </li></ul></ul><ul><ul><li>Ning </li></ul></ul><ul><ul><li>RDFa </li></ul></ul><ul><ul><li>relLicense </li></ul></ul><ul><ul><li>SVG </li></ul></ul><ul><ul><li>XBRL </li></ul></ul><ul><ul><li>XFN </li></ul></ul><ul><ul><li>xFolk </li></ul></ul><ul><ul><li>XR-XML </li></ul></ul><ul><ul><li>XSLT </li></ul></ul><ul><li>Syndication Formats: </li></ul><ul><ul><li>Atom </li></ul></ul><ul><ul><li>OPML </li></ul></ul><ul><ul><li>OCS </li></ul></ul><ul><ul><li>RSS 1.1 </li></ul></ul><ul><ul><li>RSS 2.0 </li></ul></ul><ul><ul><li>XBEL (for bookmarks) </li></ul></ul><ul><li>REST-style Web service APIs: </li></ul><ul><ul><li>Alchemy </li></ul></ul><ul><ul><li>Amazon </li></ul></ul><ul><ul><li>Apple </li></ul></ul><ul><ul><li>Best Buy </li></ul></ul><ul><ul><li>Calais </li></ul></ul><ul><ul><li>CNet </li></ul></ul><ul><ul><li>CrunchBase </li></ul></ul><ul><ul><li>Del.icio.us </li></ul></ul><ul><ul><li>Digg </li></ul></ul><ul><ul><li>Discogs </li></ul></ul><ul><ul><li>Disqus </li></ul></ul><ul><ul><li>eBay </li></ul></ul><ul><ul><li>Facebook </li></ul></ul><ul><ul><li>Flickr </li></ul></ul><ul><ul><li>Freebase (MQL) </li></ul></ul><ul><ul><li>FriendFeed </li></ul></ul><ul><ul><li>Garmin </li></ul></ul><ul><ul><li>Get Satisfaction </li></ul></ul><ul><ul><li>Google </li></ul></ul><ul><ul><li>Google Apps </li></ul></ul><ul><ul><li>Hoover's </li></ul></ul><ul><ul><li>HTTP (raw) </li></ul></ul><ul><ul><li>ISBN DB </li></ul></ul><ul><ul><li>Last.fm </li></ul></ul><ul><ul><li>Library Thing </li></ul></ul><ul><ul><li>Magnolia </li></ul></ul><ul><ul><li>Meetup </li></ul></ul><ul><ul><li>MusicBrainz </li></ul></ul><ul><ul><li>New York Times </li></ul></ul><ul><ul><li>New York Times Campaign Finance (NYTCF) </li></ul></ul><ul><ul><li>New York Times tags </li></ul></ul>

17.
RDFizers – Listing 2 <ul><ul><li>Open Library </li></ul></ul><ul><ul><li>Open Social </li></ul></ul><ul><ul><li>Open Street </li></ul></ul><ul><ul><li>OpenLink (facets) </li></ul></ul><ul><ul><li>O'Reilly </li></ul></ul><ul><ul><li>Picasa </li></ul></ul><ul><ul><li>Radio Pop (BBC) </li></ul></ul><ul><ul><li>Rhapsody </li></ul></ul><ul><ul><li>Salesforce </li></ul></ul><ul><ul><li>Slideshare </li></ul></ul><ul><ul><li>Slidy </li></ul></ul><ul><ul><li>Technorati </li></ul></ul><ul><ul><li>Tesco </li></ul></ul><ul><ul><li>They Work For You </li></ul></ul><ul><ul><li>Twine </li></ul></ul><ul><ul><li>Twitter </li></ul></ul><ul><ul><li>Weather </li></ul></ul><ul><ul><li>Wikipedia </li></ul></ul><ul><ul><li>World Bank </li></ul></ul><ul><ul><li>Yahoo! BOSS </li></ul></ul><ul><ul><li>Yahoo! Finance </li></ul></ul><ul><ul><li>Yahoo! Maps </li></ul></ul><ul><ul><li>Yahoo! Weather </li></ul></ul><ul><ul><li>Yelp </li></ul></ul><ul><ul><li>YouTube </li></ul></ul><ul><ul><li>Zemanta </li></ul></ul><ul><ul><li>Zillow </li></ul></ul><ul><li>Files (multitude of file formats and MIME types, including): </li></ul><ul><ul><li>audio (general) </li></ul></ul><ul><ul><li>BibJSON </li></ul></ul><ul><ul><li>BibTEX and others </li></ul></ul><ul><ul><li>BitTorrent </li></ul></ul><ul><ul><li>commON </li></ul></ul><ul><ul><li>CSV </li></ul></ul><ul><ul><li>Fink </li></ul></ul><ul><ul><li>Flat files </li></ul></ul><ul><ul><li>irJSON </li></ul></ul><ul><ul><li>irXML </li></ul></ul><ul><ul><li>JPEG </li></ul></ul><ul><ul><li>JSON </li></ul></ul><ul><ul><li>images </li></ul></ul><ul><ul><li>MS Office </li></ul></ul><ul><ul><li>OpenOffice </li></ul></ul><ul><ul><li>Open Document Format </li></ul></ul><ul><ul><li>Palm </li></ul></ul><ul><ul><li>RDF123 </li></ul></ul><ul><ul><li>video </li></ul></ul><ul><ul><li>XLS </li></ul></ul><ul><ul><li>etc. </li></ul></ul><ul><li>Metadata extractors: </li></ul><ul><ul><li>CRW </li></ul></ul><ul><ul><li>DEB </li></ul></ul><ul><ul><li>EXIF </li></ul></ul><ul><ul><li>OCW </li></ul></ul><ul><ul><li>RPM </li></ul></ul><ul><ul><li>XMP </li></ul></ul><ul><li>Email formats: </li></ul><ul><ul><li>EMail </li></ul></ul><ul><ul><li>Outlook </li></ul></ul><ul><ul><li>RFC822 </li></ul></ul><ul><li>Version control and related systems: </li></ul><ul><ul><li>Bugzilla </li></ul></ul><ul><ul><li>Jira </li></ul></ul><ul><ul><li>POM </li></ul></ul><ul><ul><li>Subversion </li></ul></ul><ul><li>Other Web service frameworks: </li></ul><ul><ul><li>BPEL </li></ul></ul><ul><ul><li>WSDL </li></ul></ul><ul><ul><li>XBRL </li></ul></ul><ul><ul><li>XBEL </li></ul></ul><ul><li>Data exchange formats: </li></ul><ul><ul><li>iCalendar </li></ul></ul><ul><ul><li>LDIF </li></ul></ul><ul><ul><li>vCalendar </li></ul></ul><ul><ul><li>vCard </li></ul></ul><ul><li>Relational databases and related: </li></ul><ul><ul><li>D2RQ </li></ul></ul><ul><ul><li>D2RMAP </li></ul></ul><ul><ul><li>RDF Views </li></ul></ul><ul><li>Virtuoso VADs </li></ul><ul><li>OpenLink license files </li></ul><ul><li>Third party metadata extraction frameworks: </li></ul><ul><ul><li>Aperture </li></ul></ul><ul><ul><li>Spotlight </li></ul></ul><ul><li>Miscellaneous and other related converters: </li></ul><ul><ul><li>MPEG-7/CS -> OWL </li></ul></ul><ul><ul><li>Random </li></ul></ul><ul><ul><li>XSD -> OWL </li></ul></ul><ul><li>* GRDDL (Gleaning Resource Descriptions from Dialects of Languages) accommodates a wide variety of dialects (see one listing ) and can be combined with arbitrary transformation mechanisms (though currently mostly based on XSLTs). </li></ul>

18.
scones

19.
Information Extraction <ul><li>scones ( S ubject C oncept O r N amed E ntitie S ) is our IE tagger </li></ul><ul><li>Information extraction is applied to input Web pages and unstructured text </li></ul><ul><li>May be applied after structure extraction: </li></ul><ul><li>(often, at minimum, defluffing ) </li></ul><ul><li>Settable “window” for snippet (from # of bracketing terms to full document) </li></ul><ul><li>Extraction is performed for both: </li></ul><ul><ul><li>Entities (per Wikipedia and enterprise dictionaries) </li></ul></ul><ul><ul><li>Subject concepts (per UMBEL and domain ontologies) </li></ul></ul><ul><li>Presently in prototype </li></ul>

20.
(Named) Entities <ul><li>The places , events , people , objects , and specific things of the real world </li></ul><ul><li>Literally millions of notable instances </li></ul><ul><li>Each belongs to one or more subject concept (s) </li></ul><ul><li>Currently, the predominate basis for linked data </li></ul><ul><li>Public sources include Wikipedia and Freebase, others </li></ul><ul><li>Can be readily mixed-and-matched with private entities </li></ul>

21.
Creating New Entity Dictionaries

22.
Triangulating Information Extraction

23.
irON – i nstance r ecord and O bject N otation

24.
irON Dataset Authoring Framework <ul><li>Simple authoring and dataset creation </li></ul><ul><li>irON includes an abstract notation and vocabulary for instance records </li></ul><ul><li>Serializations available for: </li></ul><ul><ul><li>XML (irXML) </li></ul></ul><ul><ul><li>JSON (irJSON) </li></ul></ul><ul><ul><li>CSV/spreadsheets (commON) </li></ul></ul><ul><li>Notations for: </li></ul><ul><ul><li>Instance records </li></ul></ul><ul><ul><li>Schema </li></ul></ul><ul><ul><li>Datasets and metadata </li></ul></ul><ul><ul><li>Linkages to other schema </li></ul></ul>

25.
Three irON Serializations <ul><li>irXML </li></ul>irJSON commON

26.
More-or-less Interchangeable Formats

27.
structWSF

28.
structWSF <ul><li>Generally RESTful Web services middleware </li></ul><ul><li>Uniform, distributed access point </li></ul><ul><li>Provides the interoperability architecture </li></ul><ul><li>Based on canonical RDF data model </li></ul><ul><li>Dataset access orientation </li></ul><ul><li>Standard tools and services: </li></ul><ul><ul><li>User permissions and access </li></ul></ul><ul><ul><li>CRUD (create, read, update, delete) </li></ul></ul><ul><ul><li>Browse </li></ul></ul><ul><ul><li>Full-text, faceted search </li></ul></ul><ul><ul><li>Import / export </li></ul></ul><ul><ul><li>Many others </li></ul></ul>

29.
RDF and Data Federation Model

30.
Advantages of a Canonical Model <ul><li>All tools can be driven from a single data format basis </li></ul><ul><li>Single converters can link in other hubs of data forms </li></ul><ul><li>‘ Round-tripping’ thru the canonical form can bring consistency and cleanliness to inputted data </li></ul><ul><li>RDF is well-suited as the canonical form: </li></ul><ul><ul><li>Structured data </li></ul></ul><ul><ul><li>Semi-structured data </li></ul></ul><ul><ul><li>Unstructured data (after IE) </li></ul></ul><ul><ul><li>Simple-to-complex data structures </li></ul></ul><ul><ul><li>Logic and inferencing </li></ul></ul><ul><ul><li>Suitable to all input data formats </li></ul></ul><ul><ul><li>Many serializations possible </li></ul></ul>

31.
A Collaborative, Distributed Network

32.
Flexible User Access Permissions

33.
Access, APIs and Endpoints <ul><li>The resulting linked data may be exposed as: </li></ul><ul><li>APIs </li></ul><ul><li>Web services </li></ul><ul><li>SPARQL endpoints </li></ul>

34.
Ontologies Layer

35.
Ontologies <ul><li>Ontologies provide the basis for: </li></ul><ul><ul><li>Interoperating </li></ul></ul><ul><ul><li>Reconciling semantics </li></ul></ul><ul><li>Multiples may be used at any time </li></ul><ul><li>Both enterprise (internal) and external ontologies </li></ul><ul><li>Best built incrementally, with participation </li></ul><ul><li>Easily modified: OK to test and experiment </li></ul>

36.
Ontologies <ul><li>The structural relationships of concepts within a domain </li></ul><ul><li>Generally class- (or set-) oriented </li></ul><ul><li>Analogous to relational database schema, only with controlled vocabularies and exact semantics </li></ul><ul><li>Sets the structure of how to organize the actual data (“instances”) in the domain </li></ul><ul><li>Semantics and mapping techniques allow disparate ontologies to be inter-related </li></ul><ul><li>Can inference or reason over the structure </li></ul>

37.
Migrating Structure to the Ontology Layer

38.
Ontologies Layer

39.
irON

40.
irON Record Vocabulary <ul><li>irON also provides the standard instance record vocabulary for all federated records </li></ul><ul><li>Each record source has its own attributes </li></ul><ul><li>But, irON provides common descriptors: </li></ul><ul><ul><li>Useful for interoperating </li></ul></ul><ul><ul><li>Unique, Web-accessible identifiers </li></ul></ul><ul><ul><li>Standard descriptions and labels </li></ul></ul><ul><ul><li>Conventions for “driving” user interfaces and tools </li></ul></ul>

41.
UMBEL <ul><li>UMBEL ( U pper M apping and B inding E xchange L ayer) </li></ul><ul><li>20,000 defined reference points in information space </li></ul><ul><li>Means to assert what a given chunk of content is about </li></ul><ul><li>Enable similar content to be aggregated </li></ul><ul><li>Place content in context with other content </li></ul><ul><li>Aggregation points for tying in instances and entities </li></ul><ul><li>Derived and a subset of the Cyc knowledge base </li></ul><ul><li>Vocabulary basis for domain-specific subject ontologies </li></ul>

42.
Notable Ontologies and Vocabularies

43.
Management Layer

44.
Management/Federation Layer <ul><li>Management/Federation Layer handles: </li></ul><ul><ul><li>Ontology mapping, management </li></ul></ul><ul><ul><li>Queries and retrievals </li></ul></ul><ul><ul><li>All Web services </li></ul></ul><ul><ul><li>Imports and exports </li></ul></ul><ul><ul><li>Inferencing and logic </li></ul></ul><ul><ul><li>Ontology creation and expansion </li></ul></ul><ul><li>Works off of many RDF datastores </li></ul><ul><li>Has efficient, full-text indexing with faceting </li></ul><ul><li>Interface to the system is structWSF </li></ul><ul><li>Can plug into many options at the Applications Layer </li></ul><ul><ul><li>(only Drupal with conStruct SCS yet deployed) </li></ul></ul>

45.
Web-oriented Architecture

46.
Applications Layer

47.
conStruct SCS

48.
conStruct Browse Screen

49.
conStruct Capabilities <ul><li>Based on Drupal </li></ul><ul><li>Single-click ( cloud ) deployment </li></ul><ul><li>Theming </li></ul><ul><li>User and group access and management </li></ul><ul><li>Data display templates </li></ul><ul><li>General content management system (CMS) </li></ul><ul><li>Publishing RDF </li></ul><ul><li>Open source </li></ul>

50.
Re-cap

51.
Summary <ul><li>Incremental, low-risk approach to the semantic enterprise </li></ul><ul><li>Maximum leverage and re-use of existing information assets </li></ul><ul><li>Conversion and federation of all available data forms </li></ul><ul><li>Excellent uses for: </li></ul><ul><ul><li>Business intelligence </li></ul></ul><ul><ul><li>Knowledge management </li></ul></ul><ul><ul><li>Master data management modernization </li></ul></ul><ul><ul><li>Taxonomy modernization </li></ul></ul><ul><ul><li>Enterprise content integration </li></ul></ul><ul><li>All baseline products are open source </li></ul>

52.
Contacts & Information <ul><li>Michael K. Bergman </li></ul><ul><ul><li>CEO </li></ul></ul><ul><ul><li>319.621.5225 </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>blog: www.mkbergman.com </li></ul></ul><ul><li>Steve Ardire </li></ul><ul><ul><li>Senior Advisor </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul><ul><li>Frédérick Giasson </li></ul><ul><ul><li>CTO </li></ul></ul><ul><ul><li>[email_address] </li></ul></ul><ul><ul><li>blog: fgiasson.com /blog </li></ul></ul><ul><li>Web Sites </li></ul><ul><ul><li>structureddynamics.com </li></ul></ul><ul><ul><li>umbel.org </li></ul></ul><ul><ul><li>umbel.structureddynamics.com (UMBEL Web services) </li></ul></ul><ul><ul><li>citizen- dan.org (community indicator systems) </li></ul></ul><ul><ul><li>openstructs.org (open source distros + documentation) </li></ul></ul><ul><ul><li>constructscs.com (Drupal structured data system) </li></ul></ul>