In the Telecommunications sector, there are a lot of complex data sets and problems that are well suited for graph models and the use of graph databases like Neo4j.
This talk gives just some ideas on where Neo4j currently is used within the TelCo sector. If you recognize problem areas that you have, don't hesitate to contact me at peter at neotechnology dot com, we are eager to learn more and help!
1. Neo4j
And TelCo goodness
Some usecases
#neo4j
Peter Neubauer @peterneubauer
peter@neotechnology.com
Neo Technology
2. NOSQL data models
Key-value stores
Data size
Bigtable clones
Document
databases
Graph databases
Data complexity
3. The Neo4j model: Property Graph
Core abstractions:
name = “Emil”
Nodes
age = 29
sex = “yes”
Relationships between nodes
Properties on both 1 2
type = KNOWS
time = 4 years 3
type = car
vendor = “SAAB”
model = “95 Aero”
4. Agenda
The NoSQL landscape
Neo4j intro
Examples for TelCos
CDR
Routing
Social graphs
Master Data Management
Spatial and LBS
Network topology analysis
Neo4j and Android
5. Neo4j?
Most widely deployed graph db in the world
ACID, persistent, embedded/server
Robust: 24/7 production since 2003
Mature: lots of production deployments
Scalable: High Availability, Master failover
Community: ecosystem of tools, bindings, frameworks
Product: OSGi, Spatial, RDF, languages
Available under AGPLv3 and as commercial product
But the f rst one is free! For ALL use-cases.
i
Links
http://neo4j.org
http://lists.neo4j.org
6. Neo4j – Nodes, Relationships, Properties
Nodes have different properties
Matrix characters: People vs. Programs
Build structure as you go
Who loves Neo?
7. Building a node space
GraphDatabaseService graphDb = ... // Get factory
// Create Thomas 'Neo' Anderson
Node mrAnderson = graphDb.createNode();
mrAnderson.setProperty( "name", "Thomas Anderson" );
mrAnderson.setProperty( "age", 29 );
// Create Morpheus
Node morpheus = graphDb.createNode();
morpheus.setProperty( "name", "Morpheus" );
morpheus.setProperty( "rank", "Captain" );
morpheus.setProperty( "occupation", "Total bad ass" );
// Create a relationship representing that they know each other
mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );
// ...create Trinity, Cypher, Agent Smith, Architect similarly
8. Building a node space
GraphDatabaseService graphDb = ... // Get factory
Transaction tx = graphdb.beginTx();
// Create Thomas 'Neo' Anderson
Node mrAnderson = graphDb.createNode();
mrAnderson.setProperty( "name", "Thomas Anderson" );
mrAnderson.setProperty( "age", 29 );
// Create Morpheus
Node morpheus = graphDb.createNode();
morpheus.setProperty( "name", "Morpheus" );
morpheus.setProperty( "rank", "Captain" );
morpheus.setProperty( "occupation", "Total bad ass" );
// Create a relationship representing that they know each other
mrAnderson.createRelationshipTo( morpheus, RelTypes.KNOWS );
// ...create Trinity, Cypher, Agent Smith, Architect similarly
tx.commit();
9. Code (2): Traversing a node space
// Instantiate a traverser that returns Mr Anderson's friends
Traverser friendsTraverser = mrAnderson.traverse(
Traverser.Order.BREADTH_FIRST,
StopEvaluator.END_OF_GRAPH,
ReturnableEvaluator.ALL_BUT_START_NODE,
RelTypes.KNOWS,
Direction.OUTGOING );
// Traverse the node space and print out the result
System.out.println( "Mr Anderson's friends:" );
for ( Node friend : friendsTraverser )
{
System.out.printf( "At depth %d => %s%n",
friendsTraverser.currentPosition().getDepth(),
friend.getProperty( "name" ) );
}
10. Ruby
gem install neo4j
require ”rubygems”
require 'neo4j'
class Person
include Neo4j::NodeMixin
property :name, :age, :occupation
index :name
has_n :friends
end
Neo4j::Transactoin.run do
neo = Person.new :name=>'Neo', :age=>29
morpheus = Person.new :name=>'Morpheus', :occupation=>'badass'
neo.friends << morpheus
end
neo.friends.each {|p|...}
11. Spatial and social data
name = ...
name = “The Tavern”
lat = 1295238237
name = “Omni Hotel”
long = 234823492 42
lat = 3492848
long = 283823423 length = 7 miles AD
RO
ROAD ROAD ROO
1 7 3 OAD
RO
13
AD name = ...
ROAD
lat, long = ...
name = “Swedland”
lat = 23410349
length = 3 miles long = 2342348852
2
name = ...
12. Financial data – fraud detection
name = ...
name = “The Tavern”
lat = 1295238237
long = 234823492 42
name = “Mr Godfather” AW
karma = veeeery-low
HDR
cash = more-than-you amount = $1000
IT
W
OWNS TRANSFER WIT
1 7 3 HDR
AW
13
S FE R
DE name = “Emil”
P OS cash = always-too-li'l
IT TRAN
title = “ATM @ Wall St”
id = 230918484233
amount = $1000 cash_left = 384204
2
name = ...
13. CDR analysis
name = flat_business
name = “Bob”
lat = 1295238237
long = 234823492 N 42
LA
name = “Mr Godfather”
karma = veeeery-low _ P
cash = more-than-you time = 2min TA
DA
CALLED CALLED CAL
1 7 3 L ED
CA
13
L name = “Emil”
LE
Y
D cash = always-too-li'l
FAMIL
Name= customer_support
time =13min
2
name = Alice
14. Call Data Records (CDR)
Forming a graph
Location based
Possible uses:
Find clusters (better plans)
Build social connections
Find influencers
20. Recommendations and big graphs
Global heuristics
Page rank
Local recommendations
Shortest paths
Hammock functions
Random walks
Dijkstra, A*, Shooting star etc
24. Spatial
Complex data
Multiple indexing (domain, Spatial, temporal)
Location entering many domains
GIS going mainstream, topologies explode
No good systems out there
Proprietary stacks rule (ESRI, Oracle)
Open Government Data
Shapefiles suck.
25. Current challenges in Spatial
Domain and Spatial interconnections
Unstructured domain data
Routing
Topology handling
No good OSS full GIS stack
28. The OpenStreetMap dataset
Wiki for Spatial info
Freely available data
Very unstructured, free tagging
Points, Ways, Relations, Tags, Changesets
Changes can be pushed back
Used for other purposes
Great coverage in interesting places (towns, disasters etc)
36. Network Topology analysis
Analytics of network coverage and frequencies
Cell towers
Drive data
Infrastructure
Analytics
Spatial signal strength
Antenna placement and azimuth
Frequency planning
Network differences over time
Reporting and charting
37. Neo4j and Android
Small footprint
Running almost unmodified
Interesting scenarios
Semantic homescreen
Connected devices
Local caches
Disconnected databases
38. Connected devices
Emil
Emils PC
Marcus
Leigh
Johan
“Link all pictures that any of my friends has taken of me into
my PicturesOfMe folder and update this every hour”
40. How ego are you? (aka other impls?)
Franz’ Alle groGraph (http://agraph.franz.com)
Proprietary, Lisp, RDF-oriented but real graphdb
Sones graphDB (http://sones.com)
.NET, Deutsche Telekom VC backed
Twitter's Floc kDB (http://github.com/twitter/flockdb)
Twitter's (graph) database for large and shallow graphs
Google Pre ge l (http://bit.ly/dP9IP)
We are oh-so-secret
Objectivity's Infnite Graph (http://infinitegraph.com)
New, closed OODB with Graph Layer on top
41. API References
Wiki, Code, API references
http://wiki.neo4j.org/content/Neo4j_Spatial
http://github.com/neo4j/neo4j-spatial
http://components.neo4j.org/neo4j-spatial
Mailing list: neo4j@lists.neo4j.org
http://neo4j.org/community/list/