SlideShare ist ein Scribd-Unternehmen logo
1 von 37
Downloaden Sie, um offline zu lesen
Using Graph Databases in Real-Time to
Solve Resource Authorization at Telenor

Graph Connect London – 19 Nov 2013
by Sebastian Verheughe
Telenor Norway
Subsidiary of the Telenor Group
~1 billion GBP in mobile revenues 2012

Sebastian Verheughe
Lead Developer for Neo4j solution
Coding Architect
Disclaimer
The presentation is not identical to the implementation
due to security reasons but shows how we have
modeled and solved the problem in general.
However, all presented data (numbers & charts) are
real, unfiltered and extracted from the production logs
A very

aspect of
our business
Telenor Norway Middleware Services
Channel

Channel

used by 42 channels
calls 35 sub-systems
10,000 code classes
500 requests/second
20,000 orders/day

Backend

Backend

Channel

Channel

MOBILE

MW

Channel

Channel

Providing business logic
and data for all channels
in the mobile value chain

BUSINESS

LOGIC
& DATA

Backend

Handles users with
access to X00,000
resources

Backend

Backend

Backend
Our Problem
20 minutes to calculate all accessible resources
1500 lines of SQL to implement the authorization logic
“solved” by caching data going stale
and the solution did not scale…
Why a Graph Database?
Access

Parent Company

User: R. Branson

Which resources
does the user
have access to?

Part of Company

Sales

Finance Production

HR

Sub

The questions we wanted answered
required traversal of tree structures.

Tablet

Subscription
Owner

Sub

Tablet

Uses
Subscription

Phone
Tailored Read Model
The Model makes read queries
as simple and efficient as possible.
First find your questions
then model your graph

graph model

=

relational model
High Level Architecture
Clients

Classic MW
Services

other
sources

tx log
RDBMS

check
access

Resource
Authorization

Message Queue

Neo4j
Conditional Rules
ACCESS is given with the following include parameters:
access to subsidiaries and access to content
Only find children of PARENT COMPANY
given access to subsidiaries is allowed
User

Only look at PART OF COMPANY
given access to content is allowed

Only look at SUBSCRIPTION OWNER
given access to content is allowed
Different Access Needs
Access Subsidiaries & Content

Super Admin

Access Content

Umbrella Admin
Access S&C

Admin
Graph Algorithm
Prerequisite: The user node
1. Follow all ACCESS relationships and
read the access parameters on the relationship
2. Follow all PARENT COMPANY relationships given
access to subsidiaries is allowed

3. Follow all PART OF COMPANY relationships given
access to content is allowed
4. Follow all SUBSCRIPTION OWNER relationships given
access to content is allowed
Solution Value
1. Performance optimized from minutes to seconds.
2. Simplicity of writing and understanding business rules
for the query traversal.

3. Scalability by performance allowing us to onboard
more corporate customers (project business case)

Autonomous Service
with it’s own life-cycle and data repository.
Authorization Complexity
• Not a collection of isolated customer trees *
• Not all users of a customer have equal access
• Not a fixed schema, form or size for all
customers
• Real-time updated with customer & product
data

The data form a highly connected living graph
* Covered later in Technical Details
How we Started with Neo4j
1. Searched the internet for articles about graph
database and different solutions.
2. Downloaded and quickly prototyped the solution we
liked that matched our requirements (Neo4j).
3. Workshop with Neo4j and our project developers to
quickly gain competence and ensure design QA.

4. Solution QA with Neo4j before production and help
with performance issues / tuning.
Lessons Learned
• Choose a solution/technology that fits your problem
• New way of thinking – build competence in org.
• Profile your java code to make it really fast
• Don’t put everything into the graph (functional creep)
• Need to know how traversal works (e.g. shortest
path)
• Benchmark the graph to evaluate your traversal
speed
Alternative In-Memory RDBMS
Option 1: Use existing database
- Performance issues due to shared data / suboptimal
structure
- Complexity since SQL not designed for traversal

Option 2: Separate database
+ Might reach same performance as graph db
+ Familiar technology
- Complexity since SQL not designed for traversal

Decided to go with our instinct

Graph Database
Different Graph Structures
get all accessible subscriptions
2 000

1 000

1 700 ms

Company X: 147 000

750 ms

Company Y: 52 000

1300 ms

Company Z: 95 000

Data from test – repeated prod sampling gave ~2.4 sec for 215,000 subscriptions
Different Graph Structures
check access to single subscription
2 000

1 000

1 ms

Company X: 147 000

1 ms

Company Y: 52 000

1 ms

Company Z: 95 000
Production Performance
retrieve all accessible resources
RDBMS Disk

RDBMS Mem Cached

Graph In-Heap

Company X

12 min

18 sec

< 2 sec

Company Y

22 min

58 sec

< 2 sec

Company Z

3 min

15 sec

< 2 sec

Check single resource access

1 ms
No operational problems in production
Technical Details
Production Details
Graph Size
heap)

27 million nodes (pre-warmed in
~1x properties, ~2x relationships

Traffic Volume

~1000 req/min during biz hours
~ 40K daily real-time updates

Performance
ms

Avg: 1 ms, 99% < 4 ms, 99.9% < 9

JVM
warmed)

Sun 6, 20 GB Heap (~15 GB pre-
Production Has Access Query

Time (ms)

Time (ms)
Production All Queries

Time (ms)
Garbage collection
Implementing the Algorithm
Lets look at the Neo4j Traversal Framework
Iterable<Node> getAccessibleResources(…) {

Evaluator myEvaluator = …
Expander myExpander = …
return Traversal.description()
.evaluator(myEvaluator)

.expander(myExpander)
.traverse(startNode).nodes();
}
Implementing the Algorithm
Evaluator is a simple filter, e.g. for Node
type
class MyEvaluator implements Evaluator {
public Evaluation evaluate(Path path) {
if <I am interested in this node>
return Evaluation.INCLUDE_AND_CONTINUE;
else
return Evaluation.EXCLUDE_AND_CONTINUE;
}
}
Implementing the Algorithm
The custom Expander contains business
rules!
class ResAuthExpander implements PathExpander<PathExpander> {
…
public … expand(Path path, BranchState<…> state) {
if (path.lastRelationship rel == ACCESS)
accToSub = rel.getProperty(ACCESS_TO_SUBSIDIARIES);
accToCont = rel.getProperty(ACCESS_TO_CONTENT);
state.set( getExpander(accToSub, accToCont) );
}
return state.get().expand(…)
}

Single expander class to control business
Implementing the Algorithm
Generates the valid relationships to traverse.
public getExpander(boolean accToSub, boolean accToCont) {
PathExpander exp = StandardExpander.DEFAULT.add(ACCESS,…);
if (accToSub)
exp.add(PARENT_COMPANY,…)
if (accToCont)
exp.add(PART_OF_COMPANY,…).add(SUBSCRIPTION_OWNER,…);
return exp;
}
}
U-Turn Strategy
4.
Access

User

Does the user
have access to
subscription X?

3.

5.

Up to find path quickly
Down to check access

6.
2.
7.
1.

8.
X

Subscription

Reversing the traversal increases performance from n/2 to
2d where n and d are tree size and depth (we went from 1s to
The Zigzag Problem
What if we also have reversed access to the subscription
payer?

User

Op

IT

E
d

Jo

Subscriptions

Solvable by adding state to the traversal (or check path)
The Many-to-Many Problem
The nodes Op & IT may be connected through many
subscriptions

Does the user
have access to
department Op?

Op

IT

Access

User

Subscription

Traversal becomes time consuming (e.g. M2M market)
However, we only needed to implement the rule for direct access to
sub.
Deployment View
• Two equal instances of Neo4j embedded in Tomcat
• Access through Java API due to need for custom logic
• Using Neo4j 1.8 without HA (did not like ZooKeeper)

Resource
Authorization

Neo4j

tx log
RDBMS

Message Queue

Resource
Authorization

Neo4j
Dual Model Cost
There are some drawbacks with dual models
also
• Not possible to simply join the ACL with resource
tables in the relational database - queries needed
redesign
• The complexity added by code and infrastructure
necessary to manage an additional model.
• Not ordinary competence (in Norway at least)
Unexplored Areas
Combining Access Control List & Graph
• Best of both worlds (simple logic, fast lookup)
Algorithm
–
–
–
–

Find all affected users when the graph is updated
Invalidate users access control list
Calculate all accessible resources for each user
Store result in users access control list

Could then skip the U-turn and many-to-many problem.
Was is worth it?

Yes!
The user experience is important in
Telenor
Questions?
Web References
• Telenor Norway
• The Project - How NOSQL Paid off for
Telenor
• JavaWorld - Graphs for Security

Más contenido relacionado

Was ist angesagt?

Igniting Audience Measurement at Time Warner Cable
Igniting Audience Measurement at Time Warner CableIgniting Audience Measurement at Time Warner Cable
Igniting Audience Measurement at Time Warner CableTim Case
 
Oracle Service Bus and Oracle SOA Suite in the Mobile World
Oracle Service Bus and Oracle SOA Suite in the Mobile WorldOracle Service Bus and Oracle SOA Suite in the Mobile World
Oracle Service Bus and Oracle SOA Suite in the Mobile WorldGuido Schmutz
 
Service Mesh Talk for CTO Forum
Service Mesh Talk for CTO ForumService Mesh Talk for CTO Forum
Service Mesh Talk for CTO ForumRick Hightower
 
Stream Analytics
Stream Analytics Stream Analytics
Stream Analytics Franco Ucci
 
Quantifying Genuine User Experience in Virtual Desktop Ecosystems
Quantifying Genuine User Experience in Virtual Desktop EcosystemsQuantifying Genuine User Experience in Virtual Desktop Ecosystems
Quantifying Genuine User Experience in Virtual Desktop EcosystemsData Con LA
 
Event-Driven Workflow: Monitoring and Orchestrating Your Microservices Landsc...
Event-Driven Workflow: Monitoring and Orchestrating Your Microservices Landsc...Event-Driven Workflow: Monitoring and Orchestrating Your Microservices Landsc...
Event-Driven Workflow: Monitoring and Orchestrating Your Microservices Landsc...confluent
 
Cisco's MultiCloud Strategy
Cisco's MultiCloud StrategyCisco's MultiCloud Strategy
Cisco's MultiCloud StrategyMaulik Shyani
 
APAC Confluent Consumer Data Right the Lowdown and the Lessons
APAC Confluent Consumer Data Right the Lowdown and the LessonsAPAC Confluent Consumer Data Right the Lowdown and the Lessons
APAC Confluent Consumer Data Right the Lowdown and the Lessonsconfluent
 
API Days Singapore
API Days SingaporeAPI Days Singapore
API Days Singaporeconfluent
 
Internet of Things - Are traditional architectures good enough?
Internet of Things - Are traditional architectures good enough?Internet of Things - Are traditional architectures good enough?
Internet of Things - Are traditional architectures good enough?Guido Schmutz
 
AppliFire Blue Print Design Guidelines
AppliFire Blue Print Design GuidelinesAppliFire Blue Print Design Guidelines
AppliFire Blue Print Design GuidelinesAppliFire Platform
 
50 Shades of Data - how, when and why Big, Fast, Relational, NoSQL, Elastic, ...
50 Shades of Data - how, when and why Big, Fast, Relational, NoSQL, Elastic, ...50 Shades of Data - how, when and why Big, Fast, Relational, NoSQL, Elastic, ...
50 Shades of Data - how, when and why Big, Fast, Relational, NoSQL, Elastic, ...Lucas Jellema
 
Modern data integration expert sessions
Modern data integration expert sessionsModern data integration expert sessions
Modern data integration expert sessionsJessicaMurrell3
 
Kafka and event driven architecture -apacoug20
Kafka and event driven architecture -apacoug20Kafka and event driven architecture -apacoug20
Kafka and event driven architecture -apacoug20Vinay Kumar
 
Cloud Architecture - Multi Cloud, Edge, On-Premise
Cloud Architecture - Multi Cloud, Edge, On-PremiseCloud Architecture - Multi Cloud, Edge, On-Premise
Cloud Architecture - Multi Cloud, Edge, On-PremiseAraf Karsh Hamid
 
Apache Kafka® Use Cases for Financial Services
Apache Kafka® Use Cases for Financial ServicesApache Kafka® Use Cases for Financial Services
Apache Kafka® Use Cases for Financial Servicesconfluent
 
Microsoft SQL Server - SQL Server Migrations Presentation
Microsoft SQL Server - SQL Server Migrations PresentationMicrosoft SQL Server - SQL Server Migrations Presentation
Microsoft SQL Server - SQL Server Migrations PresentationMicrosoft Private Cloud
 

Was ist angesagt? (19)

Igniting Audience Measurement at Time Warner Cable
Igniting Audience Measurement at Time Warner CableIgniting Audience Measurement at Time Warner Cable
Igniting Audience Measurement at Time Warner Cable
 
Oracle Service Bus and Oracle SOA Suite in the Mobile World
Oracle Service Bus and Oracle SOA Suite in the Mobile WorldOracle Service Bus and Oracle SOA Suite in the Mobile World
Oracle Service Bus and Oracle SOA Suite in the Mobile World
 
Service Mesh Talk for CTO Forum
Service Mesh Talk for CTO ForumService Mesh Talk for CTO Forum
Service Mesh Talk for CTO Forum
 
Stream Analytics
Stream Analytics Stream Analytics
Stream Analytics
 
Quantifying Genuine User Experience in Virtual Desktop Ecosystems
Quantifying Genuine User Experience in Virtual Desktop EcosystemsQuantifying Genuine User Experience in Virtual Desktop Ecosystems
Quantifying Genuine User Experience in Virtual Desktop Ecosystems
 
Event-Driven Workflow: Monitoring and Orchestrating Your Microservices Landsc...
Event-Driven Workflow: Monitoring and Orchestrating Your Microservices Landsc...Event-Driven Workflow: Monitoring and Orchestrating Your Microservices Landsc...
Event-Driven Workflow: Monitoring and Orchestrating Your Microservices Landsc...
 
Cisco's MultiCloud Strategy
Cisco's MultiCloud StrategyCisco's MultiCloud Strategy
Cisco's MultiCloud Strategy
 
APAC Confluent Consumer Data Right the Lowdown and the Lessons
APAC Confluent Consumer Data Right the Lowdown and the LessonsAPAC Confluent Consumer Data Right the Lowdown and the Lessons
APAC Confluent Consumer Data Right the Lowdown and the Lessons
 
API Days Singapore
API Days SingaporeAPI Days Singapore
API Days Singapore
 
Internet of Things - Are traditional architectures good enough?
Internet of Things - Are traditional architectures good enough?Internet of Things - Are traditional architectures good enough?
Internet of Things - Are traditional architectures good enough?
 
AppliFire Blue Print Design Guidelines
AppliFire Blue Print Design GuidelinesAppliFire Blue Print Design Guidelines
AppliFire Blue Print Design Guidelines
 
50 Shades of Data - how, when and why Big, Fast, Relational, NoSQL, Elastic, ...
50 Shades of Data - how, when and why Big, Fast, Relational, NoSQL, Elastic, ...50 Shades of Data - how, when and why Big, Fast, Relational, NoSQL, Elastic, ...
50 Shades of Data - how, when and why Big, Fast, Relational, NoSQL, Elastic, ...
 
Modern data integration expert sessions
Modern data integration expert sessionsModern data integration expert sessions
Modern data integration expert sessions
 
Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3Webinar Data Mesh - Part 3
Webinar Data Mesh - Part 3
 
Kafka and event driven architecture -apacoug20
Kafka and event driven architecture -apacoug20Kafka and event driven architecture -apacoug20
Kafka and event driven architecture -apacoug20
 
Cloud Architecture - Multi Cloud, Edge, On-Premise
Cloud Architecture - Multi Cloud, Edge, On-PremiseCloud Architecture - Multi Cloud, Edge, On-Premise
Cloud Architecture - Multi Cloud, Edge, On-Premise
 
Apache Kafka® Use Cases for Financial Services
Apache Kafka® Use Cases for Financial ServicesApache Kafka® Use Cases for Financial Services
Apache Kafka® Use Cases for Financial Services
 
IBM Cloud Direct Link 2.0
IBM Cloud Direct Link 2.0IBM Cloud Direct Link 2.0
IBM Cloud Direct Link 2.0
 
Microsoft SQL Server - SQL Server Migrations Presentation
Microsoft SQL Server - SQL Server Migrations PresentationMicrosoft SQL Server - SQL Server Migrations Presentation
Microsoft SQL Server - SQL Server Migrations Presentation
 

Andere mochten auch

Graph cafe-lightning
Graph cafe-lightningGraph cafe-lightning
Graph cafe-lightningVolker Pacher
 
Building a Distributed Reservation System with Cassandra (Andrew Baker & Jeff...
Building a Distributed Reservation System with Cassandra (Andrew Baker & Jeff...Building a Distributed Reservation System with Cassandra (Andrew Baker & Jeff...
Building a Distributed Reservation System with Cassandra (Andrew Baker & Jeff...DataStax
 
Wanderu - Lessons from Building a Travel Site with Neo4j
Wanderu - Lessons from Building a Travel Site with Neo4jWanderu - Lessons from Building a Travel Site with Neo4j
Wanderu - Lessons from Building a Travel Site with Neo4jNeo4j
 
Geschäftliches Potential für System-Integratoren und Berater - Graphdatenban...
Geschäftliches Potential für System-Integratoren und Berater -  Graphdatenban...Geschäftliches Potential für System-Integratoren und Berater -  Graphdatenban...
Geschäftliches Potential für System-Integratoren und Berater - Graphdatenban...Neo4j
 
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas WeberGraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas WeberNeo4j
 
Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016
Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016
Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016Sebastian Verheughe
 
GraphConnect Europe 2016 - How the ICIJ Used Neo4j to Unravel the Panama Pape...
GraphConnect Europe 2016 - How the ICIJ Used Neo4j to Unravel the Panama Pape...GraphConnect Europe 2016 - How the ICIJ Used Neo4j to Unravel the Panama Pape...
GraphConnect Europe 2016 - How the ICIJ Used Neo4j to Unravel the Panama Pape...Neo4j
 
Managing Genetic Ancestry at Scale with Neo4j and Kafka - StampedeCon 2015
Managing Genetic Ancestry at Scale with Neo4j and Kafka - StampedeCon 2015Managing Genetic Ancestry at Scale with Neo4j and Kafka - StampedeCon 2015
Managing Genetic Ancestry at Scale with Neo4j and Kafka - StampedeCon 2015StampedeCon
 
GraphConnect Europe 2016 - Semantic PIM: Using a Graph Data Model at Toy Manu...
GraphConnect Europe 2016 - Semantic PIM: Using a Graph Data Model at Toy Manu...GraphConnect Europe 2016 - Semantic PIM: Using a Graph Data Model at Toy Manu...
GraphConnect Europe 2016 - Semantic PIM: Using a Graph Data Model at Toy Manu...Neo4j
 
GraphConnect Europe 2016 - Building a Repository of Biomedical Ontologies wit...
GraphConnect Europe 2016 - Building a Repository of Biomedical Ontologies wit...GraphConnect Europe 2016 - Building a Repository of Biomedical Ontologies wit...
GraphConnect Europe 2016 - Building a Repository of Biomedical Ontologies wit...Neo4j
 
Neo4j GraphTalks - Semantische Netze
Neo4j GraphTalks - Semantische NetzeNeo4j GraphTalks - Semantische Netze
Neo4j GraphTalks - Semantische NetzeNeo4j
 
Identity and Access Management
Identity and Access ManagementIdentity and Access Management
Identity and Access ManagementNeo4j
 
Cassandra at eBay - Cassandra Summit 2012
Cassandra at eBay - Cassandra Summit 2012Cassandra at eBay - Cassandra Summit 2012
Cassandra at eBay - Cassandra Summit 2012Jay Patel
 
Migrating Netflix from Datacenter Oracle to Global Cassandra
Migrating Netflix from Datacenter Oracle to Global CassandraMigrating Netflix from Datacenter Oracle to Global Cassandra
Migrating Netflix from Datacenter Oracle to Global CassandraAdrian Cockcroft
 
HTTP/2 Changes Everything
HTTP/2 Changes EverythingHTTP/2 Changes Everything
HTTP/2 Changes EverythingLori MacVittie
 
Introduction to Graph Databases
Introduction to Graph DatabasesIntroduction to Graph Databases
Introduction to Graph DatabasesMax De Marzi
 

Andere mochten auch (18)

How NOSQL Paid off for Telenor
How NOSQL Paid off for TelenorHow NOSQL Paid off for Telenor
How NOSQL Paid off for Telenor
 
Graph cafe-lightning
Graph cafe-lightningGraph cafe-lightning
Graph cafe-lightning
 
Building a Distributed Reservation System with Cassandra (Andrew Baker & Jeff...
Building a Distributed Reservation System with Cassandra (Andrew Baker & Jeff...Building a Distributed Reservation System with Cassandra (Andrew Baker & Jeff...
Building a Distributed Reservation System with Cassandra (Andrew Baker & Jeff...
 
Wanderu - Lessons from Building a Travel Site with Neo4j
Wanderu - Lessons from Building a Travel Site with Neo4jWanderu - Lessons from Building a Travel Site with Neo4j
Wanderu - Lessons from Building a Travel Site with Neo4j
 
Geschäftliches Potential für System-Integratoren und Berater - Graphdatenban...
Geschäftliches Potential für System-Integratoren und Berater -  Graphdatenban...Geschäftliches Potential für System-Integratoren und Berater -  Graphdatenban...
Geschäftliches Potential für System-Integratoren und Berater - Graphdatenban...
 
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas WeberGraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
GraphTalks - Semantisches Produktdatenmanagement, Dr. Andreas Weber
 
Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016
Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016
Strategic Design by Architecture and Organisation @ FINN.no - JavaZone 2016
 
GraphConnect Europe 2016 - How the ICIJ Used Neo4j to Unravel the Panama Pape...
GraphConnect Europe 2016 - How the ICIJ Used Neo4j to Unravel the Panama Pape...GraphConnect Europe 2016 - How the ICIJ Used Neo4j to Unravel the Panama Pape...
GraphConnect Europe 2016 - How the ICIJ Used Neo4j to Unravel the Panama Pape...
 
Managing Genetic Ancestry at Scale with Neo4j and Kafka - StampedeCon 2015
Managing Genetic Ancestry at Scale with Neo4j and Kafka - StampedeCon 2015Managing Genetic Ancestry at Scale with Neo4j and Kafka - StampedeCon 2015
Managing Genetic Ancestry at Scale with Neo4j and Kafka - StampedeCon 2015
 
GraphConnect Europe 2016 - Semantic PIM: Using a Graph Data Model at Toy Manu...
GraphConnect Europe 2016 - Semantic PIM: Using a Graph Data Model at Toy Manu...GraphConnect Europe 2016 - Semantic PIM: Using a Graph Data Model at Toy Manu...
GraphConnect Europe 2016 - Semantic PIM: Using a Graph Data Model at Toy Manu...
 
GraphConnect Europe 2016 - Building a Repository of Biomedical Ontologies wit...
GraphConnect Europe 2016 - Building a Repository of Biomedical Ontologies wit...GraphConnect Europe 2016 - Building a Repository of Biomedical Ontologies wit...
GraphConnect Europe 2016 - Building a Repository of Biomedical Ontologies wit...
 
Neo4j GraphTalks - Semantische Netze
Neo4j GraphTalks - Semantische NetzeNeo4j GraphTalks - Semantische Netze
Neo4j GraphTalks - Semantische Netze
 
Telenor
TelenorTelenor
Telenor
 
Identity and Access Management
Identity and Access ManagementIdentity and Access Management
Identity and Access Management
 
Cassandra at eBay - Cassandra Summit 2012
Cassandra at eBay - Cassandra Summit 2012Cassandra at eBay - Cassandra Summit 2012
Cassandra at eBay - Cassandra Summit 2012
 
Migrating Netflix from Datacenter Oracle to Global Cassandra
Migrating Netflix from Datacenter Oracle to Global CassandraMigrating Netflix from Datacenter Oracle to Global Cassandra
Migrating Netflix from Datacenter Oracle to Global Cassandra
 
HTTP/2 Changes Everything
HTTP/2 Changes EverythingHTTP/2 Changes Everything
HTTP/2 Changes Everything
 
Introduction to Graph Databases
Introduction to Graph DatabasesIntroduction to Graph Databases
Introduction to Graph Databases
 

Ähnlich wie Using Graph Databases in Real-Time to Solve Resource Authorization at Telenor - GraphConnect London 2013

Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...
Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...
Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...Neo4j
 
Peek into Neo4j Product Strategy and Roadmap
Peek into Neo4j Product Strategy and RoadmapPeek into Neo4j Product Strategy and Roadmap
Peek into Neo4j Product Strategy and RoadmapNeo4j
 
PLNOG 3: Tomasz Mikołajczyk - Data scalability. Why you should care?
PLNOG 3: Tomasz Mikołajczyk -  Data scalability. Why you should care?PLNOG 3: Tomasz Mikołajczyk -  Data scalability. Why you should care?
PLNOG 3: Tomasz Mikołajczyk - Data scalability. Why you should care?PROIDEA
 
Maximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
Maximizing Data Lake ROI with Data Virtualization: A Technical DemonstrationMaximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
Maximizing Data Lake ROI with Data Virtualization: A Technical DemonstrationDenodo
 
Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )
Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )
Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )Rajendra Kumar Sahu
 
Importing Large Sets of Content from Trusted Partners into your Repository
Importing Large Sets of Content from Trusted Partners into your RepositoryImporting Large Sets of Content from Trusted Partners into your Repository
Importing Large Sets of Content from Trusted Partners into your RepositoryBlueFish
 
Grab: Building a Healthy Elasticsearch Ecosystem
Grab: Building a Healthy Elasticsearch EcosystemGrab: Building a Healthy Elasticsearch Ecosystem
Grab: Building a Healthy Elasticsearch EcosystemElasticsearch
 
Building data pipelines at Shopee with DEC
Building data pipelines at Shopee with DECBuilding data pipelines at Shopee with DEC
Building data pipelines at Shopee with DECRim Zaidullin
 
Presenting Data – An Alternative to the View Control
Presenting Data – An Alternative to the View ControlPresenting Data – An Alternative to the View Control
Presenting Data – An Alternative to the View ControlTeamstudio
 
Yuriy Chapran - Building microservices.
Yuriy Chapran - Building microservices.Yuriy Chapran - Building microservices.
Yuriy Chapran - Building microservices.Yuriy Chapran
 
Apricot2017 Request tracing in distributed environment
Apricot2017 Request tracing in distributed environmentApricot2017 Request tracing in distributed environment
Apricot2017 Request tracing in distributed environmentHieu LE ☁
 
Contino Webinar - Migrating your Trading Workloads to the Cloud
Contino Webinar -  Migrating your Trading Workloads to the CloudContino Webinar -  Migrating your Trading Workloads to the Cloud
Contino Webinar - Migrating your Trading Workloads to the CloudBen Saunders
 
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 20091 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009Moshe Kaplan
 

Ähnlich wie Using Graph Databases in Real-Time to Solve Resource Authorization at Telenor - GraphConnect London 2013 (20)

Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...
Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...
Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...
 
Peek into Neo4j Product Strategy and Roadmap
Peek into Neo4j Product Strategy and RoadmapPeek into Neo4j Product Strategy and Roadmap
Peek into Neo4j Product Strategy and Roadmap
 
Madhu_Resume
Madhu_ResumeMadhu_Resume
Madhu_Resume
 
PLNOG 3: Tomasz Mikołajczyk - Data scalability. Why you should care?
PLNOG 3: Tomasz Mikołajczyk -  Data scalability. Why you should care?PLNOG 3: Tomasz Mikołajczyk -  Data scalability. Why you should care?
PLNOG 3: Tomasz Mikołajczyk - Data scalability. Why you should care?
 
DevOps Case Studies
DevOps Case StudiesDevOps Case Studies
DevOps Case Studies
 
VamsiKrishna Maddiboina
VamsiKrishna MaddiboinaVamsiKrishna Maddiboina
VamsiKrishna Maddiboina
 
Subbu_WM
Subbu_WMSubbu_WM
Subbu_WM
 
CV_DebarpanMukherjee
CV_DebarpanMukherjeeCV_DebarpanMukherjee
CV_DebarpanMukherjee
 
mayank_unix_sql_Jboss
mayank_unix_sql_Jbossmayank_unix_sql_Jboss
mayank_unix_sql_Jboss
 
Maximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
Maximizing Data Lake ROI with Data Virtualization: A Technical DemonstrationMaximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
Maximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
 
Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )
Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )
Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )
 
Importing Large Sets of Content from Trusted Partners into your Repository
Importing Large Sets of Content from Trusted Partners into your RepositoryImporting Large Sets of Content from Trusted Partners into your Repository
Importing Large Sets of Content from Trusted Partners into your Repository
 
Grab: Building a Healthy Elasticsearch Ecosystem
Grab: Building a Healthy Elasticsearch EcosystemGrab: Building a Healthy Elasticsearch Ecosystem
Grab: Building a Healthy Elasticsearch Ecosystem
 
JKSQL
JKSQLJKSQL
JKSQL
 
Building data pipelines at Shopee with DEC
Building data pipelines at Shopee with DECBuilding data pipelines at Shopee with DEC
Building data pipelines at Shopee with DEC
 
Presenting Data – An Alternative to the View Control
Presenting Data – An Alternative to the View ControlPresenting Data – An Alternative to the View Control
Presenting Data – An Alternative to the View Control
 
Yuriy Chapran - Building microservices.
Yuriy Chapran - Building microservices.Yuriy Chapran - Building microservices.
Yuriy Chapran - Building microservices.
 
Apricot2017 Request tracing in distributed environment
Apricot2017 Request tracing in distributed environmentApricot2017 Request tracing in distributed environment
Apricot2017 Request tracing in distributed environment
 
Contino Webinar - Migrating your Trading Workloads to the Cloud
Contino Webinar -  Migrating your Trading Workloads to the CloudContino Webinar -  Migrating your Trading Workloads to the Cloud
Contino Webinar - Migrating your Trading Workloads to the Cloud
 
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 20091 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009
 

Último

Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FESTBillieHyde
 
The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)IES VE
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingFrancesco Corti
 
The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)codyslingerland1
 
Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3DianaGray10
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.IPLOOK Networks
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIVijayananda Mohire
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdfThe Good Food Institute
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kitJamie (Taka) Wang
 
Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxSatishbabu Gunukula
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingMAGNIntelligence
 
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptxGraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptxNeo4j
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...DianaGray10
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightSafe Software
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTxtailishbaloch
 
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechWebinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechProduct School
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveIES VE
 
Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Muhammad Tiham Siddiqui
 

Último (20)

Technical SEO for Improved Accessibility WTS FEST
Technical SEO for Improved Accessibility  WTS FESTTechnical SEO for Improved Accessibility  WTS FEST
Technical SEO for Improved Accessibility WTS FEST
 
The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)The Importance of Indoor Air Quality (English)
The Importance of Indoor Air Quality (English)
 
Where developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is goingWhere developers are challenged, what developers want and where DevEx is going
Where developers are challenged, what developers want and where DevEx is going
 
The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)The New Cloud World Order Is FinOps (Slideshow)
The New Cloud World Order Is FinOps (Slideshow)
 
Planetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile BrochurePlanetek Italia Srl - Corporate Profile Brochure
Planetek Italia Srl - Corporate Profile Brochure
 
UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3UiPath Studio Web workshop Series - Day 3
UiPath Studio Web workshop Series - Day 3
 
Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.Introduction - IPLOOK NETWORKS CO., LTD.
Introduction - IPLOOK NETWORKS CO., LTD.
 
My key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAIMy key hands-on projects in Quantum, and QAI
My key hands-on projects in Quantum, and QAI
 
2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf2024.03.12 Cost drivers of cultivated meat production.pdf
2024.03.12 Cost drivers of cultivated meat production.pdf
 
20140402 - Smart house demo kit
20140402 - Smart house demo kit20140402 - Smart house demo kit
20140402 - Smart house demo kit
 
Oracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptxOracle Database 23c Security New Features.pptx
Oracle Database 23c Security New Features.pptx
 
SheDev 2024
SheDev 2024SheDev 2024
SheDev 2024
 
IT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced ComputingIT Service Management (ITSM) Best Practices for Advanced Computing
IT Service Management (ITSM) Best Practices for Advanced Computing
 
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptxGraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
 
Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...Explore the UiPath Community and ways you can benefit on your journey to auto...
Explore the UiPath Community and ways you can benefit on your journey to auto...
 
The Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and InsightThe Zero-ETL Approach: Enhancing Data Agility and Insight
The Zero-ETL Approach: Enhancing Data Agility and Insight
 
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENTSIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
SIM INFORMATION SYSTEM: REVOLUTIONIZING DATA MANAGEMENT
 
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - TechWebinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
Webinar: The Art of Prioritizing Your Product Roadmap by AWS Sr PM - Tech
 
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES LiveKeep Your Finger on the Pulse of Your Building's Performance with IES Live
Keep Your Finger on the Pulse of Your Building's Performance with IES Live
 
Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)Trailblazer Community - Flows Workshop (Session 2)
Trailblazer Community - Flows Workshop (Session 2)
 

Using Graph Databases in Real-Time to Solve Resource Authorization at Telenor - GraphConnect London 2013

  • 1. Using Graph Databases in Real-Time to Solve Resource Authorization at Telenor Graph Connect London – 19 Nov 2013 by Sebastian Verheughe
  • 2. Telenor Norway Subsidiary of the Telenor Group ~1 billion GBP in mobile revenues 2012 Sebastian Verheughe Lead Developer for Neo4j solution Coding Architect
  • 3. Disclaimer The presentation is not identical to the implementation due to security reasons but shows how we have modeled and solved the problem in general. However, all presented data (numbers & charts) are real, unfiltered and extracted from the production logs
  • 5. Telenor Norway Middleware Services Channel Channel used by 42 channels calls 35 sub-systems 10,000 code classes 500 requests/second 20,000 orders/day Backend Backend Channel Channel MOBILE MW Channel Channel Providing business logic and data for all channels in the mobile value chain BUSINESS LOGIC & DATA Backend Handles users with access to X00,000 resources Backend Backend Backend
  • 6. Our Problem 20 minutes to calculate all accessible resources 1500 lines of SQL to implement the authorization logic “solved” by caching data going stale and the solution did not scale…
  • 7. Why a Graph Database? Access Parent Company User: R. Branson Which resources does the user have access to? Part of Company Sales Finance Production HR Sub The questions we wanted answered required traversal of tree structures. Tablet Subscription Owner Sub Tablet Uses Subscription Phone
  • 8. Tailored Read Model The Model makes read queries as simple and efficient as possible. First find your questions then model your graph graph model = relational model
  • 9. High Level Architecture Clients Classic MW Services other sources tx log RDBMS check access Resource Authorization Message Queue Neo4j
  • 10. Conditional Rules ACCESS is given with the following include parameters: access to subsidiaries and access to content Only find children of PARENT COMPANY given access to subsidiaries is allowed User Only look at PART OF COMPANY given access to content is allowed Only look at SUBSCRIPTION OWNER given access to content is allowed
  • 11. Different Access Needs Access Subsidiaries & Content Super Admin Access Content Umbrella Admin Access S&C Admin
  • 12. Graph Algorithm Prerequisite: The user node 1. Follow all ACCESS relationships and read the access parameters on the relationship 2. Follow all PARENT COMPANY relationships given access to subsidiaries is allowed 3. Follow all PART OF COMPANY relationships given access to content is allowed 4. Follow all SUBSCRIPTION OWNER relationships given access to content is allowed
  • 13. Solution Value 1. Performance optimized from minutes to seconds. 2. Simplicity of writing and understanding business rules for the query traversal. 3. Scalability by performance allowing us to onboard more corporate customers (project business case) Autonomous Service with it’s own life-cycle and data repository.
  • 14. Authorization Complexity • Not a collection of isolated customer trees * • Not all users of a customer have equal access • Not a fixed schema, form or size for all customers • Real-time updated with customer & product data The data form a highly connected living graph * Covered later in Technical Details
  • 15. How we Started with Neo4j 1. Searched the internet for articles about graph database and different solutions. 2. Downloaded and quickly prototyped the solution we liked that matched our requirements (Neo4j). 3. Workshop with Neo4j and our project developers to quickly gain competence and ensure design QA. 4. Solution QA with Neo4j before production and help with performance issues / tuning.
  • 16. Lessons Learned • Choose a solution/technology that fits your problem • New way of thinking – build competence in org. • Profile your java code to make it really fast • Don’t put everything into the graph (functional creep) • Need to know how traversal works (e.g. shortest path) • Benchmark the graph to evaluate your traversal speed
  • 17. Alternative In-Memory RDBMS Option 1: Use existing database - Performance issues due to shared data / suboptimal structure - Complexity since SQL not designed for traversal Option 2: Separate database + Might reach same performance as graph db + Familiar technology - Complexity since SQL not designed for traversal Decided to go with our instinct Graph Database
  • 18. Different Graph Structures get all accessible subscriptions 2 000 1 000 1 700 ms Company X: 147 000 750 ms Company Y: 52 000 1300 ms Company Z: 95 000 Data from test – repeated prod sampling gave ~2.4 sec for 215,000 subscriptions
  • 19. Different Graph Structures check access to single subscription 2 000 1 000 1 ms Company X: 147 000 1 ms Company Y: 52 000 1 ms Company Z: 95 000
  • 20. Production Performance retrieve all accessible resources RDBMS Disk RDBMS Mem Cached Graph In-Heap Company X 12 min 18 sec < 2 sec Company Y 22 min 58 sec < 2 sec Company Z 3 min 15 sec < 2 sec Check single resource access 1 ms No operational problems in production
  • 22. Production Details Graph Size heap) 27 million nodes (pre-warmed in ~1x properties, ~2x relationships Traffic Volume ~1000 req/min during biz hours ~ 40K daily real-time updates Performance ms Avg: 1 ms, 99% < 4 ms, 99.9% < 9 JVM warmed) Sun 6, 20 GB Heap (~15 GB pre-
  • 23. Production Has Access Query Time (ms) Time (ms)
  • 24. Production All Queries Time (ms) Garbage collection
  • 25. Implementing the Algorithm Lets look at the Neo4j Traversal Framework Iterable<Node> getAccessibleResources(…) { Evaluator myEvaluator = … Expander myExpander = … return Traversal.description() .evaluator(myEvaluator) .expander(myExpander) .traverse(startNode).nodes(); }
  • 26. Implementing the Algorithm Evaluator is a simple filter, e.g. for Node type class MyEvaluator implements Evaluator { public Evaluation evaluate(Path path) { if <I am interested in this node> return Evaluation.INCLUDE_AND_CONTINUE; else return Evaluation.EXCLUDE_AND_CONTINUE; } }
  • 27. Implementing the Algorithm The custom Expander contains business rules! class ResAuthExpander implements PathExpander<PathExpander> { … public … expand(Path path, BranchState<…> state) { if (path.lastRelationship rel == ACCESS) accToSub = rel.getProperty(ACCESS_TO_SUBSIDIARIES); accToCont = rel.getProperty(ACCESS_TO_CONTENT); state.set( getExpander(accToSub, accToCont) ); } return state.get().expand(…) } Single expander class to control business
  • 28. Implementing the Algorithm Generates the valid relationships to traverse. public getExpander(boolean accToSub, boolean accToCont) { PathExpander exp = StandardExpander.DEFAULT.add(ACCESS,…); if (accToSub) exp.add(PARENT_COMPANY,…) if (accToCont) exp.add(PART_OF_COMPANY,…).add(SUBSCRIPTION_OWNER,…); return exp; } }
  • 29. U-Turn Strategy 4. Access User Does the user have access to subscription X? 3. 5. Up to find path quickly Down to check access 6. 2. 7. 1. 8. X Subscription Reversing the traversal increases performance from n/2 to 2d where n and d are tree size and depth (we went from 1s to
  • 30. The Zigzag Problem What if we also have reversed access to the subscription payer? User Op IT E d Jo Subscriptions Solvable by adding state to the traversal (or check path)
  • 31. The Many-to-Many Problem The nodes Op & IT may be connected through many subscriptions Does the user have access to department Op? Op IT Access User Subscription Traversal becomes time consuming (e.g. M2M market) However, we only needed to implement the rule for direct access to sub.
  • 32. Deployment View • Two equal instances of Neo4j embedded in Tomcat • Access through Java API due to need for custom logic • Using Neo4j 1.8 without HA (did not like ZooKeeper) Resource Authorization Neo4j tx log RDBMS Message Queue Resource Authorization Neo4j
  • 33. Dual Model Cost There are some drawbacks with dual models also • Not possible to simply join the ACL with resource tables in the relational database - queries needed redesign • The complexity added by code and infrastructure necessary to manage an additional model. • Not ordinary competence (in Norway at least)
  • 34. Unexplored Areas Combining Access Control List & Graph • Best of both worlds (simple logic, fast lookup) Algorithm – – – – Find all affected users when the graph is updated Invalidate users access control list Calculate all accessible resources for each user Store result in users access control list Could then skip the U-turn and many-to-many problem.
  • 35. Was is worth it? Yes! The user experience is important in Telenor
  • 37. Web References • Telenor Norway • The Project - How NOSQL Paid off for Telenor • JavaWorld - Graphs for Security

Hinweis der Redaktion

  1. Why: access to secret numbers, access to modify/delete subscriptions, possibility to send/receive messages
  2. We use Neo4j for our business critical services, both customer/product services , but also operational services.A channel is client type, e.g. the web solution for corporate customer, or helpdesk solution, or app, and may consist of many clients
  3. The project business case was based on a future point in time where we could not any more onboard any large corporate customers
  4. Drawing on the white board the required logic made us understand that a graph database might be a good solution
  5. Take the hit on write, and make read easy! (for us, read performance is the problem – not write performance)Also, don’t blindly copy tables/foreign keys into nodes and relationships – drop what’s not needed and remember that relationships may have properties in graph
  6. RDBMS is still mastering the data as it is used in many different use-cases where that is beneficial.
  7. TIME LEFT: 30 minutes (10 used)
  8. The last part is important to us. It was really hard to extract the resource authorization out of the relational database, but not we can much more easy replace the current implementation with another one in the future if neccesary.
  9. Production logs does not contain user data, so just one big organization was sampled to get production data for a specific customer
  10. TIME LEFT: 20 minutes (20 used)Graphperformance based on test environment, see charts in the technical section for production numbers not specific for a unique customer
  11. We only have detailed logs for a short while back – so we cannot review all data since production.First production two years ago with limited traffic, full production since spring 2013
  12. We always continue, since we also have our custom expander. This way, we have a clean separation of concern in our code.We also have more advanced filters peaking around the node before it decides to include or exclude the node
  13. This is the most important part of the code, the one place where we now are able to write down the business logic in a simple and natural way.Note that we only have ONE class containing the business rules independently of which use-case we are running.
  14. The relationships and directions that are allowed to traverse given the different switch parameters.
  15. This is possible since we have a tree graph. Demonstrates the importance of understanding how a graph works, because than you may greatly improve performance by smart traversal strategies.
  16. TIME LEFT: 10 minutes (30 used)
  17. Extra knowledge, such as which subscription you are