SlideShare ist ein Scribd-Unternehmen logo
1 von 37
Downloaden Sie, um offline zu lesen
Using Graph Databases in Real-Time to
Solve Resource Authorization at Telenor

Graph Connect London – 19 Nov 2013
by Sebastian Verheughe
Telenor Norway
Subsidiary of the Telenor Group
~1 billion GBP in mobile revenues 2012

Sebastian Verheughe
Lead Developer for Neo4j solution
Coding Architect
Disclaimer
The presentation is not identical to the implementation
due to security reasons but shows how we have
modeled and solved the problem in general.
However, all presented data (numbers & charts) are
real, unfiltered and extracted from the production logs
A very

aspect of
our business
Telenor Norway Middleware Services
Channel

Channel

used by 42 channels
calls 35 sub-systems
10,000 code classes
500 requests/second
20,000 orders/day

Backend

Backend

Channel

Channel

MOBILE

MW

Channel

Channel

Providing business logic
and data for all channels
in the mobile value chain

BUSINESS

LOGIC
& DATA

Backend

Handles users with
access to X00,000
resources

Backend

Backend

Backend
Our Problem
20 minutes to calculate all accessible resources
1500 lines of SQL to implement the authorization logic
“solved” by caching data going stale
and the solution did not scale…
Why a Graph Database?
Access

Parent Company

User: R. Branson

Which resources
does the user
have access to?

Part of Company

Sales

Finance Production

HR

Sub

The questions we wanted answered
required traversal of tree structures.

Tablet

Subscription
Owner

Sub

Tablet

Uses
Subscription

Phone
Tailored Read Model
The Model makes read queries
as simple and efficient as possible.
First find your questions
then model your graph

graph model

=

relational model
High Level Architecture
Clients

Classic MW
Services

other
sources

tx log
RDBMS

check
access

Resource
Authorization

Message Queue

Neo4j
Conditional Rules
ACCESS is given with the following include parameters:
access to subsidiaries and access to content
Only find children of PARENT COMPANY
given access to subsidiaries is allowed
User

Only look at PART OF COMPANY
given access to content is allowed

Only look at SUBSCRIPTION OWNER
given access to content is allowed
Different Access Needs
Access Subsidiaries & Content

Super Admin

Access Content

Umbrella Admin
Access S&C

Admin
Graph Algorithm
Prerequisite: The user node
1. Follow all ACCESS relationships and
read the access parameters on the relationship
2. Follow all PARENT COMPANY relationships given
access to subsidiaries is allowed

3. Follow all PART OF COMPANY relationships given
access to content is allowed
4. Follow all SUBSCRIPTION OWNER relationships given
access to content is allowed
Solution Value
1. Performance optimized from minutes to seconds.
2. Simplicity of writing and understanding business rules
for the query traversal.

3. Scalability by performance allowing us to onboard
more corporate customers (project business case)

Autonomous Service
with it’s own life-cycle and data repository.
Authorization Complexity
• Not a collection of isolated customer trees *
• Not all users of a customer have equal access
• Not a fixed schema, form or size for all
customers
• Real-time updated with customer & product
data

The data form a highly connected living graph
* Covered later in Technical Details
How we Started with Neo4j
1. Searched the internet for articles about graph
database and different solutions.
2. Downloaded and quickly prototyped the solution we
liked that matched our requirements (Neo4j).
3. Workshop with Neo4j and our project developers to
quickly gain competence and ensure design QA.

4. Solution QA with Neo4j before production and help
with performance issues / tuning.
Lessons Learned
• Choose a solution/technology that fits your problem
• New way of thinking – build competence in org.
• Profile your java code to make it really fast
• Don’t put everything into the graph (functional creep)
• Need to know how traversal works (e.g. shortest
path)
• Benchmark the graph to evaluate your traversal
speed
Alternative In-Memory RDBMS
Option 1: Use existing database
- Performance issues due to shared data / suboptimal
structure
- Complexity since SQL not designed for traversal

Option 2: Separate database
+ Might reach same performance as graph db
+ Familiar technology
- Complexity since SQL not designed for traversal

Decided to go with our instinct

Graph Database
Different Graph Structures
get all accessible subscriptions
2 000

1 000

1 700 ms

Company X: 147 000

750 ms

Company Y: 52 000

1300 ms

Company Z: 95 000

Data from test – repeated prod sampling gave ~2.4 sec for 215,000 subscriptions
Different Graph Structures
check access to single subscription
2 000

1 000

1 ms

Company X: 147 000

1 ms

Company Y: 52 000

1 ms

Company Z: 95 000
Production Performance
retrieve all accessible resources
RDBMS Disk

RDBMS Mem Cached

Graph In-Heap

Company X

12 min

18 sec

< 2 sec

Company Y

22 min

58 sec

< 2 sec

Company Z

3 min

15 sec

< 2 sec

Check single resource access

1 ms
No operational problems in production
Technical Details
Production Details
Graph Size
heap)

27 million nodes (pre-warmed in
~1x properties, ~2x relationships

Traffic Volume

~1000 req/min during biz hours
~ 40K daily real-time updates

Performance
ms

Avg: 1 ms, 99% < 4 ms, 99.9% < 9

JVM
warmed)

Sun 6, 20 GB Heap (~15 GB pre-
Production Has Access Query

Time (ms)

Time (ms)
Production All Queries

Time (ms)
Garbage collection
Implementing the Algorithm
Lets look at the Neo4j Traversal Framework
Iterable<Node> getAccessibleResources(…) {

Evaluator myEvaluator = …
Expander myExpander = …
return Traversal.description()
.evaluator(myEvaluator)

.expander(myExpander)
.traverse(startNode).nodes();
}
Implementing the Algorithm
Evaluator is a simple filter, e.g. for Node
type
class MyEvaluator implements Evaluator {
public Evaluation evaluate(Path path) {
if <I am interested in this node>
return Evaluation.INCLUDE_AND_CONTINUE;
else
return Evaluation.EXCLUDE_AND_CONTINUE;
}
}
Implementing the Algorithm
The custom Expander contains business
rules!
class ResAuthExpander implements PathExpander<PathExpander> {
…
public … expand(Path path, BranchState<…> state) {
if (path.lastRelationship rel == ACCESS)
accToSub = rel.getProperty(ACCESS_TO_SUBSIDIARIES);
accToCont = rel.getProperty(ACCESS_TO_CONTENT);
state.set( getExpander(accToSub, accToCont) );
}
return state.get().expand(…)
}

Single expander class to control business
Implementing the Algorithm
Generates the valid relationships to traverse.
public getExpander(boolean accToSub, boolean accToCont) {
PathExpander exp = StandardExpander.DEFAULT.add(ACCESS,…);
if (accToSub)
exp.add(PARENT_COMPANY,…)
if (accToCont)
exp.add(PART_OF_COMPANY,…).add(SUBSCRIPTION_OWNER,…);
return exp;
}
}
U-Turn Strategy
4.
Access

User

Does the user
have access to
subscription X?

3.

5.

Up to find path quickly
Down to check access

6.
2.
7.
1.

8.
X

Subscription

Reversing the traversal increases performance from n/2 to
2d where n and d are tree size and depth (we went from 1s to
The Zigzag Problem
What if we also have reversed access to the subscription
payer?

User

Op

IT

E
d

Jo

Subscriptions

Solvable by adding state to the traversal (or check path)
The Many-to-Many Problem
The nodes Op & IT may be connected through many
subscriptions

Does the user
have access to
department Op?

Op

IT

Access

User

Subscription

Traversal becomes time consuming (e.g. M2M market)
However, we only needed to implement the rule for direct access to
sub.
Deployment View
• Two equal instances of Neo4j embedded in Tomcat
• Access through Java API due to need for custom logic
• Using Neo4j 1.8 without HA (did not like ZooKeeper)

Resource
Authorization

Neo4j

tx log
RDBMS

Message Queue

Resource
Authorization

Neo4j
Dual Model Cost
There are some drawbacks with dual models
also
• Not possible to simply join the ACL with resource
tables in the relational database - queries needed
redesign
• The complexity added by code and infrastructure
necessary to manage an additional model.
• Not ordinary competence (in Norway at least)
Unexplored Areas
Combining Access Control List & Graph
• Best of both worlds (simple logic, fast lookup)
Algorithm
–
–
–
–

Find all affected users when the graph is updated
Invalidate users access control list
Calculate all accessible resources for each user
Store result in users access control list

Could then skip the U-turn and many-to-many problem.
Was is worth it?

Yes!
The user experience is important in
Telenor
Questions?
Web References
• Telenor Norway
• The Project - How NOSQL Paid off for
Telenor
• JavaWorld - Graphs for Security

Weitere ähnliche Inhalte

Was ist angesagt?

[English]Medium Inc Company Profile
[English]Medium Inc Company Profile[English]Medium Inc Company Profile
[English]Medium Inc Company ProfileJaeKwon9
 
Ijmer 41025357
Ijmer 41025357Ijmer 41025357
Ijmer 41025357IJMER
 
Score based deadline constrained workflow scheduling algorithm for cloud systems
Score based deadline constrained workflow scheduling algorithm for cloud systemsScore based deadline constrained workflow scheduling algorithm for cloud systems
Score based deadline constrained workflow scheduling algorithm for cloud systemsijccsa
 
The eBay Architecture: Striking a Balance between Site Stability, Feature Ve...
The eBay Architecture:  Striking a Balance between Site Stability, Feature Ve...The eBay Architecture:  Striking a Balance between Site Stability, Feature Ve...
The eBay Architecture: Striking a Balance between Site Stability, Feature Ve...Randy Shoup
 
VMworld 2013: Data In, Data Out and Data Protected
VMworld 2013: Data In, Data Out and Data Protected VMworld 2013: Data In, Data Out and Data Protected
VMworld 2013: Data In, Data Out and Data Protected VMworld
 
DEVNET-1140 InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
DEVNET-1140	InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...DEVNET-1140	InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
DEVNET-1140 InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...Cisco DevNet
 
Microservices Architecture & Testing Strategies
Microservices Architecture & Testing StrategiesMicroservices Architecture & Testing Strategies
Microservices Architecture & Testing StrategiesAraf Karsh Hamid
 

Was ist angesagt? (8)

[English]Medium Inc Company Profile
[English]Medium Inc Company Profile[English]Medium Inc Company Profile
[English]Medium Inc Company Profile
 
Ijmer 41025357
Ijmer 41025357Ijmer 41025357
Ijmer 41025357
 
Score based deadline constrained workflow scheduling algorithm for cloud systems
Score based deadline constrained workflow scheduling algorithm for cloud systemsScore based deadline constrained workflow scheduling algorithm for cloud systems
Score based deadline constrained workflow scheduling algorithm for cloud systems
 
Ammar Murtaza-IM
Ammar Murtaza-IMAmmar Murtaza-IM
Ammar Murtaza-IM
 
The eBay Architecture: Striking a Balance between Site Stability, Feature Ve...
The eBay Architecture:  Striking a Balance between Site Stability, Feature Ve...The eBay Architecture:  Striking a Balance between Site Stability, Feature Ve...
The eBay Architecture: Striking a Balance between Site Stability, Feature Ve...
 
VMworld 2013: Data In, Data Out and Data Protected
VMworld 2013: Data In, Data Out and Data Protected VMworld 2013: Data In, Data Out and Data Protected
VMworld 2013: Data In, Data Out and Data Protected
 
DEVNET-1140 InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
DEVNET-1140	InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...DEVNET-1140	InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
DEVNET-1140 InterCloud Mapreduce and Spark Workload Migration and Sharing: Fi...
 
Microservices Architecture & Testing Strategies
Microservices Architecture & Testing StrategiesMicroservices Architecture & Testing Strategies
Microservices Architecture & Testing Strategies
 

Andere mochten auch

Neurotransmission, Neuropsychiatry, and Neuropharmacology 2013
Neurotransmission, Neuropsychiatry, and Neuropharmacology 2013 Neurotransmission, Neuropsychiatry, and Neuropharmacology 2013
Neurotransmission, Neuropsychiatry, and Neuropharmacology 2013 dfsmithdfsmith
 
Lino printing presentation
Lino printing presentationLino printing presentation
Lino printing presentationGentiana Gentaa
 
Analysing web statistics
Analysing web statisticsAnalysing web statistics
Analysing web statisticsiteclearners
 
Asterisk y Cisco HA Conceptos - Webinar Bitsense y Proydesa
Asterisk y Cisco HA Conceptos - Webinar Bitsense y ProydesaAsterisk y Cisco HA Conceptos - Webinar Bitsense y Proydesa
Asterisk y Cisco HA Conceptos - Webinar Bitsense y ProydesaLuis Adrian Amato
 
Equivalent decimal numbers
Equivalent decimal numbersEquivalent decimal numbers
Equivalent decimal numbersAmmery Stuart
 
Understanding printing techniques
Understanding printing techniquesUnderstanding printing techniques
Understanding printing techniquesaanmolgudka
 
Gamesys graph-connect-london
Gamesys graph-connect-londonGamesys graph-connect-london
Gamesys graph-connect-londonNeo4j
 

Andere mochten auch (12)

Appraisly November 2013
Appraisly November 2013Appraisly November 2013
Appraisly November 2013
 
Neurotransmission, Neuropsychiatry, and Neuropharmacology 2013
Neurotransmission, Neuropsychiatry, and Neuropharmacology 2013 Neurotransmission, Neuropsychiatry, and Neuropharmacology 2013
Neurotransmission, Neuropsychiatry, and Neuropharmacology 2013
 
Wwwwwwww
WwwwwwwwWwwwwwww
Wwwwwwww
 
computer form factor
 computer form factor computer form factor
computer form factor
 
Lino printing presentation
Lino printing presentationLino printing presentation
Lino printing presentation
 
Analysing web statistics
Analysing web statisticsAnalysing web statistics
Analysing web statistics
 
05 conjuntos
05 conjuntos05 conjuntos
05 conjuntos
 
Asterisk y Cisco HA Conceptos - Webinar Bitsense y Proydesa
Asterisk y Cisco HA Conceptos - Webinar Bitsense y ProydesaAsterisk y Cisco HA Conceptos - Webinar Bitsense y Proydesa
Asterisk y Cisco HA Conceptos - Webinar Bitsense y Proydesa
 
Equivalent decimal numbers
Equivalent decimal numbersEquivalent decimal numbers
Equivalent decimal numbers
 
Understanding printing techniques
Understanding printing techniquesUnderstanding printing techniques
Understanding printing techniques
 
Gamesys graph-connect-london
Gamesys graph-connect-londonGamesys graph-connect-london
Gamesys graph-connect-london
 
2 1
2 12 1
2 1
 

Ähnlich wie Using Graph Databases in Real-Time to Solve Resource Authorization at Telenor - Sebastian Verheughe @ GraphConnect London 2013

Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...
Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...
Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...Neo4j
 
Peek into Neo4j Product Strategy and Roadmap
Peek into Neo4j Product Strategy and RoadmapPeek into Neo4j Product Strategy and Roadmap
Peek into Neo4j Product Strategy and RoadmapNeo4j
 
PLNOG 3: Tomasz Mikołajczyk - Data scalability. Why you should care?
PLNOG 3: Tomasz Mikołajczyk -  Data scalability. Why you should care?PLNOG 3: Tomasz Mikołajczyk -  Data scalability. Why you should care?
PLNOG 3: Tomasz Mikołajczyk - Data scalability. Why you should care?PROIDEA
 
Maximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
Maximizing Data Lake ROI with Data Virtualization: A Technical DemonstrationMaximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
Maximizing Data Lake ROI with Data Virtualization: A Technical DemonstrationDenodo
 
Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )
Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )
Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )Rajendra Kumar Sahu
 
Importing Large Sets of Content from Trusted Partners into your Repository
Importing Large Sets of Content from Trusted Partners into your RepositoryImporting Large Sets of Content from Trusted Partners into your Repository
Importing Large Sets of Content from Trusted Partners into your RepositoryBlueFish
 
Grab: Building a Healthy Elasticsearch Ecosystem
Grab: Building a Healthy Elasticsearch EcosystemGrab: Building a Healthy Elasticsearch Ecosystem
Grab: Building a Healthy Elasticsearch EcosystemElasticsearch
 
Building data pipelines at Shopee with DEC
Building data pipelines at Shopee with DECBuilding data pipelines at Shopee with DEC
Building data pipelines at Shopee with DECRim Zaidullin
 
Presenting Data – An Alternative to the View Control
Presenting Data – An Alternative to the View ControlPresenting Data – An Alternative to the View Control
Presenting Data – An Alternative to the View ControlTeamstudio
 
Yuriy Chapran - Building microservices.
Yuriy Chapran - Building microservices.Yuriy Chapran - Building microservices.
Yuriy Chapran - Building microservices.Yuriy Chapran
 
Apricot2017 Request tracing in distributed environment
Apricot2017 Request tracing in distributed environmentApricot2017 Request tracing in distributed environment
Apricot2017 Request tracing in distributed environmentHieu LE ☁
 
Contino Webinar - Migrating your Trading Workloads to the Cloud
Contino Webinar -  Migrating your Trading Workloads to the CloudContino Webinar -  Migrating your Trading Workloads to the Cloud
Contino Webinar - Migrating your Trading Workloads to the CloudBen Saunders
 
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 20091 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009Moshe Kaplan
 

Ähnlich wie Using Graph Databases in Real-Time to Solve Resource Authorization at Telenor - Sebastian Verheughe @ GraphConnect London 2013 (20)

Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...
Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...
Using Graph Databases in Real-time to Solve Resource Authorization at Telenor...
 
Peek into Neo4j Product Strategy and Roadmap
Peek into Neo4j Product Strategy and RoadmapPeek into Neo4j Product Strategy and Roadmap
Peek into Neo4j Product Strategy and Roadmap
 
Madhu_Resume
Madhu_ResumeMadhu_Resume
Madhu_Resume
 
PLNOG 3: Tomasz Mikołajczyk - Data scalability. Why you should care?
PLNOG 3: Tomasz Mikołajczyk -  Data scalability. Why you should care?PLNOG 3: Tomasz Mikołajczyk -  Data scalability. Why you should care?
PLNOG 3: Tomasz Mikołajczyk - Data scalability. Why you should care?
 
DevOps Case Studies
DevOps Case StudiesDevOps Case Studies
DevOps Case Studies
 
VamsiKrishna Maddiboina
VamsiKrishna MaddiboinaVamsiKrishna Maddiboina
VamsiKrishna Maddiboina
 
Subbu_WM
Subbu_WMSubbu_WM
Subbu_WM
 
CV_DebarpanMukherjee
CV_DebarpanMukherjeeCV_DebarpanMukherjee
CV_DebarpanMukherjee
 
mayank_unix_sql_Jboss
mayank_unix_sql_Jbossmayank_unix_sql_Jboss
mayank_unix_sql_Jboss
 
Maximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
Maximizing Data Lake ROI with Data Virtualization: A Technical DemonstrationMaximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
Maximizing Data Lake ROI with Data Virtualization: A Technical Demonstration
 
Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )
Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )
Rajendra Kumar Sahu_243535(Maximo 7 5 Certified )
 
Importing Large Sets of Content from Trusted Partners into your Repository
Importing Large Sets of Content from Trusted Partners into your RepositoryImporting Large Sets of Content from Trusted Partners into your Repository
Importing Large Sets of Content from Trusted Partners into your Repository
 
Grab: Building a Healthy Elasticsearch Ecosystem
Grab: Building a Healthy Elasticsearch EcosystemGrab: Building a Healthy Elasticsearch Ecosystem
Grab: Building a Healthy Elasticsearch Ecosystem
 
JKSQL
JKSQLJKSQL
JKSQL
 
Building data pipelines at Shopee with DEC
Building data pipelines at Shopee with DECBuilding data pipelines at Shopee with DEC
Building data pipelines at Shopee with DEC
 
Presenting Data – An Alternative to the View Control
Presenting Data – An Alternative to the View ControlPresenting Data – An Alternative to the View Control
Presenting Data – An Alternative to the View Control
 
Yuriy Chapran - Building microservices.
Yuriy Chapran - Building microservices.Yuriy Chapran - Building microservices.
Yuriy Chapran - Building microservices.
 
Apricot2017 Request tracing in distributed environment
Apricot2017 Request tracing in distributed environmentApricot2017 Request tracing in distributed environment
Apricot2017 Request tracing in distributed environment
 
Contino Webinar - Migrating your Trading Workloads to the Cloud
Contino Webinar -  Migrating your Trading Workloads to the CloudContino Webinar -  Migrating your Trading Workloads to the Cloud
Contino Webinar - Migrating your Trading Workloads to the Cloud
 
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 20091 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009
1 Billion Events per Day, Israel 3rd Java Technology Day, June 22, 2009
 

Mehr von Neo4j

Webinar - IA generativa e grafi Neo4j: RAG time!
Webinar - IA generativa e grafi Neo4j: RAG time!Webinar - IA generativa e grafi Neo4j: RAG time!
Webinar - IA generativa e grafi Neo4j: RAG time!Neo4j
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeNeo4j
 
Neo4j: Data Engineering for RAG (retrieval augmented generation)
Neo4j: Data Engineering for RAG (retrieval augmented generation)Neo4j: Data Engineering for RAG (retrieval augmented generation)
Neo4j: Data Engineering for RAG (retrieval augmented generation)Neo4j
 
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdfNeo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdfNeo4j
 
Enabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge GraphsEnabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge GraphsNeo4j
 
Neo4j_Anurag Tandon_Product Vision and Roadmap.Benelux.pptx.pdf
Neo4j_Anurag Tandon_Product Vision and Roadmap.Benelux.pptx.pdfNeo4j_Anurag Tandon_Product Vision and Roadmap.Benelux.pptx.pdf
Neo4j_Anurag Tandon_Product Vision and Roadmap.Benelux.pptx.pdfNeo4j
 
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdfNeo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdfNeo4j
 
SWIFT: Maintaining Critical Standards in the Financial Services Industry with...
SWIFT: Maintaining Critical Standards in the Financial Services Industry with...SWIFT: Maintaining Critical Standards in the Financial Services Industry with...
SWIFT: Maintaining Critical Standards in the Financial Services Industry with...Neo4j
 
Deloitte & Red Cross: Talk to your data with Knowledge-enriched Generative AI
Deloitte & Red Cross: Talk to your data with Knowledge-enriched Generative AIDeloitte & Red Cross: Talk to your data with Knowledge-enriched Generative AI
Deloitte & Red Cross: Talk to your data with Knowledge-enriched Generative AINeo4j
 
Ingka Digital: Linked Metadata by Design
Ingka Digital: Linked Metadata by DesignIngka Digital: Linked Metadata by Design
Ingka Digital: Linked Metadata by DesignNeo4j
 
Discover Neo4j Aura_ The Future of Graph Database-as-a-Service Workshop_3.13.24
Discover Neo4j Aura_ The Future of Graph Database-as-a-Service Workshop_3.13.24Discover Neo4j Aura_ The Future of Graph Database-as-a-Service Workshop_3.13.24
Discover Neo4j Aura_ The Future of Graph Database-as-a-Service Workshop_3.13.24Neo4j
 
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptxGraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptxNeo4j
 
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptxEmil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptxNeo4j
 
Identification of insulin-resistance genes with Knowledge Graphs topology and...
Identification of insulin-resistance genes with Knowledge Graphs topology and...Identification of insulin-resistance genes with Knowledge Graphs topology and...
Identification of insulin-resistance genes with Knowledge Graphs topology and...Neo4j
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNeo4j
 
EY: Graphs as Critical Enablers for LLM-based Assistants- the Case of Custome...
EY: Graphs as Critical Enablers for LLM-based Assistants- the Case of Custome...EY: Graphs as Critical Enablers for LLM-based Assistants- the Case of Custome...
EY: Graphs as Critical Enablers for LLM-based Assistants- the Case of Custome...Neo4j
 
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptxGraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptxNeo4j
 
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxThe Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxNeo4j
 
KUBRICK Graphs: A journey from in vogue to success-ion
KUBRICK Graphs: A journey from in vogue to success-ionKUBRICK Graphs: A journey from in vogue to success-ion
KUBRICK Graphs: A journey from in vogue to success-ionNeo4j
 
SKY Paradigms, change and cake: the steep curve of introducing new technologies
SKY Paradigms, change and cake: the steep curve of introducing new technologiesSKY Paradigms, change and cake: the steep curve of introducing new technologies
SKY Paradigms, change and cake: the steep curve of introducing new technologiesNeo4j
 

Mehr von Neo4j (20)

Webinar - IA generativa e grafi Neo4j: RAG time!
Webinar - IA generativa e grafi Neo4j: RAG time!Webinar - IA generativa e grafi Neo4j: RAG time!
Webinar - IA generativa e grafi Neo4j: RAG time!
 
IA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG timeIA Generativa y Grafos de Neo4j: RAG time
IA Generativa y Grafos de Neo4j: RAG time
 
Neo4j: Data Engineering for RAG (retrieval augmented generation)
Neo4j: Data Engineering for RAG (retrieval augmented generation)Neo4j: Data Engineering for RAG (retrieval augmented generation)
Neo4j: Data Engineering for RAG (retrieval augmented generation)
 
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdfNeo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
Neo4j Graph Summit 2024 Workshop - EMEA - Breda_and_Munchen.pdf
 
Enabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge GraphsEnabling GenAI Breakthroughs with Knowledge Graphs
Enabling GenAI Breakthroughs with Knowledge Graphs
 
Neo4j_Anurag Tandon_Product Vision and Roadmap.Benelux.pptx.pdf
Neo4j_Anurag Tandon_Product Vision and Roadmap.Benelux.pptx.pdfNeo4j_Anurag Tandon_Product Vision and Roadmap.Benelux.pptx.pdf
Neo4j_Anurag Tandon_Product Vision and Roadmap.Benelux.pptx.pdf
 
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdfNeo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
Neo4j_Jesus Barrasa_The Art of the Possible with Graph.pptx.pdf
 
SWIFT: Maintaining Critical Standards in the Financial Services Industry with...
SWIFT: Maintaining Critical Standards in the Financial Services Industry with...SWIFT: Maintaining Critical Standards in the Financial Services Industry with...
SWIFT: Maintaining Critical Standards in the Financial Services Industry with...
 
Deloitte & Red Cross: Talk to your data with Knowledge-enriched Generative AI
Deloitte & Red Cross: Talk to your data with Knowledge-enriched Generative AIDeloitte & Red Cross: Talk to your data with Knowledge-enriched Generative AI
Deloitte & Red Cross: Talk to your data with Knowledge-enriched Generative AI
 
Ingka Digital: Linked Metadata by Design
Ingka Digital: Linked Metadata by DesignIngka Digital: Linked Metadata by Design
Ingka Digital: Linked Metadata by Design
 
Discover Neo4j Aura_ The Future of Graph Database-as-a-Service Workshop_3.13.24
Discover Neo4j Aura_ The Future of Graph Database-as-a-Service Workshop_3.13.24Discover Neo4j Aura_ The Future of Graph Database-as-a-Service Workshop_3.13.24
Discover Neo4j Aura_ The Future of Graph Database-as-a-Service Workshop_3.13.24
 
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptxGraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
GraphSummit Copenhagen 2024 - Neo4j Vision and Roadmap.pptx
 
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptxEmil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
Emil Eifrem at GraphSummit Copenhagen 2024 - The Art of the Possible.pptx
 
Identification of insulin-resistance genes with Knowledge Graphs topology and...
Identification of insulin-resistance genes with Knowledge Graphs topology and...Identification of insulin-resistance genes with Knowledge Graphs topology and...
Identification of insulin-resistance genes with Knowledge Graphs topology and...
 
Novo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4jNovo Nordisk's journey in developing an open-source application on Neo4j
Novo Nordisk's journey in developing an open-source application on Neo4j
 
EY: Graphs as Critical Enablers for LLM-based Assistants- the Case of Custome...
EY: Graphs as Critical Enablers for LLM-based Assistants- the Case of Custome...EY: Graphs as Critical Enablers for LLM-based Assistants- the Case of Custome...
EY: Graphs as Critical Enablers for LLM-based Assistants- the Case of Custome...
 
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptxGraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
GraphSummit London Feb 2024 - ABK - Neo4j Product Vision and Roadmap.pptx
 
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptxThe Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
The Art of the Possible with Graph by Dr Jim Webber Neo4j.pptx
 
KUBRICK Graphs: A journey from in vogue to success-ion
KUBRICK Graphs: A journey from in vogue to success-ionKUBRICK Graphs: A journey from in vogue to success-ion
KUBRICK Graphs: A journey from in vogue to success-ion
 
SKY Paradigms, change and cake: the steep curve of introducing new technologies
SKY Paradigms, change and cake: the steep curve of introducing new technologiesSKY Paradigms, change and cake: the steep curve of introducing new technologies
SKY Paradigms, change and cake: the steep curve of introducing new technologies
 

Kürzlich hochgeladen

NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopBachir Benyammi
 
Introduction to Quantum Computing
Introduction to Quantum ComputingIntroduction to Quantum Computing
Introduction to Quantum ComputingGDSC PJATK
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfDaniel Santiago Silva Capera
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncObject Automation
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPathCommunity
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemAsko Soukka
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfAnna Loughnan Colquhoun
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaborationbruanjhuli
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1DianaGray10
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6DianaGray10
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024SkyPlanner
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureEric D. Schabell
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?SANGHEE SHIN
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8DianaGray10
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostMatt Ray
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsSeth Reyes
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Adtran
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxMatsuo Lab
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Will Schroeder
 
20200723_insight_release_plan
20200723_insight_release_plan20200723_insight_release_plan
20200723_insight_release_planJamie (Taka) Wang
 

Kürzlich hochgeladen (20)

NIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 WorkshopNIST Cybersecurity Framework (CSF) 2.0 Workshop
NIST Cybersecurity Framework (CSF) 2.0 Workshop
 
Introduction to Quantum Computing
Introduction to Quantum ComputingIntroduction to Quantum Computing
Introduction to Quantum Computing
 
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdfIaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
IaC & GitOps in a Nutshell - a FridayInANuthshell Episode.pdf
 
GenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation IncGenAI and AI GCC State of AI_Object Automation Inc
GenAI and AI GCC State of AI_Object Automation Inc
 
UiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation DevelopersUiPath Community: AI for UiPath Automation Developers
UiPath Community: AI for UiPath Automation Developers
 
Bird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystemBird eye's view on Camunda open source ecosystem
Bird eye's view on Camunda open source ecosystem
 
Spring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdfSpring24-Release Overview - Wellingtion User Group-1.pdf
Spring24-Release Overview - Wellingtion User Group-1.pdf
 
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online CollaborationCOMPUTER 10: Lesson 7 - File Storage and Online Collaboration
COMPUTER 10: Lesson 7 - File Storage and Online Collaboration
 
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1UiPath Platform: The Backend Engine Powering Your Automation - Session 1
UiPath Platform: The Backend Engine Powering Your Automation - Session 1
 
UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6UiPath Studio Web workshop series - Day 6
UiPath Studio Web workshop series - Day 6
 
Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024Salesforce Miami User Group Event - 1st Quarter 2024
Salesforce Miami User Group Event - 1st Quarter 2024
 
OpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability AdventureOpenShift Commons Paris - Choose Your Own Observability Adventure
OpenShift Commons Paris - Choose Your Own Observability Adventure
 
Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?Do we need a new standard for visualizing the invisible?
Do we need a new standard for visualizing the invisible?
 
UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8UiPath Studio Web workshop series - Day 8
UiPath Studio Web workshop series - Day 8
 
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCostKubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
KubeConEU24-Monitoring Kubernetes and Cloud Spend with OpenCost
 
Computer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and HazardsComputer 10: Lesson 10 - Online Crimes and Hazards
Computer 10: Lesson 10 - Online Crimes and Hazards
 
Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™Meet the new FSP 3000 M-Flex800™
Meet the new FSP 3000 M-Flex800™
 
Introduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptxIntroduction to Matsuo Laboratory (ENG).pptx
Introduction to Matsuo Laboratory (ENG).pptx
 
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
Apres-Cyber - The Data Dilemma: Bridging Offensive Operations and Machine Lea...
 
20200723_insight_release_plan
20200723_insight_release_plan20200723_insight_release_plan
20200723_insight_release_plan
 

Using Graph Databases in Real-Time to Solve Resource Authorization at Telenor - Sebastian Verheughe @ GraphConnect London 2013

  • 1. Using Graph Databases in Real-Time to Solve Resource Authorization at Telenor Graph Connect London – 19 Nov 2013 by Sebastian Verheughe
  • 2. Telenor Norway Subsidiary of the Telenor Group ~1 billion GBP in mobile revenues 2012 Sebastian Verheughe Lead Developer for Neo4j solution Coding Architect
  • 3. Disclaimer The presentation is not identical to the implementation due to security reasons but shows how we have modeled and solved the problem in general. However, all presented data (numbers & charts) are real, unfiltered and extracted from the production logs
  • 5. Telenor Norway Middleware Services Channel Channel used by 42 channels calls 35 sub-systems 10,000 code classes 500 requests/second 20,000 orders/day Backend Backend Channel Channel MOBILE MW Channel Channel Providing business logic and data for all channels in the mobile value chain BUSINESS LOGIC & DATA Backend Handles users with access to X00,000 resources Backend Backend Backend
  • 6. Our Problem 20 minutes to calculate all accessible resources 1500 lines of SQL to implement the authorization logic “solved” by caching data going stale and the solution did not scale…
  • 7. Why a Graph Database? Access Parent Company User: R. Branson Which resources does the user have access to? Part of Company Sales Finance Production HR Sub The questions we wanted answered required traversal of tree structures. Tablet Subscription Owner Sub Tablet Uses Subscription Phone
  • 8. Tailored Read Model The Model makes read queries as simple and efficient as possible. First find your questions then model your graph graph model = relational model
  • 9. High Level Architecture Clients Classic MW Services other sources tx log RDBMS check access Resource Authorization Message Queue Neo4j
  • 10. Conditional Rules ACCESS is given with the following include parameters: access to subsidiaries and access to content Only find children of PARENT COMPANY given access to subsidiaries is allowed User Only look at PART OF COMPANY given access to content is allowed Only look at SUBSCRIPTION OWNER given access to content is allowed
  • 11. Different Access Needs Access Subsidiaries & Content Super Admin Access Content Umbrella Admin Access S&C Admin
  • 12. Graph Algorithm Prerequisite: The user node 1. Follow all ACCESS relationships and read the access parameters on the relationship 2. Follow all PARENT COMPANY relationships given access to subsidiaries is allowed 3. Follow all PART OF COMPANY relationships given access to content is allowed 4. Follow all SUBSCRIPTION OWNER relationships given access to content is allowed
  • 13. Solution Value 1. Performance optimized from minutes to seconds. 2. Simplicity of writing and understanding business rules for the query traversal. 3. Scalability by performance allowing us to onboard more corporate customers (project business case) Autonomous Service with it’s own life-cycle and data repository.
  • 14. Authorization Complexity • Not a collection of isolated customer trees * • Not all users of a customer have equal access • Not a fixed schema, form or size for all customers • Real-time updated with customer & product data The data form a highly connected living graph * Covered later in Technical Details
  • 15. How we Started with Neo4j 1. Searched the internet for articles about graph database and different solutions. 2. Downloaded and quickly prototyped the solution we liked that matched our requirements (Neo4j). 3. Workshop with Neo4j and our project developers to quickly gain competence and ensure design QA. 4. Solution QA with Neo4j before production and help with performance issues / tuning.
  • 16. Lessons Learned • Choose a solution/technology that fits your problem • New way of thinking – build competence in org. • Profile your java code to make it really fast • Don’t put everything into the graph (functional creep) • Need to know how traversal works (e.g. shortest path) • Benchmark the graph to evaluate your traversal speed
  • 17. Alternative In-Memory RDBMS Option 1: Use existing database - Performance issues due to shared data / suboptimal structure - Complexity since SQL not designed for traversal Option 2: Separate database + Might reach same performance as graph db + Familiar technology - Complexity since SQL not designed for traversal Decided to go with our instinct Graph Database
  • 18. Different Graph Structures get all accessible subscriptions 2 000 1 000 1 700 ms Company X: 147 000 750 ms Company Y: 52 000 1300 ms Company Z: 95 000 Data from test – repeated prod sampling gave ~2.4 sec for 215,000 subscriptions
  • 19. Different Graph Structures check access to single subscription 2 000 1 000 1 ms Company X: 147 000 1 ms Company Y: 52 000 1 ms Company Z: 95 000
  • 20. Production Performance retrieve all accessible resources RDBMS Disk RDBMS Mem Cached Graph In-Heap Company X 12 min 18 sec < 2 sec Company Y 22 min 58 sec < 2 sec Company Z 3 min 15 sec < 2 sec Check single resource access 1 ms No operational problems in production
  • 22. Production Details Graph Size heap) 27 million nodes (pre-warmed in ~1x properties, ~2x relationships Traffic Volume ~1000 req/min during biz hours ~ 40K daily real-time updates Performance ms Avg: 1 ms, 99% < 4 ms, 99.9% < 9 JVM warmed) Sun 6, 20 GB Heap (~15 GB pre-
  • 23. Production Has Access Query Time (ms) Time (ms)
  • 24. Production All Queries Time (ms) Garbage collection
  • 25. Implementing the Algorithm Lets look at the Neo4j Traversal Framework Iterable<Node> getAccessibleResources(…) { Evaluator myEvaluator = … Expander myExpander = … return Traversal.description() .evaluator(myEvaluator) .expander(myExpander) .traverse(startNode).nodes(); }
  • 26. Implementing the Algorithm Evaluator is a simple filter, e.g. for Node type class MyEvaluator implements Evaluator { public Evaluation evaluate(Path path) { if <I am interested in this node> return Evaluation.INCLUDE_AND_CONTINUE; else return Evaluation.EXCLUDE_AND_CONTINUE; } }
  • 27. Implementing the Algorithm The custom Expander contains business rules! class ResAuthExpander implements PathExpander<PathExpander> { … public … expand(Path path, BranchState<…> state) { if (path.lastRelationship rel == ACCESS) accToSub = rel.getProperty(ACCESS_TO_SUBSIDIARIES); accToCont = rel.getProperty(ACCESS_TO_CONTENT); state.set( getExpander(accToSub, accToCont) ); } return state.get().expand(…) } Single expander class to control business
  • 28. Implementing the Algorithm Generates the valid relationships to traverse. public getExpander(boolean accToSub, boolean accToCont) { PathExpander exp = StandardExpander.DEFAULT.add(ACCESS,…); if (accToSub) exp.add(PARENT_COMPANY,…) if (accToCont) exp.add(PART_OF_COMPANY,…).add(SUBSCRIPTION_OWNER,…); return exp; } }
  • 29. U-Turn Strategy 4. Access User Does the user have access to subscription X? 3. 5. Up to find path quickly Down to check access 6. 2. 7. 1. 8. X Subscription Reversing the traversal increases performance from n/2 to 2d where n and d are tree size and depth (we went from 1s to
  • 30. The Zigzag Problem What if we also have reversed access to the subscription payer? User Op IT E d Jo Subscriptions Solvable by adding state to the traversal (or check path)
  • 31. The Many-to-Many Problem The nodes Op & IT may be connected through many subscriptions Does the user have access to department Op? Op IT Access User Subscription Traversal becomes time consuming (e.g. M2M market) However, we only needed to implement the rule for direct access to sub.
  • 32. Deployment View • Two equal instances of Neo4j embedded in Tomcat • Access through Java API due to need for custom logic • Using Neo4j 1.8 without HA (did not like ZooKeeper) Resource Authorization Neo4j tx log RDBMS Message Queue Resource Authorization Neo4j
  • 33. Dual Model Cost There are some drawbacks with dual models also • Not possible to simply join the ACL with resource tables in the relational database - queries needed redesign • The complexity added by code and infrastructure necessary to manage an additional model. • Not ordinary competence (in Norway at least)
  • 34. Unexplored Areas Combining Access Control List & Graph • Best of both worlds (simple logic, fast lookup) Algorithm – – – – Find all affected users when the graph is updated Invalidate users access control list Calculate all accessible resources for each user Store result in users access control list Could then skip the U-turn and many-to-many problem.
  • 35. Was is worth it? Yes! The user experience is important in Telenor
  • 37. Web References • Telenor Norway • The Project - How NOSQL Paid off for Telenor • JavaWorld - Graphs for Security

Hinweis der Redaktion

  1. Why: access to secret numbers, access to modify/delete subscriptions, possibility to send/receive messages
  2. We use Neo4j for our business critical services, both customer/product services , but also operational services.A channel is client type, e.g. the web solution for corporate customer, or helpdesk solution, or app, and may consist of many clients
  3. The project business case was based on a future point in time where we could not any more onboard any large corporate customers
  4. Drawing on the white board the required logic made us understand that a graph database might be a good solution
  5. Take the hit on write, and make read easy! (for us, read performance is the problem – not write performance)Also, don’t blindly copy tables/foreign keys into nodes and relationships – drop what’s not needed and remember that relationships may have properties in graph
  6. RDBMS is still mastering the data as it is used in many different use-cases where that is beneficial.
  7. TIME LEFT: 30 minutes (10 used)
  8. The last part is important to us. It was really hard to extract the resource authorization out of the relational database, but not we can much more easy replace the current implementation with another one in the future if neccesary.
  9. Production logs does not contain user data, so just one big organization was sampled to get production data for a specific customer
  10. TIME LEFT: 20 minutes (20 used)Graphperformance based on test environment, see charts in the technical section for production numbers not specific for a unique customer
  11. We only have detailed logs for a short while back – so we cannot review all data since production.First production two years ago with limited traffic, full production since spring 2013
  12. We always continue, since we also have our custom expander. This way, we have a clean separation of concern in our code.We also have more advanced filters peaking around the node before it decides to include or exclude the node
  13. This is the most important part of the code, the one place where we now are able to write down the business logic in a simple and natural way.Note that we only have ONE class containing the business rules independently of which use-case we are running.
  14. The relationships and directions that are allowed to traverse given the different switch parameters.
  15. This is possible since we have a tree graph. Demonstrates the importance of understanding how a graph works, because than you may greatly improve performance by smart traversal strategies.
  16. TIME LEFT: 10 minutes (30 used)
  17. Extra knowledge, such as which subscription you are