See all on-demand Graph + AI Sessions: https://www.tigergraph.com/graph-ai-world-sessions/ Get TigerGraph: https://www.tigergraph.com/get-tigergraph/

1. 1
Graph Hardware Architecture
Enterprise graphs deserve great hardware!
Dan McCreary, Distinguished Engineer, Optum
Nikhil Deshpande, Director, AI & HPC, Intel Corp.
Graph + AI World 2020
September 29, 2020

2. About Dan
2
• Distinguished Engineer at Optum Healthcare
(330K employees and 32K technical staff)
• Focused on AI enterprise knowledge graphs
• Help create the worlds largest healthcare
graph
• Coauthor of book "Making Sense of NoSQL”
• Worked at Bell Labs as a VLSI circuit designer
• Worked for Steve Jobs at NeXT

3. Clinical Data Fits Well With Graph Database Models
3
Clinical
Data
Graph
• There is a natural fit between clinical data and graph technologies
• Clinical data is both complex and shows high-variability
• Graphs are ideal for storing complex and highly-variable data

4. Business Impact of Moving from RDBMS to Graph
4
Relational Graph Business Impact of Using Graph for
Clinical Data
Relationships calculated at
query time
Relationships calculated at
load time
Relationship-intensive clinical data is
queried faster
Fixed data model Flexible data model It is easier to add new clinical data to a
graph at any time
Limited analytical algorithms Rich library of analytical
algorithms such as pathfinding,
PageRank, clustering, random
walk and similarity
It is easy to execute patient clustering,
similarity and care path recommendation
engines
Difficult to build custom
hardware to optimize
algorithms
Easy to build custom hardware
to optimize algorithms
Hardware for optimizing algorithms is
low cost

5. The Real Time Clinical Decision Support Challenge
5
• A new patient walks into the ER, urgent care or our clinic
• We gather patient data into the Electronic Medical Record (EMR)
• How quickly can we compare this patient with the best outcomes of 250
million other patients? Goal: 200 msec
250M Patients
Patient Data
Recommended
Care Path

6. How We Got Here: Pointer Hopping
• Several years ago we started working on graph scaling
problems
• Graph traversal is they key function that must scale in
parallel
• Graph traversal is all about pointer hopping
• Having a VLSI circuit design background help me
visualize how narrow the instruction sets were needed to
do this quickly
• I realized that both the instruction set and the memory
access patterns were not optimized with modern
general purpose CISC hardware
• We needed a new distributed graph architecture with
many cores and faster memory access patterns
6

7. Example: Learning Embeddings Via Random Walk
• NLP technologies have shown us that we can
create low-dimensional embeddings for every
word by looking at the sequence of words in text.
• Using Random Walk algorithms we can create a
similar sequence of items to create embeddings
for primary Vertices (Customer, Product etc.)
• We can use these embeddings for 50 msec
comparisons over 100M items if we do the
calculations in parallel
• Random Walks are very CPU intensive and
require traversal of hundreds of millions of
edges
7
Embedding: 200 32-bit integer

8. 8Intel Confidential 8
RoleofHardware...
q Graph based analytics is the next frontier
q Architectures need to evolve to support next-gen graph
usages in healthcare
q Intel® Programmable Integrated Unified Memory
Architecture (PIUMA)
q ~1,000x Speedup on graphs!
Copyright © 2020, Intel Corporation

9. 9Intel Confidential 9
Regular versus GraphApplications
Branch
Prediction
Locality
Data Access
Regular Graph
Branches have regular
pattern
Same or neighboring
data likely used
Same operation on
neighboring data
Branch outcome is
data dependent
(“Pointer hopping”)
Data is randomly
scattered
Operations on
scattered data
Copyright © 2020, Intel Corporation

10. 10Intel Confidential 10
TraditionalComputeBottlenecksforGraph
AdvancedNIC
AdvancedNIC
Core level: Cache
Socket level: DRAM
limitations
Network Level: high
latency/SW layers
AdvancedNIC
AdvancedNIC
Rack Network
Infrastructure Level: designed
for big messages
Copyright © 2020, Intel Corporation

11. 11Intel Confidential 11
Predictive Analytics / Graph Characteristics
Data Size
Communications
Intensity
Search, Hadoop
Web services,
IT, CRM, E-
commerce,
Gaming,
Security,
SaaS, IaaS
Predictive
Analytics in
Graphs
Scalability
Wall
Dense Matrix Multiplication
Random walk
Sub-graph search
Page Rank
The scalability wall is
the limit to
performance scaling
Copyright © 2020, Intel Corporation

12. 12Intel Confidential 12
ImpactofHitting Scalability Wall
See backup for configuration details. For more complete information about performance and benchmark results,
visit www.intel.com/benchmarks.
GTEPS: Giga Traversed Edges
Per Second
GTEPS(measureaffectedbycomms.intensity)
Random Walks – OpenMP
Input Data Size (GB)
0
20
40
60
80
100
120
140
160
16 80 125 226 318 429 459 744
GFLOPS
Input Data Size (MB)
Page Rank / SpMV
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 4 8 12 24
28
Copyright © 2020, Intel Corporation

13. 13Intel Confidential 13
Intel®PIUMA Design Principles
• New architecture to address
scalability wall
• Balanced I/O, memory and
compute
• General programmable
accelerator
• Real time query on large
graph datasets constantly
being updated
Intel® PIUMA
Compute Memory I/O
Traditional
Optim
ized
for
TOPS
Optim
ized
for
TOPS
and
TTEPS
TOPS: Tera-Operations Per Second
TTEPS: Tera-Traversed Edges Per Second
Copyright © 2020, Intel Corporation

14. 14Intel Confidential 14
Intel®PIUMA core
Cores are designed for throughput or single-thread
performance
Novel instruction set with native 64-bit physical
addresses and graph operation optimizations
Distributed global address space (GAS) for all memory
visible by all cores
Memory and fabric subsystems optimized for both 8B
and 64B transfers
Copyright © 2020, Intel Corporation

15. 15Intel Confidential 15
Intel® PIUMA System
Intel® PIUMA
HW
Runtime/Tools
C/C++/Python
Graph
Scalable SW and
Hardware
• Extremely high bandwidth, low energy and latency
direct fabric (TB/s Vs GB/s)
• Optimized for Random Accesses to PB of memory
• Balanced Compute to I/O Ratio
C
M
M
M
C
M
C
M
M
C
M
C
M
M
C
Intel® PIUMA
M
Multi-threaded
compute
Memory optimized
for TEPs
C CC
C
Compute Nodes
Today’s Architecture
• Limited I/O BW & High Latency Networks
• Low bandwidth to Memory
• High Compute to I/O Ratio (Unbalanced)
C M
C
M
M
M
C
M
C
M
C
M
C
M
C
M
C
M
C
C
Traditional Compute Memory optimized
for FLOPs
Compute Nodes
Traditional Switch
Copyright © 2020, Intel Corporation

16. 16Intel Confidential 16
Intel®PIUMA:SpeedUpGraphTraversalby>1,000x
Performance
Graph Search (Random Walks)
>1,000x
No. of Communicating Systems
See backup for configuration details. For more complete information about performance and benchmark results,
visit www.intel.com/benchmarks.
Intel® PIUMA
System and Fabric
Traditional Fabric
Linear Scaling
Copyright © 2020, Intel Corporation
Accelerated
Enterprise
Knowledge Graph
Queries

17. Conclusion: Think BIG about your EKG!
• We encourage everyone to continue to
“think big” when doing strategic planning
around your Enterprise Knowledge Graphs
• Many organizations besides Google,
Amazon, Facebook etc. will be building 100B
to 1TB vertex graphs in the next few years to
better serve their customers
• New hardware will soon be arriving that will
dramatically accelerate today’s graph
queries
• Real time product recommendation systems
that take complex content into account will
be cost effective a accelerate data-driven
decision making in healthcare and other
industries
17