Dr. James Coomer discusses DataDirect Network's partnership with Intel for exascale at the Intel Theater

1. ddn.com©2012 DataDirect Networks. All Rights Reserved.
DDN and Intel – Partnered for Exascale
ISC’13 edition
June 2013
Dr. James Coomer
Senior Technical Advisor for EMEA

2. ddn.com©2012 DataDirect Networks. All Rights Reserved.
DDN | About Us
► Established: 1998
► Revenue: $250M+ – Profitable, Fast Growth
► Main Office: Sunnyvale, California, USA
► Employees: ~650 Worldwide
► Worldwide Presence: 20 Countries
► Installed Base: 1,000+ End Customers; 50+ Countries
► Go To Market: Global Partners, Resellers, Direct
► Top HPC Sites:
• > 60% of Top100
• World’s Fastest File System
DDN is a Leader in Massively Scalable Platforms and
Solutions for Big Data and Cloud Applications
World-Renowned & Award-Winning
2

3. ddn.com©2012 DataDirect Networks. All Rights Reserved.
DDN | The “Big” In Big Data
800%
Paypal accelerates
stream processing
and fraud analytics
by 8x with DDN,
saves $100Ms.
1TB/s
The world’s fastest
file system, to power
the US’s fastest
supercomputer, is
powered by DDN.
Tier 1
Tier1 CDN accelerates
the world’s video traffic
using DDN technology
to exceed customer
SLAs.
If any company is well poised to take on the challenges of exascale
computing and big data, it's DDN, since this is its heritage.
451 Group
3

4. ddn.com©2012 DataDirect Networks. All Rights Reserved.
DDN Infrastructure Innovation Can
Bridge Applications To Exascale
The Exascale Objective The Benefits of Exascale… Today.
4
RAM NVRAM Hard Disk
Middleware
Application
‘Application Defined Infrastructure’
This is long-range ambition, w/ challenges:
• Applications Need To Be Re-coded
• IT Standards Need to Be Reinvented
RAM NVRAM Hard Disk
Middleware
Application
Application-Aware Data Infrastructure:
• Avoid Application Re-architecture
• Build Widely-Portable Application Appliances
• And, over time, integrate with applications
which will define HPC infrastructures…
I/O &
Data
Control
I/O &
Data
Control

5. ddn.com©2012 DataDirect Networks. All Rights Reserved.
Big Data & Cloud Infrastructure
Portfolio
hScaler™
50-2000 Nodes
200+ Data Sources
High Speed ETL
Up to 500% Faster Results
Enterprise
Hadoop Appliance
EXAScaler™
10Ks of Clients
1TB/s+, HSM
Linux/Windows Clients
NFS & CIFS
Petascale
Lustre® Storage
Enterprise
Scale-Out File Storage
GRIDScaler™
1Ks of Clients
1TB/s+, HSM
Linux/Windows Clients
NFS & CIFS
SFA12K
40GB/s/1.7M IOPS
1,680 Drives:
16U-2 Racks
Embedded Computing
SFA7700
10GB/s, 600K IOPS
60 Drives in 4U;
396 Drives in 20U
Storage Fusion Architecture™ Core Storage Platforms
SATA SSD
Flexible Drive Configuration
SAS
SFX Automated Flash Caching
• Read
• Write
• Context Commit
• Instant Commit
WOS® 2.5
256 Billion Objects
GeoReplicated
8 Million Objects/Second
Self-Healing
Mobile Cloud Access
Cloud Foundation
Big Data Platform
Management
DirectMon
Cloud
Tiering
5

6. ddn.com©2012 DataDirect Networks. All Rights Reserved.
Democratizing HPC
6
Mainstream HPC &
Enterprise Big Data
Hyperscale /
Supercomputing
Mid-Range
Arrays
SFA7700
Modular
Parallel FS
Appliances
SFA7700E
Hybrid Flash
Appliances
SFX Context
Commit
DDN WOS
Cloud
Appliances
Hyperscale
Appliances
SFA12K
Embedded
Processing
SFA12K-20E
Context-
Aware Cache
SFX API
Exascale
Object
Storage

7. ddn.com©2012 DataDirect Networks. All Rights Reserved.
SFA12K™ | Models
™
Maximum Drives
System Interface
Drive Types
System Capacity
Bandwidth
Cache IOPS
Flash IOPS
In-Storage
Processing™
1,6801 1,6801 1,6801
FDR IB
16Gb FC2
FDR IB
16Gb FC2
FDR IB
10/40GbE
3.5” & 2.5” SSD, SAS & SATA (inter-mixable)
6.72PB (w/ 4TB HDDs)1
20GB/s
(raw I/O)
40GB/s
(raw I/O)
20GB/s
(file I/O)
850K 1.7M850K
700K 1.4M700K
N/A N/A
Yes.
ExaScaler, GridScaler
Customer Provided
1 840 Drives Until Q1’13
2 16Gb FC available Q1’13
SFA12K-20 SFA12K-20E SFA12K-40
7

8. ddn.com©2012 DataDirect Networks. All Rights Reserved.
PCIe x8 Bandwidth
2004 2006 2008 2010 20142012 2016
GB/S1.63.26.4
PCIe G1
PCIe G2
PCIe G3
IVB
SNB
Westmere
Nehalem
Intel Tick Tock
Strategy allows DDN
to provide the fastest
storage products in
the Industry
12K-40 – 40GB/S
8

9. ddn.com©2012 DataDirect Networks. All Rights Reserved.
SFA12K-40 World’s Fastest Storage
SFA12K-40 WRITE(GB/s) READ(GB/s)
Raw device 34.8 44.2
EXAScaler (obdfilter) 28.5 33.4
EXAScaler % of Raw 78% 84%
9

10. ddn.com©2012 DataDirect Networks. All Rights Reserved.
SFA12K-40 EXAScaler Performance
10

11. ddn.com©2012 DataDirect Networks. All Rights Reserved.
Today’s Media Challenges
11
Database Logs
Structured Data
FS Metadata
Columnar Tables
File System Data
Flat Files
MB Access
Transactions Transfers
KB Access
Multi-Dimensional Performance
Requirements
Efficient Mixed Media Utilization
Great performance
but expensive
Massive capacity
but flat performance
Flash Disk Drives
SSD SATA
Delivering Flash performance at Disk cost
SFX

12. ddn.com©2012 DataDirect Networks. All Rights Reserved.
SFA
OS
SFA
OS
SFA
OS
Ram Flash Hard Disk
SFX Read SFX Write
Ram Flash Hard Disk
SFX Instant Commit
Ram Flash Hard Disk
SFX Context Commit
Application
Control
Accelerate Reads by
caching frequently
accessed data in Flash
Accelerate Writes by
instantly acknowledging
new data that is written.
Intelligentily pre-
popullate critical
data in Flash to
improve application
performance
SFA
OS
Ram Flash Hard Disk
Leverage Flash for new
Writes to Accelerates
subsequent Reads
Flash Acceleration:
SFX - Storage Fusion eXcelerator
SFX
12

13. ddn.com©2012 DataDirect Networks. All Rights Reserved.
SFA7700 Hybrid Big Data Appliance
13
300% ROI
Improvement
Modular, Highest-Performing
Mid-Range Array
• 4U Fully-Integrated Appliance
o Expandable to 3 Additional Storage
Enclosures For Cost-Effective Capacity
Tiering
o Tiered HDD & Flash Support
o Simple, Enterprise Management GUI
• High Speed IB & FC Connectivity
o 4 FDR IB OR 4 FC-16 ports
o OR 8 FC-8 ports (possibly 16 ports)
• Balanced Performance for Big Data
Applications
o 10+ GB/s of Sustained Bandwidth
o 350K Sustained SSD IOPS

14. ddn.com©2012 DataDirect Networks. All Rights Reserved.
SFA7700 Example Configuration
14
ExaScaler & GridScaler
Active/Active Parallel File Storage Servers
SFA7700
Integrated, Hybrid Storage Appliance
SS7000
4U 60-Drive Storage Enclosures (1-3)
Appliance
Capacity
540 TB in 14U
(using 3TB 7.2K)
Appliance
Performance
8.5GB/s
With File System

15. ddn.com©2012 DataDirect Networks. All Rights Reserved.
DDN & Intel Hardware Partnership
► IA compatibility
• Fast Platform Porting due to generation compatibility and ubiquity
► Technology leadership
• First to Market with the latest technologies - PCIeG3, DDR3
► Storage Feature Integration
• Transparent Bridging for mirroring data across dual controllers
• Crystal Beach DMA Engines to automate RAID, CRC’s and accelerate data
transfer
► Xeon Data Integrity features
• Advanced memory ECC
• Error checking across all major components and communication pathways
► Support infrastructure
• Support staff and Advanced Diagnostic Tools like VTune
► Regular cadence on new products
• Tick Tock strategy assures newer and higher performance storage products
► Highest Performance
• New MicroArchitectures continually increase the bar
15

16. ddn.com©2012 DataDirect Networks. All Rights Reserved.
DDN & Intel Software Partnership
A Rich History of Innovation
► 10 Years of Close Collaboration
Around Accelerating Computing
• First Engagement in 2003
• Many 100 Petaflops deployed since
► Partnered to provide the best
support for a new class of enterprise
Lustre File System adopters
► Collaborating on Intel® - Enterprise
Edition for Lustre* software
16

17. ddn.com©2012 DataDirect Networks. All Rights Reserved.
DataDirect Networks, Information in Motion, Silicon Storage Appliance, S2A, Storage Fusion Architecture, SFA, Storage Fusion Fabric, Web Object Scaler, WOS, EXAScaler, GRIDScaler,
xSTREAMScaler, NAS Scaler, ReAct, ObjectAssure, In-Storage Processing and SATAssure are all trademarks of DataDirect Networks. Any unauthorized use is prohibited.
Questions?
17