The document discusses designing for hyperscale computing. Hyperscale computing is defined as distributed infrastructure designed to handle increased demand for computing resources without additional physical space, cooling, or power. Key aspects of hyperscale include standardization, automation, redundancy, and high availability. When designing for hyperscale, the focus is on being cost-effective at large scale and allowing independent scaling of sections. The document recommends looking for partners with experience in open solutions, large AI systems management, and applying HPC discipline around areas like networking topology and latency, GPU acceleration, and infrastructure.

1. Pro Tips:
Building for HyperScale
William Wu
Director of Product Management, Penguin Computing
Penguin Computing Theater SC18

2. © 2018 Penguin Computing
How I Define Hyperscale Computing
2
▪ A distributed infrastructure
▪ Designed to handle increased demand for computing resources
▪ With no new additional physical space, cooling or electrical
power
▪ Often features standardization, automation, redundancy, and
high availability (HA)

3. © 2018 Penguin Computing
Why You Should Care
3
▪ Designing for hyperscale is NOT the same as designing for regular
data center or HPC
▪ Hyperscale is designed to be cost-effective at thousands of
instances/petabytes of data
 Deployment should be able to theoretically
scale indefinitely
 Should be able to scale sections
independently (not like hyperconvergence)
▪ BUT - same design concepts also help
improve efficiency of smaller sized
systems

4. © 2018 Penguin Computing
What to Look for in a Hyperscale Partner
4
▪ Technology focus in all key areas: storage, HPC, and
networking
▪ Experience with Open Solutions, such as Open Compute
Project (OCP)
 Standardization is required at hyperscale or is too costly to
build
▪ For AI, also look for vendors who have not just built but
manage large AI systems
 Running AI infrastructure teaches you a lot about how to
build it

5. © 2018 Penguin Computing
Application of HPC Discipline
5
▪ Example Solution Blocks
❑ Relion XO1132g – Intel® Xeon® Scalable Processors
❑ Altus XO1132g – AMD EPYC™
❑ Relion XO1114GT – Intel® Xeon® Scalable Processors, 4x GPU – PCIe Form Factor
❑ Relion XO1114GTS – Intel® Xeon® Scalable Processors, 4x GPU – SXM2 w/ NVLINK2
❑ Altus XO1114GT – AMD EPYC™, 4x GPU – PCIe Form Factor
▪ Penguin Computing HPC Discipline
❑ Skylake UPI Connections
❑ PLX Topology (Single, Dual Root Complex)
❑ Network Latency
❑ PCIe Gen3 vs PCIe Gen4

6. © 2018 Penguin Computing
Application of HPC Discipline
6
▪ Example Solution Blocks
❑ Training – NVIDIA® Tesla® V100-PCIe, GeForce Titan V
❑ Inference – NVIDIA® Tesla® V100-PCIe, Tesla P4
❑ Inference/Edge Computing – Jetson, Drive PX
❑ Solution Appliance - NVIDIA® DGX-1™, DGX-2, ODM HGX-2
▪ Penguin Computing HPC Discipline
❑ Application Accelerated Specification
■ Tensor Core, FP64/FP32, HBM2, NVLINK, NVSwitch, PCIe
■ PLX Topology, P2P
❑ Inference technology in scale out solutions
❑ Network Latency
❑ Network Topology (Scale out, expansion)

7. © 2018 Penguin Computing
Application of HPC Discipline
7
▪ Compute & GPU Accelerator
❑ Fan Speed Profile, Liquid Cooling (fan monitoring), air
flow simulation, power spikes, power shelf slew rate
❑ GPU monitor (or lack thereof w/ GTX)
❑ Liquid Cooling (fan monitor), power delivery (v1)
▪ Infrastructure
❑ 12V DC Pass Through (OPA Switch)
❑ Open Bridge Rack (v1 OCP)
❑ Angus Rack (v1 OCP and EIA)
▪ Network Topology
❑ Network Configuration
❑ Subscription Rate
❑ Expansion and Scale Out (Redundancy)
Core 1 Core 2 Core X
Edge 1 Edge 2 Edge 3 Edge X
GPU
GPU
GPU
GPU
GPU
GPU
GPU
GPU
GPU
GPU
GPU
GPU
GPU
GPU
GPU
GPU
Core 3 ….
….

8. © 2018 Penguin Computing
Solution Validation Tree Diagram
8
CPU
Server System
BMC
• Functional Test / BVT.FVT
• Stability – Stress . Cycling
• Performance
• FW Flash, Driver Update
• Failure Detection & Recovery
Memory, NVDIMM
HDD/SSD/M.2
• Feature / Functional Test
• Stability – Stress . Cycling
• Performance
• Power consumption
• Failure Detection
FW/SW
BIOS/uEFI
Driver/FirmwareApplications
MB
Mezz. / FPGA
Add-on card
HW
GPU Card

9. © 2018 Penguin Computing
Test Coverage
9
Pre-Run
• Check Rack
PDU/Switch
• Update Unit
Firmware
• Check Unit
Configuration
• Check Unit
Component Status
• Check Unit OS /
BMC Log
Run-in
• Unit Component
Stress Test
• Unit Idle Test
• Unit System
Stress Test
• Rack Stress Test
Final
• Check Firmware
• Check Unit
Configuration
• Check SMART
Info
• Check Unit OS /
BMC Log
• Reset Unit /
Switch as Default
Configuration

10. © 2018 Penguin Computing 10
To learn more, go to:
www.penguincomputing.com/ai-practice
Want More Info?
PENGUIN COMPUTING
@PenguinHPC
PENGUIN COMPUTING
PENGUIN COMPUTING

11. www.penguincomputing.com
1-888-PENGUIN