An overview of 100GbE technology, now and the future

1© 2018 MellanoxTechnologies | Confidential
July 2018 – Darren J. Harkins
100Gbps Ethernet
in the Data Centre

Ethernet – Existing in the Data Centre
• 10 Megabits per second (1 Lane – 10BASE-xx)
• 100 Megabits per second (1 Lane – 100BASE-xx)
• 1 Gigabit per second (1 Lane – 1000BASE-xx)
• 10 Gigabits per second (1 Lane – 10GBASE-xx)
• 40 Gigabits per second (4 Lane – 40GBASE-xx4)

Ethernet – Future in the Data Centre

Many 100GbE Layer 0/1 solutions
• WAN – Between Data Centres
• QAM-16 (DWDM) Single-mode 100km
• LR4 (direct) Single-mode 10km
• LAN – Within Data Centres
• SR10 Multi-mode – 100metres
• SR4 Multi-mode – 100metres
• PSM4 Single-mode – 500metres-2km
• LR4 Single-mode – 10km

Glass

Many 100GbE Layer 0 solutions – 10x10Gbps
CFP
SR10
CFP
SR10
100GBASE-SR10 - MPO
CFP – 13.6 mm × 82 mm × 144.8 mm
850nm - OM3 Multi-mode 20-Core
100m

100GBASE-SR4 - MPO
QSFP28 – 13.5 x 18.4 x 72.4 mm
850nm – OM4 Multi-mode8-Core
100m
QSFP28
SR4
PSM4
QSFP28
SR4
PSM4

100GBASE-LR4 - LC
QSFP28 – 13.5 x 18.4 x 72.4 mm
1295.56, 1300.05,1304.59,1309.14nm – Single-mode 2-Core
500m or 2km
QSFP28
LR4
QSFP28
LR4

100GBASE-PSM4 - MPO
QSFP28 – 13.5 x 18.4 x 72.4 mm
1310nm or 1550nm – Single-mode 8-Core
500m or 2km
QSFP28
SR4
PSM4
QSFP28
SR4
PSM4

100Gbe to 4x25GbE - Optics
100GBASE-SR4 to 4x25GBASE-SR
(or 100GBASE-PSM4 to 4x25GBASE-LR)
QSFP28
SR4
SFP28
SR
SFP28
SR
SFP28
SR
SFP28
SR

100Gbe to 4x25GbE - Optics
100GBASE-SR4 to 4x25GBASE-SR
MPO 8-Core to 8xLC 1-Core (Paired) Multi-Mode
(or 100GBASE-PSM4 to 4x25GBASE-LR
MPO 8-Core to 8xLC 1-Core (Paired) Single-Mode)
MPO
LC
LC
LC
LC
LC
LC
LC
LC
QSFP28
SR4
SFP28
SR
SFP28
SR
SFP28
SR
SFP28
SR

200GbE – 4x50Gbps
200GBASE-SR4 - MPO
QSFP56 – 13.5 x 18.4 x 72.4 mm
850nm – OM4 Multi-mode8-Core
30m+
QSFP56
SR4
QSFP56
SR4

25,000,000,000 pulses per second
0 1 0 0 1 0 1 1 0 0 1 0
0,0 0,1 0,0 1,1 1,0 1,0 1,1 0,1 1,1 0,0 1,0 0,0
1 bit
per
pulse
2 bits
per
pulse
NRZ
25Gbps
PAM4
50Gbps

25,000,000,000 pulses per second
4 bits
per
pulse
https://en.wikipedia.org/wiki/Quadrature_amplitude_modulation
QAM-16
100Gbps

Questions so far?

Significant Growth of Ethernet Market at 25Gb/s and Above
25G and above growing to 64%
of the market
CrehanResearch:LongRange Forecast – Server-classAdapters&LOM(Jul 2017)
17%
Of Total
Market
10G 83%13%
1%
1%
2%
High Speed
Adapters
2016 ($1.2B)
10G
25G
14%40G
2%
50G
15%
100G
27%
200G
6%
36%
High Speed Ethernet
Adapter Forecast
2021 TAM ($2.0B)
10GbE 25GbE 40GbE 50GbE 100GbE
64%17% →
200GbE
40G

Why is there Significant Growth at 25Gb/s and Above?
0.5Gbps per core
Core
Cache
Core
Cache
Core
Cache
Core
Cache
Cache
Cache
Core Core Core Core
Memory
Cache
Buffer
PCIe3bus
Core
Cache
Core
Cache
Core
Cache
Core
Cache
Cache
Cache
Core Core Core Core
Cache
Intersocketcommunication
Adaptor
Cache Cache Cache Cache
Cache
Cache Cache Cache Cache
Cache

25/100GbE in the Server and Switch
SW1 SW1
Pt1
Server
Pt2
SW1 SW1
Pt1
Server
Pt2 Pt1 Pt2
4x10GbE 2x25GbE

Transporting data between boxes (TCP/IP)
Application
Transport
Internet
Link
ProcessorMemory
Adaptor
Switch
Application
Transport
Internet
Link
ProcessorMemory
Adaptor
Switch

RDMA without touching the CPU – Offloading
Adaptor
Switch
Processor
Memory
Adaptor
Switch
Processor
Memory

Mellanox Single Root I/O Virtualization (SR-IOV)
 Running a Virtual NIC takes CPU resource
 SR-IOV – Placing part of the NIC (VF) inside a VM – using less CPU resource
 Migration via Para-Virtualisation – Built in to Hypervisors
VF Device Driver
VM
VF Device Driver
VM
VF Device Driver
VMn
Virtual NIC
VM
Physical Function Device Driver
PF VF VFVF

The need for OVS Offload – Moving switching away from the CPU
Running software OVS in the hypervisor costs CPU resource:
• High CPU utilization – CPU does switching tasks
• Limited Throughput – Core performance limited
• Higher Latency
Mellanox Solution
• SR-IOV - Allows direct connectivity to the NIC
- Bypasses the virtual switch and the frees the CPU
• Embedded Switch (eSwitch)
- Switching of flows in hardware frees the CPU

Server
VM1 VM2 VM3 VM4
Overlay Networks – Offloading State and Encapsulation from the CPU
Overlay Network Virtualization: Isolation, Simplicity, Scalability
Virtual Domain 3
Virtual Domain 2
Virtual Domain 1
Physical
View
Server
VM5 VM6 VM7 VM8
Mellanox SDN Switches & Routers
Virtual
View
L2 Overlay Networks
Virtual Overlay Networks Simplifies
Management and VM Migration
Overlay Accelerators Enable
Bare Metal Performance
OpenFlow
Virtual Network
Management API

Scaling with 25/100GbE– simple, scalable, re-usable
Half Rack
SW1 SW1 SW1 SW1 SW1 SW1 SW1 SW1
Spine 1 Spine 2
Full Rack Multi Rack

Questions so far?

Ethernet
ConnectX-4 Adapter Family
OPN
(ports)
MCX413A-B (1)
MCX414A-B (2)
MCX413A-G (1)
MCX414A-G (2)
MCX415A-B (1)
MCX416A-B (2)
MCX415A-G (1)
MCX416A-G (2)
MCX415A-C (1)
MCX416A-C (2)
Speed 40/56GbE 50GbE 40/56GbE 50GbE 100GbE
PCIe 3.0 PCIe x8 PCIe x8 PCIe x16 PCIe x16 PCIe x16

ConnectX-4 Lx PCIe Adapter
ConnectX-4 Lx Adapter Family
OPNs
(Ports)
MCX4111A-X(1)
MCX4121A-X (2)
MCX4111A-A(1)
MCX4121A-A(2)
MCX4131A-B(1) MCX4131A-G(1)
MCX4411A-A(1)
MCX4421A-A(2)
MCX4431A-B (1) MCX4431M-B(1) MCX4431A-G(1) MCX4431M-G(1)
Single/
Multi
Host
Single Host Single Host Single Host Single Host Single Host Single Host Multi Host Single Host Multi Host
Connect
or
Port
Speed
SFP28
10GbE
SFP28
25GbE
QSFP28
40GbE
QSFP28
50GbE
SFP28
25GbE
QSFP28
40GbE
QSFP28
40GbE
QSFP28
50GbE
QSFP28
50GbE
PCIe PCIe Gen3 x8
ConnectX-4 Lx OCP Adapter

Open Ethernet 10/40 & 25/50/100 Switch Portfolio
SN2410
10/25GbE ToR for servers and
storage
 48x 10/25GbE + 8x
40/100GbE
 Supports 1GbE ports
SN2700
40/100GbE aggregation for
servers and storage
 32x 40/100GbE
 64x 10/25/50GbE
SN2100
Ideal high-speed ToR for
HCI and storage
 ½ width ToR
 16x 40/100GbE
 32x 50GbE or 64x
10/25GbE
SN2010
Optimized 10/25G ToR for
HCI and storage
 ½ width ToR
 18x10/25GbE +
4x40/100GbE
NEW

200Gb/s Interconnect Solutions
Transceivers
Active Optical and Copper Cables
(10 / 25 / 40 / 50 / 56 / 100 / 200Gb/s) VCSELs and Copper
40 HDR (200Gb/s) InfiniBand Ports
80 HDR100 InfiniBand Ports
Throughput of 16Tb/s, <90ns Latency
200Gb/s Adapter, 0.6us latency
200 million messages per second
(10 / 25 / 40 / 50 / 56 / 100 / 200Gb/s)
16 400GbE, 32 200GbE, 128 25/50GbE Ports
(10 / 25 / 40 / 50 / 100 / 200 GbE)
Throughput of 6.4Tb/s
MPI, SHMEM/PGAS, UPC
For Commercial and Open Source Applications
Leverages Hardware Accelerations

Questions on Mellanox?

Darren J. Harkins
Staff Systems Engineer
darren@mellanox.com
+44 7944 786208
Thank You

An overview of 100GbE technology, now and the future

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie An overview of 100GbE technology, now and the future

Ähnlich wie An overview of 100GbE technology, now and the future (20)

Mehr von Jisc

Mehr von Jisc (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

An overview of 100GbE technology, now and the future