Backplane Technology Overview for AdvancedTCA

Backplane Technology Overview
B k l T h l O i
for AdvancedTCA

Hui Chen Theresienstr. 90
Supervisor: Rainer Ohlendorf 80290 Munich
Germany

July, 13th 2007 www.lis.ei.tum.de

© Institute for
Integrated Systems

Outline

Introduction to AdvancedTCA
Interconnect Protocols
I t tP t l
① Ethernet
② InfiniBand
③ StarFabric
④ PCI Express
⑤ RapidIO

Comparison of Interconnect Protocols
① Feature
② Protocol Efficiency
③ Effective Bandwidth
④ System Level Consideration

Conclusion
C l i

Hauptseminar am Lehrstuhl für Integrierte Systeme © Institute for
2
Hui Chen, July 13th 2007 Integrated Systems

What is AdvancedTCA ?
• AdvancedTCA (Advanced Telecom Computing Architecture)
Standard chassis/ backplane solution for high-availability
communications equipment
Adopted in 2003

• AdvancedTCA Chassis

3

Base Spec of AdvancedTCA
• Board dimension
1.2 inches in width
280 mm deep, ~322 mm high

• Power supply & Cooling
-48 V power feeds
200 Watts per board
Board of AdvancedTCA

• Backplane architecture
Zone 1: P
Z 1 Power, MManagement t
Zone 2: Data Transport Interface
Zone 3: User-defined I/O

Backplane of AdvancedTCA
4

Data Transport Interface
• Base Interface: Dual Star
Supports only Ethernet

• Fabric Interface: Dual Star/ Full Mesh
Supports various protocols
Max. 4 channels

(a) Dual Star configuration
• Dual Star
Redundant switch: eliminates single point of
failure
Effective for large redundant systems

• F ll M h
Full Mesh
+) Higher data throughput
-) Higher pins count & cost
Effective for
Eff ti f small systems
ll t
(b) Full Mesh configuration

5

Outline

① Ethernet
② InfiniBand
③ StarFabric
④ PCI Express
⑤ RapidIO

① Feature

Conclusion
6

Protocol ① ― Ethernet
• Application:
Initial in Local Area Network
Successful in backplane
but defective in congestion management

• Congestion management
PAUSE function: all data flows are paused when only
one flow is congested
→ congestion spreading, when complex data p
g p g, p paths

• Topology
P2P connections: Ethernet Switch ↔ Endpoints Ethernet Topology

• Frame

Header: large

7

Protocol ② ― InfiniBand ⑴
• Application Processor Node

Initial in System Area Network
now also i b k l
l in backplane, chassis-to-chassis
h i t h i
Supercomputer, Linux cluster

• Topology
InfiniBand Switches + Channel Adapters
HCA: Host Channel Adapter in processor node
TCA: Target Channel Adapter in I/O node
I/O Node
HCA talks TCAs to enable remote
InfiniBand Topology
storage and connectivity

• Packet format

Larger payload size than Ethernet

8

Protocol ② ― InfiniBand ⑵
• Three link widths x1

Links x1 x4 x12 x4
Data Rate (Gb/s) 2 8 24
x12

• Supports Quality of Service (QoS)
QoS: a method to prioritize network traffic
Ensures most important data gets thru the network as quickly as possible
Defines traffic to many streams, streams to differentiated classes;
reserves bandwidth to streams and classes

9

Protocol ② ― InfiniBand ⑶
• Supports Quality of Service
Thru 16 Virtual Lanes (VLs)
Multiplex independent data streams
→ the same physical link
Represent a set of send & receive
p
buffers in a port
Each port: one management VL (VL15)
& up to 15 data VLs
Priorities

VL15 (highest)
VL14
…

VL0 Virtual Lanes

10

Protocol ③ ― StarFabric ⑴
• Application
Backplane, chassis-to-chassis
Embedded distributed processing
(e.g. multiprocessor systems)

• Topology
StarFabric Switch: six-port
Connects other switches/ bridges
Cascadable
StarFabric Bridge StarFabric Topology

Translates other protocols (e.g. PCI) into
StarFabric traffic
St F b i t ffi

• Not scalable
Operates only at 2 Gb/s
O t l t Gb/

11

Protocol ③ ― StarFabric ⑵
• Packet format

7 classes of traffic
asynchronous traffic (data)
multicast
…
isochronous traffic (voice & video)

• Backwards compatible to PCI
Converting a serial fabric to parallel PCI

12

Protocol ④ ― PCI Express ⑴
• Successor to PCI
Backplane, chip-to-chip interconnect
PC (e.g. Graphics cards)

• Topology
Switches
Multiple Endpoints: I/O devices
Host Bridge
To create an I/O hierarchy
PCI Express Topology

• Six link widths:
Links x1 x2 x4 x8 x16 x32
Data Rate (Gb/s) 2 4 8 16 32 64

13

Protocol ④ ― PCI Express ⑵
• Packet format

Thru
Thr 8 Traffic Classes (TCs) & 8 Virt al Channels (VCs)
Virtual
Each port : up to 8 VCs. VC0 (lowest priority), VC7 (highest)
TCs: differentiated services
Each
E h packet a TC t determine which VC b ff t d
k t TC: to d t i hi h buffer to drop packet i t
k t into
TC/VC mapping: ☺ TC[0:6]→VC0 TC7→VC[0:1]

An Example of TC/VC mapping
14

Protocol ⑤ ― RapidIO ⑴
• Application
Backplane, chip-to-chip interconnect
Embedded systems (e.g. DSPs)
Networking (e.g. Carrier Cards)

•T
Topology
l
A Switch, multiple Endpoints
Large systems: one / more host processors
For system exploration, initialization
exploration
Monitors system-level activity
RapidIO Topology

• Two link widths (x1 x4)
(x1,
Links x1 x4

Data Rate (Gb/s) 1 2 2.5 4 8 10

15

Protocol ⑤ ― RapidIO ⑵
• Packet format

Header: smaller than Ethernet:
1-byte source address ~ 28 Endpoints → sufficient for backplane applications
Payload size: relatively small

Six logical flows: as prioritized classes

16

Outline
① Ethernet
② InfiniBand
③ StarFabric
④ PCI Express
⑤ RapidIO

① Feature

Conclusion
17

Comparison – Feature

Feature GigE InfiniBand StarFabric PCI Expr
Expr. RapidIO

Transport PCB, Copper & PCB, Copper & PCB & PCB, Copper &
Media Fiber Fiber Optical
PCB only
CAT5 cable
Scalable Link
None Yes None Yes Yes
Widths

Encapsulated, Yes, Yes, Yes,
PCI Migration None requires SW
q transparent
p transparent
p requires SW

Max Signal
1 Gbps 2 Gbps 2 Gbps 2 Gbps 2.5 Gbps
Channel Rate

y
Max Payload
1500 B tes
Bytes 4096 B t
Bytes 128 B t
Bytes 4096 B t
Bytes 256 B tes
Bytes
Size

18

Comparison – Protocol Efficiency
• Protocol Efficiency = Payload / Packet size
• Payload < ~500B : RapidIO, highest efficiency smallest overhead
Payload > ~500B : PCI Express, highest
Express

19

Comparison – Effective Bandwidth
• Effective Bandwidth = Protocol efficiency × Bandwidth
• InfiniBand x4; PCIe x4; RapidIO 4x 2Gbps
• O
Overall: Gi E l
ll GigE, lowest
t

20

Comparison – System Level
• Quality of Service (QoS)

GigE
Gi E InfiniBand
I fi iB d StarFabric
St F b i PCI E
Expr. RapidIO
R idIO

Prioritized Traffic
－ 16 7 8 6
Classes

•P f
Performance
GigE InfiniBand StarFabric PCI Expr. RapidIO

Max Bandwidth for
M B d idth f
1 Gbps 8 Gbps 2 Gbps 8 Gbps 10 Gbps
AdvancedTCA, x4

Latency High Medium Medium Low Low
(~100 us) (1.29-2.6 us) (< 3 us) (0.3-0.7 us) (< 100 ns)

21

Outline
① Ethernet
② InfiniBand
③ StarFabric
④ PCI Express
⑤ RapidIO

① Feature

Conclusion
22

Conclusion
• Low bandwidth backplane applications
Ethernet +) well understood, low risk
-) high overhead, hi h l
) hi h h d high latency
-) not scalable

• High bandwidth (tradeoffs of flexibility vs. latency, overhead)
g ( y y, )
InfiniBand and StarFabric
Complex header: +) for complex routing
-) reduces effective bandwidth
RapidIO
Smaller packets: +) to lower latency
-) overhead: a higher percentage of the packet
-) reduces effective b d idth
) d ff ti bandwidth
PCI Express
Middle: +) design goal: software transparency
-) constrains the protocol raises latency
protocol,

23

Thank you for your attention !

Any questions ?

Special thanks to:
Rainer Ohlendorf

24

Backplane Technology Overview for AdvancedTCA

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Was ist angesagt?

Was ist angesagt? (20)

Ähnlich wie Backplane Technology Overview for AdvancedTCA

Ähnlich wie Backplane Technology Overview for AdvancedTCA (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Backplane Technology Overview for AdvancedTCA