TesKng
an
SDN
Idea
• Physical
switches/hosts?
• Testbed?
• Simulator?
• Emulator?
Mininet
Mininet
Your
SDN
Command-‐Line
Interface
ScripKng
API
VM,
naKve
laptop,
or
naKve
server
To
start
up
Mininet:
> sudo mn
Open-‐Source
System
w/AcKve
User
Community
6
• 1000+
users
• AcKve
mailing
list
• 700+
users,
200+
domains
• 20
Google
Summer
of
Code
apps
• GitHub
pull
requests
mininet.github.com
Verifying
Network
ProperKes
• “Does
my
SDN
work?”
– E.g.,
func&onal
correctness
– Same
control
program
+
OF
funcKonal
fidelity
• Does
not
try
to
solve
a
harder
problem:
– “How
does
my
SDN/network
perform?”
– That
is,
performance
proper&es.
– No
guarantee
or
expectaKon
of
perf.
fidelity.
Example:
ConnecKvity
in
a
Fat
Tree
servers
switches
10
20x
4-‐port
switches
16x
servers
sudo mn –-custom ft.py -–topo ft,4 –test pingall
Verifying
Network
ProperKes
• “Does
my
SDN
work?”
– E.g.,
func&onal
correctness
– Same
control
program
+
OF
funcKonal
fidelity
• “How
does
my
SDN
perform?”
– E.g.,
performance
proper&es
– No
guarantee
or
even
expectaKon
here
hosts
switches
12
A
B
Y
Z
full
throughput
Two
1
Gb/s
flows.
Disjoint
paths.
Example:
Performance
in
a
Fat
Tree
hosts
switches
13
A
B
Y
Z
X
collision
half
throughput
Two
1
Gb/s
flows.
Overlapping
paths.
Example:
Performance
in
a
Fat
Tree
Throughput
might
reduce.
But
by
how
much?
How
do
you
trust
the
results?
14
overlapping
events
real
Kme
…
…
…
A B
Link
Events
A: send request
B: init
xmit 1
xmit 2
B: send reponse
A B B HiFiA x1 x2B A B
Packet xmit
2
// A: Client
while(1) {
send_request(socket);
wait_for_reply(socket);
}
// B: Server
init();
while(1) {
wait_for_request(socket);
send_response(socket);
}
1
Real
Setup
B: wait
idle
B
B: send reponse
// B: Server
init();
while(1) {
wait_for_request(socket);
send_response(socket);
}
Real
S
Sources
of
Emulator
Infidelity
Event
Overlap
15
real
Kme
…
…
…
A B
Link
Events
A: send request
B: init
xmit 1
xmit 2
B: send reponse
A B B HiFiA x1 x2B A B
Packet xmit
2
// A: Client
while(1) {
send_request(socket);
wait_for_reply(socket);
}
// B: Server
init();
while(1) {
wait_for_request(socket);
send_response(socket);
}
1
Real
Setup
B: wait
idle
B
B: send reponse
// B: Server
init();
while(1) {
wait_for_request(socket);
send_response(socket);
}
Real
S
Sources
of
Emulator
Infidelity
So8ware
Forwarding
variable
delays
How
can
we
trust
emulator
results?
real
Kme
…
…
…
A B
Link
Events
A: send request
B: init
xmit 1
xmit 2
B: send reponse
A B B HiFiA x1 x2B A B
Packet xmit
2
// A: Client
while(1) {
send_request(socket);
wait_for_reply(socket);
}
// B: Server
init();
while(1) {
wait_for_request(socket);
send_response(socket);
}
1
Real
Setup
B: wait
idle
B
B: send reponse
// B: Server
init();
while(1) {
wait_for_request(socket);
send_response(socket);
}
Real
S
CPU
<=
50%,
so
not
overloaded,
right?
Wrong.
A
Workflow
for
High
Fidelity
EmulaKon
19
Create
experiment
Run
the
experiment
on
a
PC,
with
logging
Analyze
experiment
fidelity
using
“network
invariants”
Invariants
hold:
High
Fidelity
EmulaKon!
Instance(s)
of
behavior
that
differ
from
hardware
Run
again:
increase
resources
or
reduce
experiment
scale
1:
what
to
log?
2:
which
invariants?
3:
how
close?
20
Packet
Gap
Invariants
queue
link
switch
queue
packet
spacing
(when
queue
occupied)
Rmeasured
≤
Rconfigured
?
link
capacity
Example
Workflow
for
One
Invariant
21
Create
experiment
Run
the
experiment
on
a
PC,
with
logging
Analyze
experiment
fidelity
using
“network
invariants”
Invariants
hold:
High
Fidelity
EmulaKon!
Instance(s)
of
behavior
that
differ
from
hardware
Run
again:
increase
resources
or
reduce
experiment
scale
2:
Measure
packet
spacing
3:
Is
any
packet
delayed
by
more
than
one
packet
Kme?
1:
Log
Dequeue
Events
If
this
workflow
is
valid,
“pass”
same
result
as
hardware.
DCTCP
Data
Center
TCP
(DCTCP)
Kme
packets
in
queue
TCP
DCTCP
22
marking
threshold
Queue
occupied
100%
throughput
Queue
occupied
100%
throughput
Packet
spacing
we
should
see:
Emulator
Results
24
Does
checking
an
invariant
(packet
spacing)
idenKfy
wrong
results?
same
result
wrong;
limits
exceeded
80
Mb/s
100%
tput
6
pkts
var
same
result
160
Mb/s
100%
tput
6
pkts
var
320
Mb/s
Packet
Spacing
Invariant
w/DCTCP
25
1
pkt
med.
low
high
CCDF
Percent
(log)
25
pkts
Error:
(log)
10%
of
the
Kme,
error
exceeds
one
packet
x
Percentage deviation from expected
0
1
10
100
Percent
26
1
pkt
error
10
20
40
numbers
are
in
Mb/s
80
CCDF
Percent
Packet
Spacing
Invariant
w/DCTCP
Percentage deviation from expected
0
1
10
100
Percent
27
10
20
40
numbers
are
in
Mb/s
80
1
pkt
error
CCDF
Percent
Packet
Spacing
Invariant
w/DCTCP
Percentage deviation from expected
0
1
10
100
Percent
28
10
20
40
numbers
are
in
Mb/s
80
CCDF
Percent
Packet
Spacing
Invariant
w/DCTCP
160
Mb/s:
failed
emulaKon?
Beauty
of
networks
invariants
is
that
it
catches
and
quanKfies
the
error
in
this
run.
1
pkt
error
DemonstraKng
Fidelity
• Microbenchmarks
• ValidaKon
Tests
• Reproducing
Published
Research
– Do
complex
results
match
published
ones
that
used
custom
hardware
topologies?
• DCTCP
[Alizadeh,
SIGCOMM
2010]
• Router
Buffer
Sizing
[Appenzeller,
SIGCOMM
2004]
• Hedera
ECMP
[Al-‐Fares,
NSDI
2010]
Why
might
results
be
different?
• Student
error
/
out
of
Kme:
Incast
• Original
result
fragile:
RED
• Insufficient
emulator
capacity
to
match
hardware
of
original
experiment
– OpKon
1:
Scale
up
– OpKon
2:
Slow
down:
Time
DilaKon
– OpKon
3:
Scale
out:
Cluster
EdiKon
QuesKons?
• Check
out
Bob’s
Cluster
EdiKon
demo
Nikhil
Handigol
Brandon
Heller
Vimal
Jeyakumar
Bob
Lantz
[Team
Mininet]