OpenFlow controller programmer does not have any method to confirm how reflected their code to the flow control, directly and intuitively “This code, how does work on... which flow?” “This flow, which code does make it?” This slides shows the basic design of the mechanism for binding code and flow to see them. It enables cross referencing logic and flow each other and also enable tracing the flow over switches. It had been presented at the 16th IOT conference of IPSJ, March 2012.

1. A proposal of the OpenFlow controller development support tool
- using Logic and Flow ID label -
Yutaka Yasuda, Kyoto Sangyo University

2. Introduction
• OpenFlow
Switch control mechanism by software program
• Problem of controller programming
Programmer does not have any method to confirm how
reflected their code to the flow control, directly and
intuitively
“This code, how does work on... which flow?”
“This flow, which code does make it?”

3. Goal of today : development support tool proposal
• bind code to flow control information
• enable to trace the affected flow
Controller
sw1 sw3
host1 host3
host2 host4
sw2 sw4

4. step 1:
Binding Code and Flow Entry

5. ex. : simple 2 ports switch and typical code for that
1 2
(inport=1) [output=2] convention of here :
(inport=2) [output=1] (match fields) [actions]
NOX like pseudo code:
def packet_in_callback(dpid, inport, reason, len, bufid, packet): callback
flow = extract_flow(packet)
make match fields
flow[core.IN_PORT] = inport
if inport == 1 :
outport = 2
else: make actions
outport = 1
actions = [[openflow.OFPAT_OUTPUT, [0, outport]]]
inst.install_datapath_flow(dpid, flow,..., actions, ...) set flow entry
return CONTINUE

6. Logic ID label setting
def packet_in_callback(dpid, inport, reason, len, bufid, packet
flow = extract_flow(packet)
flow[core.IN_PORT] = inport
if inport == 1 :
(inport=1) [output=2] outport = 2
else:
(inport=2) [output=1] outport = 1
actions = [[openflow.OFPAT_OUTPUT, [0, outport]]]
inst.install_datapath_flow(dpid, flow,..., actions, ...)
return CONTINUE
You can see the correspondence relation between code and flow entry

7. Logic ID label setting
def packet_in_callback(dpid, inport, reason, len, bufid, packe
flow = extract_flow(packet)
flow[core.IN_PORT] = inport
if inport == 1 :
inst.markLogicPoint( 101 )
#101 (inport=1) [output=2] outport = 2
#102 (inport=2) [output=1] else:
inst.markLogicPoint( 102 )
outport = 1
actions = [[openflow.OFPAT_OUTPUT, [0, outport]]]
inst.install_datapath_flow(dpid, flow,..., actions, ...)
return CONTINUE
Insert the function to set the Logic ID label to flow entry

8. Logic ID label setting
def packet_in_callback(dpid, inport, reason, len, bufid, packe
flow = extract_flow(packet)
flow[core.IN_PORT] = inport
if inport == 1 :
inst.markLogicPoint( 101 )
#101 (inport=1) [output=2] outport = 2
#102 (inport=2) [output=1] else:
inst.markLogicPoint( 102 )
outport = 1
actions = [[openflow.OFPAT_OUTPUT, [0, outport]]]
inst.install_datapath_flow(dpid, flow,..., actions, ...)
return CONTINUE
#102 ( inport=1 ) [ outport=2 ]
flow entry: cookie Match Fields Counters Actions
(64bits)
Cookie field is good to keep the label information

9. Result : cross reference of code and flow entries
switch controller program
1
flow entries 6
cookie match fields, etc..
2 7
(inport=1) [ outport=6 ]
3 8
101 ………………… def getOutport(dpid, vport, packet):
9 if not OPORTS.has_key(dpid):
4 ………………… log.err('Unknown dpid=%x on getoutput' % (dpid),system='paniersw')
return False
5 ………………… 10 if not OPORTS[dpid].has_key(vport):
log.err('Invalid (dpid,vport)=(%x,%d) on getOutport' % (dpid, vport),system='paniersw')
………………… return False
ops=OPORTS[dpid][vport]
for (rule, outport, cookie) in ops:
if rule == RULE80 and testTCPport(80, packet):
inst.setLogicMark( 101 )
return (outport, cookie)
elif rule == RULE443 and testTCPport(443, packet):
inst.setLogicMark( 102 )
return (outport, cookie )
Reference from flow to code can tell : elif rule == RULE25 and testTCPport(25, packet):
“This flow entry comes from this code” return (outport, cookie)
elif rule == ANY:
return (outport, cookie)
return False
Reference from code to flow can tell :
“Which flow entries are affected by this code”
def forward_l2_packet(dpid, inport, vport, packet, buf, bufid):
dstaddr = packet.dst.tostring()
if not ord(dstaddr[0]) & 1 and inst.st[dpid].has_key(dstaddr):
prt = inst.st[dpid][dstaddr]
if prt[0] == vport:
inst.setLogicMark( 122 )
log.err('**warning** learned port = inport', system="paniersw")
myFlood(dpid, bufid, buf, vport, packet)
else:
inst.setLogicMark( 123 )
log.msg('installing flow for ' + str(packet), system="paniersw")

10. Single branch : OK
switch controller program
1
flow entries 6
cookie match fields, etc..
2 7
(inport=1) [ outport=6 ]
3 8
101 ………………… def getOutport(dpid, vport, packet):
9 if not OPORTS.has_key(dpid):
4 ………………… log.err('Unknown dpid=%x on getoutput' % (dpid),system=
return False
5 ………………… 10 if not OPORTS[dpid].has_key(vport):
log.err('Invalid (dpid,vport)=(%x,%d) on getOutport' % (dpid
………………… return False
ops=OPORTS[dpid][vport]
for (rule, outport, cookie) in ops:
if rule == RULE80 and testTCPport(80, packet):
inst.setLogicMark( 101 )
return (outport, cookie)
elif rule == RULE443 and testTCPport(443, packet):
inst.setLogicMark( 102 )
return (outport, cookie )
elif rule == RULE25 and testTCPport(25, packet):
return (outport, cookie)
A recorded Logic ID directly means the branch path that elif rule == ANY:
the code actually passed through. return (outport, cookie)
return False
(Programmer puts the Logic ID record function to the
important and memorable points of the code. ) def forward_l2_packet(dpid, inport, vport, packet, buf, bufid):
dstaddr = packet.dst.tostring()
if not ord(dstaddr[0]) & 1 and inst.st[dpid].has_key(dstaddr):
prt = inst.st[dpid][dstaddr]
if prt[0] == vport:
inst.setLogicMark( 122 )
log.err('**warning** learned port = inport', system="panie
myFlood(dpid, bufid, buf, vport, packet)
else:
inst.setLogicMark( 123 )
log.msg('installing flow for ' + str(packet), system="panie

11. Multiple branches : needs the past record
def getOutport(dpid, vport, packet):
if not OPORTS.has_key(dpid):
log.err('Unknown dpid=%x on getoutput' % (dpid),s
return False
if not OPORTS[dpid].has_key(vport):
log.err('Invalid (dpid,vport)=(%x,%d) on getOutport'
return False
102 ops=OPORTS[dpid][vport]
flow entries
1 6 101 for (rule, outport, cookie) in ops:
if rule == RULE80 and testTCPport(80, packet):
cookie match fields, etc.. inst.setLogicMark( 101 )
2 7 return (outport, cookie)
(inport=1) [ outport=6 ] elif rule == RULE443 and testTCPport(443, packet):
3 8 inst.setLogicMark( 102 )
101 …………………
return (outport, cookie )
4 ………………… 9 elif rule == RULE25 and testTCPport(25, packet):
return (outport, cookie)
………………… 10 elif rule == ANY:
5 return (outport, cookie)
………………… return False
def forward_l2_packet(dpid, inport, vport, packet, buf, bu
dstaddr = packet.dst.tostring()
123 125 if not ord(dstaddr[0]) & 1 and inst.st[dpid].has_key(dsta
overwriting prt = inst.st[dpid][dstaddr]
if prt[0] == vport:
123 122 inst.setLogicMark( 122 )
log.err('**warning** learned port = inport', system
myFlood(dpid, bufid, buf, vport, packet)
If memorable points are (multiple) exists and located them else:
inst.setLogicMark( 123 )
the isolated branch, it needs to record the both of them. (in log.msg('installing flow for ' + str(packet), system=
right figure, 101 and 123 are.) flow = extract_flow(packet)
flow[core.IN_PORT] = inport
(outport, cookie) = getOutport(dpid, prt[0], packe
It is not good to overwrite by the recent single label (123). actions = [[openflow.OFPAT_OUTPUT, [0, outport
inst.install_datapath_flow(dpid, flow, CACHE_TIM
openflow.OFP_FLOW_PERMAN
bufid, openflow.OFP_DEFAULT_
inport, buf, cookie)
else:
inst.setLogicMark( 124 )
myFlood(dpid, bufid, buf, vport, packet)

12. Queuing for multiple label recording
overwriting queuing
1
flow entries 6 1 6
cookie match fields, etc.. cookie match fields, etc..
2 7 2 7
(inport=1) [ outport=6 ] (inport=1) [ outport=6 ]
3 8 3 8
101 ………………… 101,123 …………………
4 ………………… 9 4 ………………… 9
5 ………………… 10 5 102,124 ………………… 10
………………… …………………
123
overwriting 123
(8 x 8 bits)
... ... ... ... ... ... 101 123
38 queuing
To keep the past record, Logic ID label will be queuing to Cookie field (64bits). The right figure shows the
typical case, 8bits label queuing in 8 stages. The combination of bit width and the number of stages are
flexible to fit the target controller program.

13. Restrictions : binding code and flow entry
• Some controller SDKs may use Cookie for their purpose
Floodlight uses 32bits (12bit AppID, 20bit reserve)
• Increase switch loads to collect information
“In short, the existing statistics mechanisms are (both)
relatively high overhead” (J. Mogul et al., 2010)
J. Mogul et al. “DevoFlow: Cost-Effective Flow Management for High Performance
Enterprise Networks”, In Proc. ACM SIGCOMM HotNets-IX, 2010

14. step 2:
Tracing Flow Over the Switch

15. Flow tracing (idea)
• Demand : want to see what happen on the flow from here on
Now, flow entries and code are ready to refer each other by
this proposal
Next, want to observe the behavior of flows after then
• Problem : Flow entries have insufficient info to reconstruct
the original path
• Reason : Wildcards and Maskbits
There are many cases which cannot figure out the correct
path from flow entry info on switches

16. Wildcard
Flow merging by wildcards / maskbits
• Typical case to fail to trace the flow path
There are wildcards in different match fields and it makes fail to reconstruct the
original flow path correctly, to check up flow entries in two switches
Ex: The upper flow of sw1 (to 1.0.0.1) will be split into multi flows on sw2. (red lines)
And flow entries of sw2 includes the other flows. (blue lines)
Nobody knows how many packets will go upper flow of sw1 to upper flow (port 80) of sw2.
sw1 sw2
host1 1 4 1 4 to http proxy
(1.0.0.1, *, *) [#4] (*, 80, *) [#4]
2 5 2 5
3 (1.0.0.2, *, *) [#4] 6 3 (*, 25, *) [#5] 6 to smtp gateway
host2
convention : ( IP, src port, dst port ) [ outport ]
note:
If all flow entries specify the same match fields set, it is possible to check up completely.
Typical case of that is “all flows specify all match fields exactly”.

17. Flow tracing (how to)
• Set the label to packet and bring it by packet themselves to
the next hop
• Label setting and detecting should be done by switch
It means label should be on the field of flow entry
• Set the label by “Action” then send to the next switch
• Detect the labeled flow by “Match Fields” then treat it
separately from unlabeled flow

18. Wildcard
Flow separation under wildcards / maskbits
• Separation the flow by the label
The separated flow entry will be made in the next switch,
then it is fully traceable
Ex : The case a flow will be split on the next switch. (red line)
All “5” labeled packets will make only “5” matching flow entries on sw2
And “5” labeled flows on sw2 does not include any unlabeled flow.
sw1 sw2
host1 1 4 1 (*, 80, *, 5) [ =, #4] 4 to http proxy
(1.0.0.1, *, *, 0) [5, #4]
2 5 2 5
(*, 80, *, 0) [ =, #4]
3 (1.0.0.2, *, *, 0) [=, #4] 6 3 6 to smtp gateway
host2 (*, 25, *, 5) [ =, #5]
(*, 25, *, 0) [ =, #5]
convention : ( IP, src port, dst port, label ) [ label, outport ]

19. Flow tracing (the structure)
flow table
flow table
(to servers)
flow table
flow table
Controller
(to servers)
cookie Match Counter Action cookie Match Counter Action
101, 123 IP(dst)=X out=6 ????? ????? out=3
Packets have info for tracing and it will be a key to check up flow entries of switches

20. Ex : select outport by IP and the code for it
convention :
1 (1.0.0.1, *, * ) [ #4] 4
(IP, port-dst, port-src) [ #outport ]
2 5
3 6
def packet_in_callback(dpid, inport, reason, len, bufid, packet):
match = extract_flow(packet)
iph = packet.find('ipv4')
if ip_to_str(iph.dstip) == "1.0.0.1" :
outport = 4
else
outport = 5
actions = [[openflow.OFPAT_OUTPUT, [0, outport]]]
inst.install_datapath_flow(dpid, flow,..., actions, ...)
return CONTINUE
Here is a switch that selects output port depends on the destination IP address of the packet

21. Flow ID Label setting
def packet_in_callback(dpid, inport, reason, len, bufid, packet):
match = extract_flow(packet)
iph = packet.find('ipv4')
if ip_to_str(iph.dstip) == "1.0.0.1" :
outport = 4
inst.markFlowLabel( 5 )
(1.0.0.1, *, *, 0 ) [ 5, #4] else
outport = 5
actions = [[openflow.OFPAT_OUTPUT, [0, outport]]]
inst.install_datapath_flow(dpid, flow,..., actions, ...)
return CONTINUE
( label=0 ) [ set label=5 ]
flow entry: Match Fields Counters Actions
Insert the function to set the Flow ID Label and the label will be set by Action feature of the switch

22. The place to record the Flow ID label
• Use ToS field (6bits) for IPv4
OpenFlow supports ToS as Match Field and Action
Not regularly used than other fields
• It is possible to use VLAN id or MPLS label but....
Very commonly used for OpenFlow applicable field, such as
datacenter
unexpected VLAN or MPLS tag insertion may occur some
malfunctions to existing controller programs
• for IPv6, Flow Label (20bits) is good to use

23. Label overlapping : assigning to bit
• Overlapping : set different labels to single flow by multiple
switches
Cannot overwrite existent label info
• Label as bit pattern
label=n means ToS=2^(n-1)
independently writable for overlapping case
• Very few labels to able to set (ToS has only 6 bits!)
Use string label in the code, dynamically assign the number
when it turned on
Turn on/off by remote operation of controller program

24. Restriction : Tracing flow over the switch
• Diverted field does not usable for original purpose
( v4: ToS / v6: flowlabel)
• Need to remove the label on the edge
To prevent fail-reaction by the ToS value at the outside of
the target Open Flow network
There are some topology investigation researches
. OFDP : OpenFlow Discovery Protocol by GENI
. Topology module by Trema (LLDP base)
• Not applicable for not IP protocols

25. For not IP protocols?
• Ideally...”Should propose to add flow-id-field as OF std.”
It's a chicken and egg situation
New spec might not applicable on existent switch
• Realistically... “not a big issue?”
OpenFlow is focusing IP (v4) strongly
In L2 use case, processed by exact match of MACs
(label does not required, just checking them up is enough)

26. Full picture
20=2^(5-1)
cookie Match Counter Action +2^(3-1)
99, 121 tos=16 tos=20
if dpid == 25
markLogicPoint( 121 )
markFlowLabel( 3 ) dpid 25
else if dpid == 11
markLogicPoint( 101 ) Controller
markFlowLabel( 5 )
dpid 11
cookie Match Counter Action cookie Match Counter Action
16=2^(5-1)
88, 101 tos=0 tos=16 tos=16 ( keep tos )

27. OpenFlow standard !
System layout
red color means flow table flow table
“this part should
be implemented”
Controller
flow table flow table
label Controller
on/off App
operation tool
console
Controller SDK
flow table
aggregator
statistics data
visualizer

28. Conclusion
• Proposed the OpenFlow Controller development support tool
• Basic design of the mechanism for binding code and flow
It enables cross referencing the logic and flow each other
It enables tracing the flow over switches
• Required preparation
Need to add some functions to controller SDK
• To use this feature
Insert the function call to your code lines, but that’s all
No special requirement for switch, just OpenFlow capability

29. Conclusion
• Future work
• Finishing up the NOX implementation
• Confirming the applicability for other SDK by porting
• Visualization
• Proposing a new “flow id field” as OF switch standard
on/off Controller App flow table flow table
operation tool
console Controller SDK flow table flow table
statistics data
flow table
visualizer aggregator

30. Thank you.

31. if you have any interest of this idea,
porting to the other controller,
visualization,
making a commercial product,
or joint research,
feel free to contact me please.