1. USP Team
FIBRE-BR Camp, 28-29 April 2012
Ouro Preto (MG), Brazil

2. USP Team members
• General coordinator
– Tereza C. M. B. Carvalho – carvalho@larc.usp.br
• Team members
– Marco A. Torrez Rojas – matrojas@larc.usp.br
– Marcos F. Schwarz – mschwarz@larc.usp.br
– Fernando F. Redígolo – fernando@larc.usp.br
– Cristina K. Dominicini – cdominic@larc.usp.br
– Charles C. Miers – cmiers@larc.usp.br
– Paulo H. Fertonani – pfertonani@larc.usp.br
– Raphael Penteado – rpenteado@larc.usp.br
2

3. Agenda
• Part I - Emulab
• Emulab LARC/USP
• How does Emulab Works ?
• Part II – ProtoGENI
• What is ProtoGENI
• ProtoGENI Control Monitoring Framework
• ProtoGENI Backbone
• ProtoGENI Federation
• Considerations
• Part III – Demo
3

4. Agenda
• Part I - Emulab
• Emulab LARC/USP
• How does Emulab Works ?
• Part II – ProtoGENI
• What is ProtoGENI
• ProtoGENI Control Monitoring Framework
• ProtoGENI Backbone
• ProtoGENI Federation
• Considerations
• Part III – Demo
4

5. Emulab
• Experimental Facility
– Developed for perform experimental network and distributed systems
research that provides: network emulators, simulators and live
networks.
• Design Principals
– Transparency
– Virtualization
– Automation
– Efficiency
• Provided Resources
– Local nodes (PCs)
• Edge nodes, wireless nodes, software-defined radio (GNU Radio)
– Virtual Network Links
• Arbitrary topologies, local-area links and wide-area links
5

6. Emulab
• Main Emulab Testbed
– University of Utah
• Around 500 PC nodes;
• Design to support research and education (students and
professors)
6

7. Emulab LARC/USP
• Emulab LARC:
– Sponsors: Fapesp / Kyatera & Ericsson
– 24 nodes for experiments
• 17 Intel Pentium 4 (3GHz), 3GB RAM (DDR2), 80GB HD
• 7 Xeon Quad-Core 2.4Ghz, 4GB RAM, 500 GB HD
• All with 3 x Gb Ethernet interfaces for experiments
– 3 servers
• Storage of user data / OS images / Clearinghouse
– 5 Gigabit switches (Cisco and Juniper)
7

8. Emulab LARC/USP
Internet pc1.emulab.larc.usp.br
200.144.187.130/26
200.144.187.61/30
Datacenter’s
Network
pc22.emulab.larc.usp.br x3
200.144.187.151/26
Rede LARC cs00 – Cisco 3750G-24TS – L3
192.168.189.x/24 200.144.187.62/30
200.144.186.225/29
200.144.187.241/29 pc23.emulab.larc.usp.br
200.144.187.129/2 200.144.187.152/26 x3
192.168.189.2/24
x5
pc24.emulab.larc.usp.br
cs01 – Juniper EX3200 24 – L2
200.144.187.153/26 x5
192.168.189.3/24
Control Network Switch Stack with
48 x Gigabit Ethernet ports
es00
es01
boss.emulab.larc.usp.br es02
200.144.186.226/29 (48+24+24 Stack) 96 port
192.168.189.5/24 Gigabit Ethernet
kvm.emulab.larc.usp.br console.emulab.larc.usp.br 192.168.189.4/24
192.168.189.7/24 192.168.189.6/24
VM
ops.emulab.larc.usp.br clrhouse.emulab.larc.usp.br
200.144.186.242/29 200.144.187.226/29
Caption : Author(s): Network Connections
Marcio / Fernando
Control hardware (vlan44) Control net (vlan45) Experimental net
External net (vlan42) Public net (vlan43) Private net (vlan46)
Revisor(s):
26/10/11 V.2.0 1 de 3 8

9. How does it work ?
1
1 User define the desired topology in a ns-2 file (directly or
through a Java apple) and submit it to Emulab control system
9

10. How does it work ?
2
2a Swap-in: control system allocates the physical nodes for the
experiment
• Virtual topology is mapped to the physical resources, according to
what is available in any given moment
• These nodes are loaded with the desired OS and some basic
config is done
• VLAN creation for the links
• Experiment customization
• Default images / Custom images
• Software instalation / Script execution
• Static routes definition / Dynamic routing enabling 10

11. Experiment Creation Process
Experiment
Virtual Real Hardware
Patch Pannel Programável (snmp)
Node Node
Node
Emulab Web A A
A
pc 7 pc 7
Ethernet Switches
VLAN 1
Traffic Traffic
Latency
Shapper Shapper
50 ms
50ms 50ms
Ns File pc 12 pc 12
VLAN 2
Node Node
Node
B B
B
pc15 pc15
11

12. How does it work ?
2
2b System sends an e-mail to the user notifying that the experiment
is ready for usage
12

13. Resource Allocation Process
13

14. How does it work ?
3
3 Resources remotelly accesible
Tipically through ssh and/or serial console
SSH nodes
addresses
14

15. How does it work ?
4
4 Swap-out: resources are freeded for other experiments
• Process of freeing allocated resources
• 2 main limits for an automatic swap-out
• Idle-Swap: number of hours that the resources haven´t been
used (e.g.: 2hs)
• Max-duration: number of hours that the resources have been
allocated (e.g.: 16hs)
• Usually defined at experiment creation time
• Manual swap-out: researcher frees resources
15

16. Agenda
• Part I - Emulab
• Emulab LARC/USP
• How does Emulab Works ?
• Part II – ProtoGENI
• What is ProtoGENI
• ProtoGENI Control Monitoring Framework
• ProtoGENI Backbone
• ProtoGENI Federation
• Considerations
• Part III – Demo
16

17. What is ProtoGENI
ProtoGENI was created to provide the integration of Emulab testbeds
in order to build a GENI facility.
The integration consists of three key components:
• CMF: Control and Monitoring Framework software from the
University of Utah, based on an enhanced version of the Emulab
testbed management software
• Backbone: A USA nationwide, high-speed dedicated backbone on
Internet2’s wave infrastructure
• Federation: A set of federated testbeds, including a variety of
wireless networks (CMULab), residential broadband (CMULab), and
programmable edge clusters (Emulab) that are connected to the
backbone
17

18. ProtoGENI CMF
• Enhanced version of the Emulab Control Framework:
• SFA Architecture: Aggregate Manager (AM), Clearing
House (CH), Slice Authority (SA), RSpec;
• XMLRPC Interface: to AM, CM, SA;
• I&M Tools: INSTOOLS and LAMP (based on perfSONAR);
• Flack GUI: a Flash-based interface that allows allows
visualization and manipulation of federated resources.
18

19. SFA Architecture
The Clearing House behaves as a central registry coordinating
the federation.
Each Aggregate Manager (Component Manager) is
responsible for the allocation, operation and control of one or
more resources.
Slice Authorities are responsible for coordination, accounting
and mediation of resources provided by many aggregate
managers.
The RSpec is a data interchange format describing resources
properties. It is used in four major contexts: advertisements,
requests, tickets, and manifests.
19

20. Flack
Flack is a Flash-based Google Map client that allows
visualization and manipulation of federated
resources.
20

21. ProtoGENI Backbone
• Composed of 8 nodes connected by
Backbone node Example
1Gbps waves on the Internet2
infrastructure;
• The backbone runs Ethernet on these
waves, and is sliced with VLANs;
• Each node contains a set of sliceable,
user-programmable components:
• PCs with NetFPGA cards
• Ethernet Switches (HP Procurve)
with OpenFlow-capability under
implementation.
21

22. ProtoGENI Federation
Integration of Tesbeds
22

23. GENI Federation
Other GENI
Facilities
(eg. PlanetLab)
23

24. Considerations
ProtoGENI CMF is installed over Emulab
infrastructure and is highly dependent on Emulab
source code.
ProtoGENI @ USP: Integrate Emulab resources
under FIBRE federation
Clearing House documentation and
code/dependecies are not properly updated.
24

25. Agenda
• Part I - Emulab
• Emulab LARC/USP
• How does Emulab Works ?
• Part II – ProtoGENI
• What is ProtoGENI
• ProtoGENI Control Monitoring Framework
• ProtoGENI Backbone
• ProtoGENI Federation
• Considerations
• Part III – Demo
25

26. Demo
Pre-requisites
An account setup on one of the Emulab aggregates.
The account can be requested at www.emulab.net, clicking
on the "Request Account" button.
Note that account creation require approval and may take a
while.
26

27. Log on to Flack
First logging in to Flack on http://protogeni.net/flack.
Flack uses the credentials from your issuing authority.
Select your authority and download the credentials.
27

28. Create a Slice
28

29. Experiment Definition
Nodes can be added to the slice by dragging from the aggregate where
you want to create the node.
29

30. Experiment Definition
Nodes can be added to the slice by dragging from the aggregate where
you want to create the node.
Common node types:
• PC (exclusive physical machine);
• VM (virtual machine shared with others).
30

31. Experiment
Definitiion
Links between nodes are created by dragging a line from
one node to another.
31

32. Experiment Definition
Optionally load a RSpec description of the experiment.
32

33. Slice Creation
When the topology is ready the slice can be created.
Nodes access is done by ssh (as in Emulab)
33

34. Questions
34