Monitoring in Federated Future Internet Testbeds - the FIBRE case - Authors: José Augusto Suruagy (UFPE) and Joberto S. B. Martins (UNIFACS) - 2nd perfSONAR WorkshopArlington – February 20, 2014.

1. José Augusto Suruagy (UFPE)
Joberto S. B. Martins (UNIFACS)
2nd perfSONAR Workshop
Arlington – February 20, 2014
Monitoring in Federated Future
Internet Testbeds: the FIBRE case

2. The FIBRE Project
The main objective of this project is to create a common space
between the EU and Brazil for Future Internet (FI)
experimental research into network infrastructure and
distributed applications, by building and operating a federated
EU-Brazil Future internet experimental facility
The project is designing, implementing and validating a shared Future Internet
research facility between Brazil and Europe, supporting joint Future Internet
experimentation of European and Brazilian researchers
2

3. FIBRE Partners
3

4. FIBRE Testbeds
4

5. Monitoring & Federation Context
• Monitoring Scope:
• Infrastructure:
• Objective: support NOC/administrators in FIBRE facility management and
experimenters in finding out alternatives in terms of available and usable
resources (links, machines, switches, …)
• Experiment (slice):
• Objective: support the experiment itself monitoring parameters like,
throughput, loss, … in a specific slice (experiment context)
• Monitoring Types:
• Active Measurements
• Passive Measurements
• Measurement tools per CMF:
• OCF: none native
• OMF: OML (-> GIMI)
• ProtoGENI: LAMP (pS) + INSTOOLS (-> GEMINI)
5

6. Monitoring & Federation Context
• Portal/Monitoring GUI and gateway(s) global view:
6
OCF OMF
Proto
GENI
Portal
/ Tool
GW
MDIP, TopHat GW, …
GW
FIBRE-BR I&M Portal,
TopHat, …
GW
CMFs (BR & EU;
communities)

7. Monitoring & Federation Context
(Related but Independent Solutions)

8. Monitoring Federation Use Cases
Measurement Configuration and Types
• Measurement Configuration:
• Intra-CMF
• Inter-CMF
• Measurement Use Cases
• Infrastructure measurements:
• UC 1: Active measurements
• UC 2: Passive measurements
• Experiment/Slice measurements
• UC 3: Intra-CMF
• UC 4: Inter-CMF
8

9. Monitoring Federation Use Cases
Measurement Configuration Alternatives
• “Intra” and “Inter”- CMF measurements
9

10. Monitoring Federation Use Cases
Infrastructure Measurements
• UC 1: Infrastructure active measurements
10

11. Monitoring Federation Use Cases
Infrastructure Measurements
UC 1: Infrastructure active measurements
• Proposal
– Install perfSONAR PS Toolkit nodes at each FIBRE island
– Provide a Dashboard at NOC with global view of the
testbed’s performance
– Develop a gateway to translate perfSONAR data to TopHat
standard (in progress)
11

12. Monitoring Federation Use Cases
Infrastructure Measurements
• perfSONAR Dashboard prototype implementation
12

13. Monitoring Federation Use Cases
Infrastructure Measurements
• UC 2: Infrastructure passive measurements
– Monitoring a variety of metrics (CPU, memory, NIC
utilization, etc.)
13

14. Monitoring Federation Use Cases
UC 3: Intra-CMF measurements
14
Portals
MI
Service
Data Plane
Control Plane
4
ii
2
MDA
iv
Portals
GUIs
A multi-CMF Slice
OCF
intra-CMF
measurement Experimenter
Gateway
A
P
I
1
3
i
iii
ProtoGENI
Gateway
A
P
I

15. • UC 3: Intra-CMF measurements
• Proposal:
– CMF’s native I&M software (e.g.: GEMINI on ProtoGENI) is
responsible for the measurement management and
configuration (the experimenter will use the CMF’s I&M
Portal)
– Gateways (such as MDIPs) are responsible for exposing the
measurement data collected by the CMF’s I&M software in
a standardized format
15
Monitoring Federation Use Cases
Experiment/Slice Measurements

16. 16
Monitoring Federation Use Cases
UC3: ProtoGENI MDIP (Implemented)

17. 17
Monitoring Federation Use Cases
UC3: ProtoGENI MDIP (Proposed)

18. Monitoring Federation Use Cases
UC3: OCF MDIP

19. 19
Monitoring Federation Use Cases
UC 4: Inter-CMF measurements
Portals
MI
Service
Data Plane
Control Plane
MDA
Portals
GUIs
A multi-CMF Slice
Experimenter
Orch.
Service
1
2
4
OCF
ProtoGENI
Gateway
A
P
I
Gateway
A
P
I
M. Tool
M. Tool
3 inter-CMF
measurement

20. • UC 4: Inter-CMF measurements
• Proposal:
– Each node, regardless of “its CMF”, comes with a common
measurement tool pre-installed. This way, measurements
become possible between any node pair
– A gateway abstracts the configuration and data exposal
complexities of the common tool through a standard API, so:
– The experimenter can use ANY Portal to configure
measurements and retrieve data
• Alternatives:
– perfSONAR BUOY
– OML
20
Monitoring Federation Use Cases
Experiment/Slice Measurements

21. 21
Monitoring Federation Use Cases
UC4: Instrumentation Phase
ProtoGENI slice
Orchestration
Service
Portals
Portals
Portals
RESTAPI
registerSlice()
addNode()
1

22. 22
Monitoring Federation Use Cases
UC4: Measurement Configuration Phase
ProtoGENI slice
Orchestration
Service
Portals
Portals
Portals
1
Agent
GW API
BUOY
BUOY
measure!
Agent
GW
API
RESTAPI
addTest()
3
3 fetch data

23. Goal:
• Turn OML into a perfSONAR-enabled solution
• PS-like query for experiment data:
• NMWG measurement data format
• All communication will be made through Web Services to
receive and retrieve data
23
Monitoring Federation Use Cases
UC4: OML MDIP

24. Monitoring Federation Use Cases
UC4: OML MDIP
24

25. • Testbeds Federation x Multi-Domain
– Specially for Infrastructure measurements (each testbed uses its own
tools)
• Need for a common data measurement report format and
protocol
– Webservices, NMWG (XML x JSON), NML, extended with other
resources (servers, VMs, etc.) measurement report formats
• Need for measurement tool gateway in case of similar but
heterogeneous tools
– E.g., Owamp x ETOMIC
• Extend SFA with a sort of MSpec (Measurement Specification)
– To retrieve, reserve (if necessary), schedule tests, and fetch data from
available infrastructure measurement resources
25
Monitoring Federation in FI Testbeds
Challenges/Issues

26. • Universal Measurement Information?
• Measurement software tools deployed as experimenter own
software:
– Hopefully without requiring that the experimenter be a measurement expert
– Avoids heterogeneity problems
• Specialized Hardware Measurement Tools:
– E.g., DAG cards, wireless band interference monitors, etc.
– Availability and heterogeneity issues
• Persistent data archiving (format and location issues)
• Experiment repeatability issues:
– Specialized resources may not be always available
26
Monitoring Federation in FI Testbeds
Challenges/Issues

27. • Constraint based resource orchestration:
– Orchestration agent for each CMF and one summary agent for the island
27
Monitoring Federation in FI Testbeds
Challenges/Issues

28. Thank you
José Augusto Suruagy/UFPE
suruagy@cin.ufpe.br
Joberto S. B. Martins/UNIFACS
joberto@unifacs.br
twitter.com/FIBRE_project
www.facebook.com/fibre.project
www.fibre-ict.eu
28