SlideShare verwendet Cookies, um die Funktionalität und Leistungsfähigkeit der Webseite zu verbessern und Ihnen relevante Werbung bereitzustellen. Wenn Sie diese Webseite weiter besuchen, erklären Sie sich mit der Verwendung von Cookies auf dieser Seite einverstanden. Lesen Sie bitte unsere Nutzervereinbarung und die Datenschutzrichtlinie.
SlideShare verwendet Cookies, um die Funktionalität und Leistungsfähigkeit der Webseite zu verbessern und Ihnen relevante Werbung bereitzustellen. Wenn Sie diese Webseite weiter besuchen, erklären Sie sich mit der Verwendung von Cookies auf dieser Seite einverstanden. Lesen Sie bitte unsere unsere Datenschutzrichtlinie und die Nutzervereinbarung.
Workshop on Network Management and Monitoring Summary
Workshop on Network Management and
Monitoring - Summary
Maria Isabel Gandia, CSUC/RedIRIS
GN4-3 WP6 T3 / CNaaS
10th SIG-NOC Meeting
Prague, 14 November 2019
The Workshop on Network Management and Monitoring
• NORDUnet, Copenhagen, 21-22 October (before the STF meeting)
• 51 (38 in person + 13 remote) participants
• 28 NRENs/countries
• Explore several topics:
• Organising network management for end-institutions
• Tools for end institution management
• Monitoring end institution networks
• Automating management functions
• Four sessions:
1. End institution management: an introduction (Intro+8 LT)
2. End institution network management outsourcing (4)
3. Technical solutions for monitoring of the outsourced networks (3)
4. Technical solutions for network management (2 + conclusions)
End institution management
• 10 years ago – “absolutely no way we are going to do this”
• CNaaS initiative from SUNET and UNINETT - NRENs started to plan/offer
the service to manage the university campus networks – Campus Network
management as a Service
• CNaaS – a subtask of GN4-3 WP6T3 (Monitoring and management)
• But also…
• SURFnet is a pioneer among NRENs in automated management
• ARNES, CARNET, AMRES, KIFU/Hungarnet are managing parts of
the school infrastructures and/or WiFi infrastructures in the end
• FUNET manages the CPEs at the institutions
• And we heard other NRENs are investigating whether they should go into
• Why did NRENs start to think about and do this?
Tech talent shortage
• 63% of senior execs indicated that a talent shortage was a key concern
for their organisation.
A retiring baby boomer generation, a deficiency in STEM graduates, and an
increase in millennials’ lack of interest in technical careers or a career path
New (cool) skills needed
So, what could be the solution?
• End institutions are losing tech people
• NRENs are here, we know them, they have a good
reputation, let’s ask them…(The same regulation, data
privacy rules, no issues as with cloud services)
• Pressure from other NREN stakeholders (government)
• Adding new services in a situation with a tech talent
• So the NRENs are pushed to do more, while suffering
from the same problems as end institutions
• Automation is one part of the solution
Session 1: Lightning Talks (I)
• UNINETT (Vidar Faltinsen), Norway
• Dedicated Department for services in the Campus network.
• CNaaS services development started this year: improved security and better quality for Campus
networks. Running one pilot with a university.
• Digitalisation strategy 2017-2021 from the Ministry of Education use common services.
• UNINETT buys the equipment for the customer
• Local hands and heads still needed (rack mounting, patching…). UNINETT NOC involved.
• CNaaS package: management and monitoring, but also DHCP, NAT, RADIUS, VPN
• Planning FW, DNS, IDS
• Monitoring/Automation with NAV (developed by UNINETT)
• SUNET (Dennis Wallberg), Sweden
• 2 full-time developers hired for CNaaS
• Initial production planned in early 2020, with one customer. Equipment already procured.
• Only greenfield installations, no brownfield
• Helpdesk, hands and feet at the university, SUNET NOC second level
• Building the NMS/automation architecture
• Planning Zero Touch Provisioning in the near future and monitoring with NAV
Session 1: Lightning Talks (II)
• FUNET (Asko Hakala), Finland
• Started in 2012 with CPE management - 17 customers, 33 routers
• 3-person team from January
• FUNET Kampus service started in 2019
• 2 big and 7 small deployments
• FUNET buys and leases the equipment to the customers
• Installation done by the customer (if not, it has a cost).
• Everything automated using Ansible, configuration stored in YAML files.
• Same alert and monitoring tools as for the Funet network.
• SURFNET (Peter Boers), Netherlands
• 53 FTE for network (7 full-time developers), 25% externalised.
• First Campus service was Surfwireless
• Strategy is on SURFNET. Day-to-day management is outsourced to Quanza.
• Everything automated, connecting blocks through standardised interfaces.
Session 1: Lightning Talks (III)
• ARNES (Matej Vadnjal), Slovenia
• Operations existing NOC team of 5 members. New project planning external contractors (2 people
reviewing the documentation). Software development dedicated team of 4 members (+1 student)
• Already managing the last mile circuit (650 routers, 1,300 switches).
• WiFi Project WLAN2020 to provide a centralised managed eduroam/WiFi service in the country for every
primary and secondary school. Offering RADIUS as a Service.
• ARNES runs the procurement for the equipment, that is owned by the institution.
• Expect to manage ~20,000 APs, 2,000 switches, 450 routers, 955 campus networks by 2020.
• ARNES network service orchestration stack, automation based on Ansible.
• Running brownfield networks is challenging.
• CARNET (Darko Parić/Bojan Schmidt), Croatia
• E-Schools project started in 2015
• 35,000 APs, 80,000 switches, 70,000 devices (laptops, tablets…), LAN, interactive equipment for
• Upgrade in the backbone needed.
• Everything that can be migrated to the cloud, they pull it out from schools.
• 1st level support at school, 2nd level at CARNET, 3rd level at CARNET/vendor.
• GDPR 360: system for user data management
• Working on automation, Looking for solutions for school LAN management
Session 1: Lightning Talks (IV)
• KIFU/HUNGARNET (Attila Gyürke), Hungary
• Responsible for all the schools networking. StudentNet programme.
• 7,000 monitored CPE devices.
• The plan is to insource outsourced services like the call centre.
• AMRES (Bojan Jakovljević), Serbia
• Three flavours:
• CPE management since 2013 (equipment bought and owned by AMRES): 250 CPE routers
• AMRES managed wireless infrastructure, since 2014 (donated equipment, owned by
AMRES): 6,000 AP installed, 6 controllers (through SP managed services).
• LAN infrastructure in schools (2019-2021): Ministry of telecommunication runs tender and
buys equipment. In 2020: 15,000 APs, 2021: 24,000 APs (1,500 institutions)
• AMRES services are free of charge for the institutions. Best effort.
• Fewer engineers. Grown from 290 institutions in 2016 to 1,930 now.
• They see the benefits of automation, but are too busy operating the network.
• Need to hire and outsource some operational tasks.
Session 2(I): CNaaS Service Definition/Checklist (MI Gandia, CSUC)
• What do you need to think about, beyond the technical stuff?
• A Service Definition template/checklist, including:
• Contacts/Roles for the provider and the customer
• Service Delivery Model (Service packages, service elements…)
• Service Policy (KPI, SLT, Responsibilities…)
• Duration, Changes and Termination
• Prices and Billing
• GDPR Privacy Note
Session 2(II): Software Architectures
• SURFNET network management architecture and orchestration (Peter Boers,
• It’s not just automation or CNaaS. Orchestration is the heart of SURFNET8. No CLI
• Doing orchestration for 2+ years, 100+ products, 2,000+ changes.
• Running 3500 background jobs every day to check the network.
• Defining processes and workflows correctly is the key.
• The orchestrator is a home grown application using python and postgres.
• 10 FTE directly involved.
• Outsourcing automation software architecture in SUNET (Johan Marcusson,
• Goals of CNaaS NMS: ZTP
• Design principles defined.
• Design decisions made: Nornir/NAPALM instead of Ansible, to make the process easier.
• Config replace instead of config merge makes the management easier too.
• All configurations made via git. First, dry run, then live.
• They have run tests in 1,000 mock devices (no customers in production).
Session 2(III): Software Architectures
• Outsourcing service Management architecture in FUNET (Asko Hakala,
• Everything is done using Ansible and Jinja2. Configuration stored in YAML files.
• They can configure the routers before sending them. Everything quite
• Fully automated.
• Important to test before running and have git up-to-date.
• Separated customer management server.
• The initial configuration is done via a 4G OOB.
• CNaaS dashboard with HTTP and DNS measurements with Linux
namespaces (Tsotne Gozalishvili, GRENA)
• Monitoring probe in the fixed network. Box that connects to the network.
• They visualize results from perfSONAR measurements with the ELK stack.
• Several dashboards, like for DNS test results, VLAN status, etc.
• WiFiMon: Overview & Summary of Y1 Activities (Nikos Kostopoulos,
• Monitoring probe in the WiFi network. Raspberry Pi devices.
• It monitors the performance from the perspective of end users.
• Correlating accounting data from RADIUS and performance data from users.
• WiFiMon service planned to be released in 2020.
Session 3: Monitoring of the Outsourced Networks (I)
Session 3: Monitoring of the Outsourced Networks (II)
• Monitoring and alert aggregation (Morten Brekkevold, UNINETT):
• Network Monitoring toolkit for campuses since 2006 NAV for CNaaS
• NAV is not multi-tenant one instance per customer.
• Need for SSO support.
• They built an aggregator. Developed by students.
• Requirements defined by UNINETT.
Session 4: Technical Solutions for Network Management (I)
● NMaaS as a platform for management service outsourcing (Lukasz Lopatowski, PSNC)
● Kubernetes/docker platform for providing per-tenant management apps.
● Suitable for small NRENs and other teams in the GÉANT project.
● Options: supported by GÉANT or NREN deploys its own instance.
● Portfolio: Oxidized, LibreNMS, NAV, Prometheus, Grafana. PerfSONAR soon.
Session 4: Technical Solutions for Network Management (II)
● RENATER's White Box CPE in Normandy Regional network (Xavier Jeanin,
● RARE: Router for Academia, Research and Education
● Features developed: IPv4, IPv6, MPLS, SR-MPLS, L3VPN, XConnect, VPLS, EVPN, 6VPE
● No SNMP, but streaming telemetry
● White boxes:
● Switch/router manufactured from commodity components that allows different Network Operating
Systems (NOS) to be run on the same piece of hardware (Dell VEP 4600 servers, FRRouting)
● Initially designed for data centre use.
● Use case in the Normandy Regional network, for school CPE routers.
● Features: BGP peering, IGP, VLAN, Logical interface, VRF lite, management (SSH, Syslog,
SNMPv2) and security (line-rate IPv4/IPv6 L3 ACLs, Broadcast storm protection)
● Ansible based automation
Some Conclusions (I)
● NRENs are pushed to offer CNaaS services, without increasing
the number of employees:
● The use of automation is key to allow these services to grow.
● Some NRENs are also outsourcing some functions to offer CNaaS
● Services can differ from NREN to NREN, there’s no single
approach: CNaaS, e-Schools, WiFi2020, management of CPEs…
● User groups define the functionalities of a service - a service can
differ per user group inside the same NREN.
Some Conclusions (II)
● What can the GÉANT project do?
● Sharing information is important: organise more meetings to share stories
and how-to guides.
● Having a set of recommendations to create Service Definition documents is
● Contributions from multiple people (including students' work) is
managed through fully integrated CI/CD (Continuous
Integration/Continuous Delivery), code audits, well defined and
regularly executed tests.
● Kubernetes/docker based multi-tenant app provisioning seems to be
the way forward (NMaaS).
● A very lightweight perfSONAR (on rPi) for monitoring boxes could be
useful, perhaps integrated with WiFiMon on the same device.