Automating Networks by using API

0
Copyright IDC Frontier Inc. All rights reserved.
Supporting the FutureYour Innovative
Partner
0
Automating Networks by using API/Webs
-story of a Network Engineer struggling with Programming-
IDC Frontier Inc.
UX Development Dept.
Issei Inoue
Feb 24, 2016

11
(C) IDC Frontier Inc. All Rights Reserved.
Agenda
・What Is Network Automation, Network API?
・Case Study (Entering Settings to a Router by Web/APIEntry)
・Software Design for Network API
・Tools Used (Sinatra, MySQL, ActiveRecord, NETCONF, JavaScript, jQuery, etc.)
・Required Skills and Issues

22
What is Automating Networks ?

33
client
HTTP
Rest API
Sinatra DB
Jobs Setting devices
Workers
Router
Worker
Netconf
Worker
Netconf
Worker
Netconf
MySQL
Ruby
CRM
Application
form
【Conventional Network Operation】【Automated Network Operation】
CLI
Router
client

44
What is Network API?
• APIs for network devices
They can be programatically configured by CLI/GUI, however,
APIs are used to make configuration more easily programmable
• APIs for network services run by operators, etc.
End users use APIs to set cloud networks such as IaaS
ex) AWS CreateVpc, AcceptVpcPeeringConnection , etc.
In the first place, what are "network APIs"?
The former is made by manufacturers,
and the latter is made by operators.

55
What is Network Automation
• SDN
• Creation of virtual networks on physical networks
• NFV
• Virtualisation of network functions on x86 servers
• White box
• Switch that separates hardware from software
• Configuration management tools such as Ansible/Chef are used for
management
• Automatic configuration by grouping together several
hundred switches (Zero Touch Provisioning)
• Automation of configuration tasks (AutoProvisioning)
In the first place, what is "automation"? The use of software?
This time, this is the
main theme
* Although SDNs or NFVs make AutoProvisioning easier
they are defined as something else here

66
Programming of Tasks
client
HTTP
Rest API
Sinatra DB
Jobs Setting devices
Job registration
Workers
Router
Worker
Netconf
Worker
Netconf
Worker
Netconf
Not executed
Job confirmation
Reflection of settings
Tasks done by people are put into
logical patterns and loaded into a
program.
MySQL
Ruby
Process ⇒⇒⇒⇒ API
Information ⇒⇒⇒⇒ DB
+----+-----------+-------------+------------+------+--------+--------+--------------------+----+
| id | vpn_num | domain |username | vlan | vrf | zone | segment |job |
+----+-----------+-------------+------------+------+--------+--------+--------------------+----+
| 72 | NPVN00619 | 70000000619 |issei06192 | 20 | 619 | Henry | 192.168.20.254/24 | 1 |
| 71 | NPVN00619 | 70000000619 |issei0619 | 19 | 619 | Pascal | 192.168.19.254/24 | 1 |
| 67 | NPVN00618 | 70000000618 |issei0618 | 618 | 18 | Tesla | 192.168.18.254/24 | 1 |
| 68 | NPVN00618 | 70000000618 |issei0618 | 617 | 18 | Tesla | 192.168.60.254/24 | 1 |
| 65 | NPVN00111 | 70000000111 |issei01 | 111 | 110 | Tesla | 192.168.111.254/24 | 1 |
| 58 | NPVN00615 | 70000000615 |issei0615 | 15 | 6 | Pascal | 192.168.15.254/24 | 1 |
| 59 | NPVN00615 | 70000000615 |issei0615-2 | 152 | 6 | Tesla | 192.168.152.254/24 | 1 |
| 64 | NPVN00615 | 70000000615 |issei0615-4 | 154 | 6 | Pascal | 192.168.154.254/24 | 1 |
| 57 | NPVN00608 | 70000000608 |Issei | 50 | 60 | Pascal | 192.168.1.0/24 | 1 |
| 63 | NPVN00615 | 70000000615 |issei0615-3 | 615 | 6 | Henry | 192.168.156.254/24 | 1 |
| 62 | NPVN01615 | 70000001615 |issei1615 | 16 | 15 | Tesla | 192.168.16.254/24 | 1 |
| 56 | NPVN00612 | 70000000612 |issei0612 | 612 | 612 | Tesla | 192.168.61.254/24 | 0 |
+----+-----------+-------------+------------+------+--------+--------+--------------------+----+
there are quite a few
provisioning tools like these

77
Illustration of Linking
APIAPI
PXC(Percona XtraDB Cluster)
netconfnetconf
(Standby)
IP-VPN
(Active)
3 replication
east-regionwest-region

88

99
Tools I used
・Sinatra
・Flask
・Django
Language
・Ruby
・Python
・Go
・PHP
・Bash
DB
・Percona
・ActiveRecord
・MongoDB
・Drizzle
・MariaDB
・Percona
・Redis
Web framework
Configuration
management
provisioning tools
・Ansible
・Puppet
・Chef
・Fabric
Job management
・RabbitMQ
・Celery
・rundec
・cron
・(Bash)
・(MySQL)
Web server
・Apache
・Nginx
・Webrick
・
・
・・
・
・
・
・
・Web UI
O/R mapping
・Rack
・WSGI
・PSGI
・Netconf
・Bash
・API
・JavaScript
・jQuery
・Ajax
*This is not to say that I recommend these tools.
I might even go so far to say that any tool would be good enough.

1010
API Design
Method API Path Argument Return Value
Addition of new records Post /privateconnect record Success/Fail
VPN settings Post /privateconnect Job id Success/Fail
Acquisition of information of VRFs to be set Get /routinginstance Job id xml-based VRF information
Acquisition of information of interface to be deleted Get /interface_stats Job id xml-based interface information
Deletion of records Delete /privateconnect_record Job id Success/Fail
Deletion of VPN settings Delete /privateconnect_setting Job id Success/Fail
Acquisition of 10 latest records in JSON format Get /privateconnect.json n/a VPN information in JSON format
APIs and Classes/methods for each action

1111
Tips For Rest APIs
API visibility is also important If possible. I'd like notation methods to be unified, too.
It would be very difficult though.
spinal-case snake-case camel-case dot
How to join - (hyphen) _ (underscore) Uppercase
characters
.(dot)
Examples /v1/idcf-inoue /v1/idcf_inoue /v1/idcfInoue /v1/idcf.inoue
Main Corporation
That Apply These
Google
LinkedIn
Facebook
Twitter
Instagram
slack
YouTube
CloudStack
OpenStack
Facebook
Others Apparently good in
terms of SEO
What's the latest
fashion?
This is often the
case with JavaScript,
jQuery and Ruby.
Not seen a lot
There are several notation methods to connect two or more words.

1212
UI Illustration
1. Extraction of information
2. Addition of settings
3. Deletion of settings
Information targeted in full text searches
can be easily extracted
mysql> select * from private_connects;
+----+-----------+-------------+----------------+-------------+------+--------+-----------+--------+--------------------+---------------------+---------------------+----------+
| id | vpn_num | domain | account | username | vlan | vrf | bandwidth | zone | segment | created_at | updated_at | job_done |
+----+-----------+-------------+----------------+-------------+------+--------+-----------+--------+--------------------+---------------------+---------------------+----------+
| 72 | NPVN00619 | 70000000619 | inoue06192 | issei06192 | 20 | 619 | 1000 | Henry | 192.168.20.254/24 | 2015-06-19 10:38:01 | 2015-06-19 10:39:15 | 1 |
| 71 | NPVN00619 | 70000000619 | inoue0619 | issei0619 | 19 | 619 | 1000 | Pascal | 192.168.19.254/24 | 2015-06-19 10:34:24 | 2015-06-19 10:36:51 | 1 |
| 67 | NPVN00618 | 70000000618 | inoue0618 | issei0618 | 618 | 18 | 1000 | Tesla | 192.168.18.254/24 | 2015-06-18 04:23:04 | 2015-06-18 06:10:22 | 1 |
| 59 | NPVN00615 | 70000000615 | inoue0615-2 | issei0615-2 | 152 | 6 | 1000 | Tesla | 192.168.152.254/24 | 2015-06-15 07:23:34 | 2015-06-15 07:29:34 | 1 |
| 64 | NPVN00615 | 70000000615 | inoue0615-4 | issei0615-4 | 154 | 6 | 1000 | Pascal | 192.168.154.254/24 | 2015-06-15 10:32:31 | 2015-06-15 10:33:01 | 1 |
| 57 | NPVN00608 | 70000000608 | Inoue | Issei | 50 | 60 | 1000 | Pascal | 192.168.1.0/24 | 2015-06-12 05:20:00 | 2015-06-15 06:53:06 | 1 |
| 63 | NPVN00615 | 70000000615 | inoue0615-3 | issei0615-3 | 615 | 6 | 1000 | Henry | 192.168.156.254/24 | 2015-06-15 10:01:53 | 2015-06-15 10:02:54 | 1 |
| 62 | NPVN01615 | 70000001615 | iinoue1615 | issei1615 | 16 | 15 | 1000 | Tesla | 192.168.16.254/24 | 2015-06-15 09:50:14 | 2015-06-15 09:50:49 | 1 |
| 73 | NPVN00619 | 70000000619 | %,$,#,&,?,!etc.| issei06193 | 193 | 619 | 1000 | Henry | 192.168.193.254/24 | 2015-06-19 12:32:39 | 2015-06-19 12:36:37 | 1 |
| 76 | NPVN01619 | 70000000619 | inoue06195 | issei06195 | 195 | 650000 | 1000 | Henry | 172.16.0.254/24 | 2015-06-19 12:45:35 | 2015-06-19 12:45:52 | 1 |
+----+-----------+-------------+----------------+-------------+------+--------+-----------+--------+--------------------+---------------------+---------------------+----------+
18 rows in set (0.00 sec)

1313
Existing line information is shown
to keep erroneous entry to a minimum
UI Illustration
Static route Next Hop
192.168.10.0/24 192.168.1.100

1414
Erroneous entry is kept to a minimum by displayed
the usage status of the interface targeted for deletion
UI Illustration

1515
Difficult and Important Points to Popularize Automation
• Integration and standardization of input interface
Bash Use of expect, general-purpose in a sense??
Netconf RPC-based, annotated in XML
SOAP API Outdated.. Suited to complex input/output?
Rest API The latest de facto standard URL resource support
• Substantiation of libraries, substantiation of sample code
Netconf is supported on the product side, which is meaningless even if
supported
Having a mechanism so that Netconf is used is important.
• Software ecosystem comprising network engineers
Even if manufacturers create plugins and libraries, they will not be
popularized if they are not passed onto users as well

1616
Main Libraries Located in Github, Etc.
Juniper
https://github.com/Juniper/net-netconf
https://github.com/Juniper/netconf-perl
https://github.com/Juniper/netconf-java
https://github.com/Juniper/netconf-php
https://github.com/leopoul/ncclient
Cisco
https://github.com/jtimberman/ruby-cisco
https://github.com/nickpegg/ciscolib
Brocade
https://github.com/brocade/ncclient
https://github.com/brocade/brocade （OpenStack Plugin）
https://github.com/BRCDcomm/BVC （VyattaController ）
https://github.com/zapman449/brocade_switchshow_aliases （Fiber switches）
Alaxala
https://github.com/sumikawa/netconf
Ideally, there should be standard APIs (APIs that allow all switches share a common configuration.),
however, this would be difficult in a number of ways. I would at least like official libraries from manufacturers
Cumulus
https://github.com/CumulusNetworks/cumulus-linux-ansible-modules
https://github.com/CumulusNetworks/cumulus-linux-chef-modules
https://github.com/CumulusNetworks/net-next
https://github.com/CumulusNetworks/quagga
https://github.com/CumulusNetworks/cumulus-cl-interfaces-puppet
https://github.com/OpenRTMFP/Cumulus （MonaServerusedSW）
https://github.com/cotdsa/cumulus
http://cumulusnetworks.com/blog/cumulus-linux-2/
It would be better to register project names
(account names) early....φ(・ω・｀ )

1717
Topics and Talk About
Hardships

1818
Setting Netconf on Rubyfor Junos
Netconf was difficult.
XML format as it is a bit troublesome for human eyes.
However, In Junos, ”show configuration | display xml” is really handy!
When using with Ruby, hyphens are taken as a regular expression,
so it is recommended to use the send method.
RPC is difficult. Sometimes the program cannot get unlocked...
RestAPI is the best . It is easy for beginners.
interfaces {
”interface” {
unit ”vlan” {
description NPVN00615:70000000615:inoue0615;
vlan-id 15;
family inet {
filter {
input 1G;
}
address 192.168.15.253/24 {
vrrp-group 163 {
virtual-address 192.168.15.254;
priority 150;
}
}
}
}
}
}
Junos ConfigXML format

1919
Checking Netconf information on Rubyfor Junos
Show related information is in XML so the output cannot fit on single screen.
show interfaces xe-2/0/0.111 detail results 3 scrolls

2020
BIG-IP Rest API (Ver11.5 or later)
$ curl -sk -H "Authorization: Basic xxxxxxxxxxxxxx" https://x.x.x.x/mgmt/tm/ltm/node -H 'Content-Type: application/json' -X
POST -d '{"address": "192.168.0.1","description": "testdescription","name": "testname"}'
{"kind":"tm:ltm:node:nodestate","name":"testname","fullPath":"testname","generation":36,"selfLink":"https://localhost/mgmt/t
m/ltm/node/testname?ver=11.5.1","address":"192.168.0.1","connectionLimit":0,"description":"testdescription","dynamicRatio":
1,"logging":"disabled","monitor":"default","rateLimit":"disabled","ratio":1,"session":"monitor-enabled","state":"checking"}[
Node addition
Addition of member to pool
$ curl -sk -H "Authorization: Basic xxxxxxxxxxxxxx" https://x.x.x.x/mgmt/tm/ltm/pool/~Common~pool_test/members -H
'Content-Type: application/json' -X POST -d '{"name": "testname:80"}'
{"kind":"tm:ltm:pool:members:membersstate","name":"testname:80","fullPath":"testname:80","generation":38,"selfLink":"https
://localhost/mgmt/tm/ltm/pool/~Common~pool_test/members/testname:80?ver=11.5.1"}
Setting Sync
$ curl -sk -H "Authorization: Basic xxxxxxxxxxxxxx" https://x.x.x.x/mgmt/tm/ltm/pool/~Common~pool_test/members -H
'Content-Type: application/json' -X POST -d '{"name": "testname:80"}'
{"kind":"tm:ltm:pool:members:membersstate","name":"testname:80","fullPath":"testname:80","generation":38,"selfLink":"https
://localhost/mgmt/tm/ltm/pool/~Common~pool_test/members/testname:80?ver=11.5.1"}

2121
libraries that were useful
Ruby NetAddr Package
Programming broadcast addresses or gateway addresses (end addresses) from scratch was
more difficult than I expected.
For example, it was surprisingly difficult to programmatically generate the gateway address
192.168.1.254 from the specified value from 192.168.1.0/24 using just the IPAddr Class.
I have solved the problem by using the NetAddr library.
https://rubygems.org/gems/netaddr/versions/1.5.0
http://www.rubydoc.info/gems/netaddr/1.5.0/NetAddr
netaddr1 = NetAddr::CIDR.create('192.168.1.0/24')
# broadcast address calculation
bc_address = netaddr1.last
# Convert to numerical form and subtract 1
gateway_int = NetAddr::CIDR.create(bc_address).to_i - 1
# That numerical value is returned to address notation
gateway = NetAddr.i_to_ip(gateway_int)
# /24 section only is extracted
netmask = netaddr1.netmask
p bc_address
# => "192.168.1.255"
p gateway
# => "192.168.1.254"
p gateway + netmask
# => 192.168.1.254/24
various handy methods are available, such as
output in ARPA format or range designation
and IPv6 compatibility

2222
Many other issues…
• What happens when a user enters in non-IP address format?
• What happens when logically invalid items are entered?
• In what format are errors returned to users?
• How is rollback performed when processing ends midway?
• Where and how are operation logs saved?
• What happens when a target device is in the middle of maintenance or malfunctions?
• How do you go about making a redundant system or DR? What about data
maintenance?
• How do you perform maintenance on the code when devices are replaced?
• Generation management, testing, CI (Continuous Integration)
• How do you transfer knowledge?
There is no end to software development...
How far can we anticipate errors?

2323
Summary
Well, it is not easy… But
How would you like jobs been automated?
Would you like networks to be manageable like
applications?
We can explore new services.
We can control network to make it more stable and secure
With Network Automation,

2424
Summary
New things come into existence by sharing various
people's intelligence together.
That is the Internet.
Networks cannot be created by one person,
nor one company.
What is important is connecting people and things.
A next generation Internet may be born
by the combination of networks and software!

2525
Finally, a little about me...
http://qiita.com/inoueissei
https://github.com/inoueissei
https://github.com/netops-coding
https://www.facebook.com/inoue.issei
https://twitter.com/inoueissei
IDC Frontier Inc.
Technology Development Division
UX Development Dept.
Issei Inoue
I'm not very good at disseminating information.
However, I'm going to do my best from here on.

2626
Thank you for your kind attention.

2727
Supporting the FutureYour Innovative Partner

Automating Networks by using API

Empfohlen

Empfohlen

Weitere ähnliche Inhalte

Ähnlich wie Automating Networks by using API

Ähnlich wie Automating Networks by using API (20)

Kürzlich hochgeladen

Kürzlich hochgeladen (20)

Automating Networks by using API