Meetup #21 Anniversaire 6 ans OpenStack – Analyse réseau – Legacy apps

1. Sylvain Baubeau
Senior Software Engineer
05/07/2016
Skydive
A real-time network analyzer
Sylvain Afchain
Principal Software Engineer

3. WHY ?

4. SDN IS COMPLEX
Troubleshooting/monitoring is even more
complex

6. Implementations
Management
Control plane
● OpenFlow
● XMPP
● BGP
● AMQP
● etc...
Data plane
● VLAN
● VXLAN
● GRE
● MPLS
● OVS, Linuxbridge, other

7. Troubleshooting
Where...
packets are dropped ?
packets are fragmented ?
choke point occurs ?
What…
packet layers path ?
kind of traffic for this virtual network ?
number of flows on this link ?
number of TCP Sessions ?
bandwidth for this tenant ?

8. Current toolbox
● iproute2
● ovs-vsctl, ovs-ofctl, ovs-dpctl...
● ethtool
● brctl
● tcpdump
● SDN CLI/API
● SSH
● ...

9. What we need
● Flow centric
● Easy to deploy
● SDN Agnostic solution
● Non-intrusive / Lightweight
● Open, API
● Connectors to SDN

10. What we need
● Topology capture
a. interfaces, bond, mtu, vlan
b. bridges
c. Network namespaces
d. etc..
● Flow capture
a. on-demand traffic capture
b. on-demand counter capture
c. filtering
d. underlay/overlay informations
● Topology/flow aggregation
a. mapping topology/flow
b. analysis

11. Skydive Use-cases
● Detection of common configuration errors
● Detection of live network issues
○ bad performances, helping to find the root cause
○ DDOS and any unattended traffic
● Possibility to capture traffic at any point
○ History of all the captured metrics
○ Post mortem analysis thanks to the versioning
● Detection of bad application performance, bad RTT, wrong security groups
● Capacity planning

12. Topology capture
● Graph engine, event based
● Gremlin like query language
● Populated from :
○ netlink
○ netns
○ ovsdb
○ ethtool
● External connectors :
○ Docker
○ Neutron

13. Topology capture
$ ip netns add vm1
$ ip link add vm1-eth0 type veth peer
name eth0 netns vm1
$ ip link set vm1-eth0 up
$ ip netns exec vm1 ip link set eth0
up
$ ip netns exec vm1 ip address add
10.0.0.1/24 dev eth0
$ ovs-vsctl add-port br-int vm1-eth0

14. Topology capture
$ skydive client topology query -q 'G.V().Has("Name", "vm1")'
[{
"Host": "localhost.localdomain",
"ID": "07236227-b280-4947-5ceb-c1f98e8515f3",
"Metadata": {
"Name": "vm1",
"Type": "netns"
}
}]

15. Topology capture
$ skydive client topology query -q 'G.V().Has("Type", "ovsbridge").Out().Out().Has("Name",
Without("br-int"))
[ { "Host": "localhost.localdomain",
"ID": "a190409e-f76e-4c8f-55b9-985e662a37c0",
"Metadata": {
"Driver": "veth",
"IfIndex": 168,
"MAC": "3e:88:b9:65:04:7e",
"MTU": 1500,
"Name": "vm1-eth0",
"State": "UP",
"Type": "veth",
"UUID": "b6e9bf79-9b58-4b65-800e-1ddf9909d9dc" }}]

16. Topology capture
$ docker run --name=webserver
-p 80:80 -d eboraas/apache
$ docker run --name database
postgres

17. Topology capture
$ skydive client topology query -q 'G.V().Has("Type", "netns")'
[{ "Host": "localhost.localdomain",
"ID": "5674d492-e2e1-4e6f-63f4-3b9f1073da03",
"Metadata": {
"Docker.ContainerID": "5841d117701051542496d….994e5c2f2284e86c0ce17f2662",
"Docker.ContainerName": "/webserver",
"Docker.ContainerPID": 17216,
"Manager": "docker",
"Name": "webserver",
"Type": "netns"
}
}]

18. Flow capture
● Flow table centric
● Local mapping flow/topology
● Layer metrics
● Packet data from
○ sFlow
○ Pcap

19. Flow capture
$ skydive client capture create --gremlin
“G.V().Has(‘Name’,‘br-int’,‘Type’,‘ovsbridge’)"

20. Flow capture
$ ip netns exec vm1 ping 10.0.0.2
$ skydive client topology query --gremlin
“G.V().Has(‘Name’,‘br-int’).Flows()”
Flow schema
● Metrics per layer
● Unique ID per flow
● Unique ID per flow/capture
● Origin/Destination
● Capture point

21. Skydive architecture
Agents :
● Capture topology
● Capture flows, maintains flow table
● Local topology/flow mapping
● Forward topology/flow to analyzers
Analyzers :
● Aggregate topology/flow
● Global topology/flow mapping
● Stores topology/flow in a database
● Stores API objects in etcd

22. Openstack integration

23. Openstack integration
● Devstack plugin
● Keystone authentication backend
● Neutron OpenvSwitch and Linuxbridge based deployment
● Retrieves informations about networks and tenants
○ Tenant ID
○ VNI
○ Network ID
○ Port ID

24. Kubernetes integration

25. Kubernetes integration
● Deployed as a Kubernetes service
● Retrieves informations about Docker images

26. Demo

27. Skydive Roadmap
● Topology/Flow capture
○ OpenFlow, nDPI, eBPF
○ L3 informations and tunneling inspection
● Live distributed capture
○ Filtering
● Analysis
○ More protocols
○ Alerting
● Security
○ SSL
○ IP anonymization

28. How to start
Devstack
enable_plugin skydive https://github.com/skydive-project/skydive.git
enable_service skydive-agent skydive-analyzer
Docker compose
cd contrib/docker; docker-compose up
Kubernetes
cd contrib/kubernetes; kubectl create -f ./skydive.yaml

29. ● Open source (Apache License)
● Written in Go
● Contributions are welcome

30. Questions ?
https://github.com/skydive-project/skydive
IRC: #skydive-project @freenode.net
skydive-dev@redhat.com