Talk from a meetup of the Swiss OpenStack User Group http://www.meetup.com/openstack-ch/events/219017178/

1. SCALE/SWITCHengines
IaaS for the Swiss Academic Community
CHOSUG meeting, Zurich, 5 March 2015
Simon Leinen
simon.leinen@switch.ch

2. © 2015 SWITCH
• SWITCH: National Research & Education Network (NREN)
–non-profit foundation to provide ICT to academic community, est.1987
–largely funded by service fees, “upward delegation” from universities
–10/100Gb/s IPv4/v6 backbone, SAML-based national AAI,
many other services (lecture recording, spam marking etc.)
• Cloud Project
–Goal: Build modern infrastructure for “community cloud” offerings:
• SaaS: “SWITCHdrive”–ownCloud-based sync & share
• IaaS: “SWITCHengines”–OpenStack/Ceph-based VM/storage on demand
• …
–National co-funding from CUS P-2 program as project “SCALE”
Background
2

3. © 2015 SWITCH
• Two locations (University DCs Lausanne/Zurich), each with
–32 2RU dual-Xeon (E5-2650v2) + 128 GB RAM + 2*10GE + 2*SSD
–16 servers also have 12*4TB 3.5” disks  Ceph OSDs
–2*48-port 10GE (+6-port 40GE) switches + 1*48-port GigE
–Uplink: 2*10GE w/BGP-4 (IPv4+IPv6) directly to backbone
• Currently two racks used per location
–Each can grow up to ~20 racks
• Plus a staging setup with two (tiny) sites
–in one of the two production locations
The Infrastructure
3

4. © 2015 SWITCH
• Icehouse
–Installed using Foreman + Puppet (puppet-openstack)
–Keystone, Nova, Glance, Cinder, Neutron, Heat, Ceilometer(?)
–RBD (Ceph) for all block storage
• Using RBD integration patch
• Controller functions in “pet” service VMs on “prov” hosts
–prov hosts = two boxes per site in DRBD/LVM setup
–service VMs: controller, network, foreman, dhcp, …
• Dual-site setup
–IPv6: all routable / IPv4: mostly RFC 1918, API/ext nets routable
–Separate Ceph clusters
–Separate regions (no cells)
OpenStack Installation
4

5. © 2015 SWITCH
• Everybody who uses L3 overlay (GRE/…) for tenant
isolation seems to run into this
• Standard “fix” seems to be to decrease tenant MTU
–I don’t believe in this – 1500 bytes is too ingrained in the Internet now
• We increased the underlay MTU (to 1600 bytes)
• Everything starts to work nicely  highly recommended
The MTU Issue
5

6. © 2015 SWITCH
• Simpler Dashboard for simple uses
–Working with external supplier
• IPv6 support (should be easy with Juno!?)
• Scale-out network (leaf/spine) beyond two racks
• Push performance boundaries of OVS+VXLAN
–Keeping the many alternatives in mind – e.g. Snabb Switch
• “VPC” – VPN between VMs and customer campus
Future Plans (Infrastructure)
6

7. © 2015 SWITCH
• Elaborate suitable business/charging model(s)
–with both university IT services and researchers
• Proposed follow-up project “SCALE-UP”
–Develop higher-level services in collaboration with universities:
• Distributed computing/big data
• Container management
• Virtual Private Cloud
• “Academic” SaaS
• …
Future Plans for SCALE/SWITCHengines
7

8. © 2015 SWITCH
• We’re hiring!
–see https://swit.ch/cj15
Questions?
8