This deep dive will address the questions on how to install, deploy and operate OpenStack by providing informative slides which will help users get ahead start with this awesome project

1. Deep Dive To
OpenStack
Hands On Session
Haim Ateya
Group Manager, EMC

2. Session Goals
§ Provide the audience with an hands-on experience on how to operate
OpenStack Cloud Infrastructure based on RDO distribution
§ Learn how to operate the various OpenStack components, configuration
files, command line usage and log files
§ Review the different installation types and methods, how to begin and
deploy your First OpenStack Cloud
§ Provide you with tools and knowledge on how to troubleshoot problems
when they happen, and how to fix them

3. Before We Begin
§ Demonstration will be done over an RDO “Liberty” installation of OpenStack
§ A pre-installed version of OpenStack server based on PackStack will be used
§ OpenStack is purely based on Linux and python
§ To master OpenStack – you need to master Linux
§ To master OpenStack – you need to master Python
§ Troubleshooting is a by product of using OpenStack
§ Take notes, capture your screens, ask questions

4. § Stands for: Redhat Distribution OpenStack
§ RDO is a community of people using and deploying OpenStack on CentOS,
Fedora, and Red Hat Enterprise Linux
§ The Open Source version launched by Red hat
§ Provides stable, documented and secured releases of OpenStack with full
documentation and support tool
About RDO
RDO To RHOS
Is Like
Fedora To RHEL

5. Seminar Lab Deployment
§ Our deployment consist of the following components:
o Distribution: Centos 7.2 installed with latest RDO
o Nodes: 2 physical servers (16G each)
o Compute (hypervisor engine): KVM
o Cinder (block storage): LVM iSCSI
o Glance (disk images): Images are stored locally on the disk

6. Installing OpenStack – How ?
§ Several methods to Install OpenStack:
§ PackStack – basic installer based on python & puppet that works with
configuration files to establish a working cloud instance
§ TripleO – an advanced installer that is aimed at installing, upgrading and
operating OpenStack clouds using OpenStack's own cloud facilities as the
foundations
§ Manual – installation of sources (tar balls) on a machine manually

7. About PackStack
§ Based on configuration files
§ Script that automates the installation process using puppet, RPM and YUM
§ Server must be pre-configured with all required RPM sources
§ Divided into sections for each one of the components {key: value}
§ Control if and where to install Nova, Swift, Glance, Cinder, etc’.
§ Support AIO distributed Standalone installations
§ Control users, passwords and tenants

8. Installation Process–PackStack
[root@rdo-openstack ~]# yum install openstack-packstack
Resolving Dependencies
--> Running transaction check
---> Package openstack-packstack.noarch 1:7.0.0-0.10.dev1682.g42b3426.el7 will be
installed
--> Finished Dependency Resolution
Dependencies Resolved
Package Arch Version
Repository Size
Installing:
openstack-packstack noarch 1:7.0.0-0.10.dev1682.g42b3426.el7
openstack-liberty 234 k
Installed:
openstack-packstack.noarch 1:7.0.0-0.10.dev1682.g42b3426.el7
Complete!
§ First – install the basic package openstack-packstack

9. Installation Process–PackStack
§ Second – Generate a basic configuration file from template
§ Then – edit the generated conf file using your favorite text editor
[root@rdo-openstack ~]# vim /tmp/rdo-openstack.conf
[root@rdo-openstack ~]# packstack --gen-answer-file=/tmp/rdo-openstack.conf
[root@rdo-openstack ~]#
[root@rdo-openstack ~]# ls -l /tmp/rdo-openstack.conf
-rw-------. 1 root root 46355 Mar 5 14:34 /tmp/rdo-openstack.conf

10. Configuration File–PackStack
§ Set desired services to install

11. Configuration File–PackStack
§ Configure keystone User & Password for each all services

12. Start The Installation–PackStack
§ Finally, after all changes are made to the configuration file, start the
installation (hint: better use screen)
[root@lg860 ~]# packstack --answer-file=/tmp/rdo-openstack.conf
Welcome to the Packstack setup utility
The installation log file is available at: /var/tmp/packstack/20160305-151722-
_Fn1J9/openstack-setup.log
Installing:
Clean Up [ DONE ]
Discovering ip protocol version [ DONE ]
Setting up ssh keys [ DONE ]
^CAdditional information:
* Time synchronization installation was skipped. Please note that unsynchronized time on
server instances might be problem for some OpenStack components.

13. PackStack
What Could Go Wrong?
§ Everything.
§ In case of failure, inspect the following:
o Read log file and understand the error message
o Check network connectivity to all external servers
o Check security enhanced Linux (SELinux) configuration
o Make sure YUM repositories are configured correctly
o Make sure server has enough free disk space

14. OpenStack Storage Concepts
§ Ephemeral storage:
o Persists until VM is terminated
o Accessible from within VM as local file system
o Used to run operating system and/or scratch space
o Managed by Nova
§ Block storage:
o Persists until specifically deleted by user
o Accessible from within VM as a block device (e.g. /dev/vdc)
o Used to add additional persistent storage to VM and/or run operating system
o Managed by Cinder
§ Object storage:
o Persists until specifically deleted by user
o Accessible from anywhere
o Used to add store files, including VM images
o Managed by Swift

15. Block Storage–Cinder
“ Cinder virtualizes pools of block storage devices and
provides end users with a self service API to
request and consume those resources without
requiring any knowledge of where their storage
is actually deployed or on what type of device.

16. Block Storage–Cinder
§ Responsible for block device provisioning of virtual machines
§ Provides an API for various storage array vendors to manage their block
device and translate commands between Nova and other services
§ Best used for performance-sensitive scenarios, such as
database storage or expandable file systems

17. Features-Cinder
§ Volumes:
o Persistent R/W Block Storage devices
o Can be attached to VMs as secondary storage
o Can be root store to boot VMs
o Can be attached only to one instance at a time
o Keep their state independent of instances
§ Snapshots:
o Read-only point in time copy of a volume
o Can then be used to create a new instance
§ Backups:
o An archived copy of a volume

18. Architecture–Cinder

19. Data & Control–Cinder

20. Default Configuration–Cinder
§ Default storage backend is LVM (Logical Volume Management)
o Each virtual machine disk is a logical volume (LV)
o Each virtual machine snapshot (volume) is a logical volume (LV)
§ Default SCSI transport mechanism is iSCSI using target-cli
[root@rdo-openstack ~(keystone_admin)]# lvs cinder-volumes 2>/dev/null
LV VG Attr LSize Pool Origin
Data% Meta% Move Log Cpy%Sync Convert
_snapshot-985c2a51-1fd7-4380-8005-3141b1ff3c0c cinder-volumes swi-a-s--- 25.00g volume-
a6fd39ae-d4d3-4eb3-a561-2b887c3d9097 0.00
_snapshot-f5fa8bda-ba84-4cf8-8387-9db25680cf92 cinder-volumes swi-a-s--- 25.00g volume-
a6fd39ae-d4d3-4eb3-a561-2b887c3d9097 0.00
volume-a6fd39ae-d4d3-4eb3-a561-2b887c3d9097 cinder-volumes owi-a-s--- 25.00g

21. Log Files–Cinder
§ All cinder log files are stored under /var/log/cinder/
o /var/log/api.log
o /var/log/backup.log
o /var/log/scheduler.log
o /var/log/volume.log
§ Start with scheduler.log and volume.log

22. Commands–Cinder
create Create a volume
delete Delete a volume
rename Rename a volume
manage Manage an existing volume
list List all volumes
show Show volume details
snapshot-create Creates a snapshot
snapshot-delete Removes one or more snapshots
migrate Migrates volume to a new host

23. Image Repository–Glance
§ Provides image repository (store) services to virtual machines
in the OpenStack Cloud.
§ Provides discovering, registering, and retrieving virtual machine images.
§ Registered images can be stored in the Object Storage service or in other
locations, such as simple file systems or external Web servers.
§ Provides an easy to use API to manage the repository

24. Image Repository–Glance
“ The Glance project provides services for
discovering, registering, and retrieving virtual
machine images."

25. Features-Glance
§ Boot from image
§ Create snapshots from image
§ Create image from volume

26. Architecture–Glance

27. Commands–Glance
Image-list List images you can access
Image-create Create a new image
Image-delete Delete an image
Image-show Show a specific image
Image-update Update an existing image
Image-upload Upload data for a specific image
member-list Describe sharing permissions by image
location-add
Add a location (and related metadata) to
an image.
location-delete Delete a location from an image

28. Logs Files–Glance
§ All log files are stored under /var/log/glance/
o api.log
o registry.log

29. Object Storage–Swift
§ Provides an HTTP-accessible storage system for large amounts
of data
§ Objects are stored as binaries along with stored metadata
§ Usually stores videos, images, email messages, files, or VM images
§ Each object is stored with 3 replicas to ensure availability and redundancy of
data
§ Runs on commodity hardware

30. Architecture–Swift
§ Proxy Server
§ The Ring
§ Storage Policies
§ Object Server
§ Container Server
§ Account Server
§ Replication
§ Auditors

31. Architecture–Swift

32. Commands–Swift
delete delete a container or objects within a container
download Download objects from containers
list Lists the containers for the account or the objects for a
container
post Updates meta information for the account, container, or
object; creates containers if not present
stat Displays information for the account, container, or
object
upload Uploads files or directories to the given container
capabilities List cluster capabilities
tempurl Create a temporary URL

33. Compute-Nova
§ Responsible for instance (VM) life cycle management
§ Support multiple hypervisors (KVM, Xen, LXC, Hyper-V, ESX)
§ Exposes a rich API and a set of components that interact with other
OpenStack components such as Glance, Cinder and Swift.

34. Components-Nova
§ Restful API – nova-api
interface exposed on TCP
port 8774
§ AMQP – message queue
used for RPC
communication
§ Nova Scheduler – handles
hypervisor selection for
instance placement

35. Components-Nova
§ Nova compute – acts as the
compute agent, interacts
with all relevant hypervisor
API’s to launch image
guests
§ Nova Conductor – handles DB
access (no-db-compute)
§ Console auth and proxies –
handles guest console services
for novnc, spice, etc.

36. Flavors –Nova
§ Virtual hardware templates which defines sizes for RAM disk,
number of cores, and so on
§ Default install provides five flavors
§ Admin may want to customize depending of the workload and hardware

37. Commands–Nova
list List active servers
host-list List all hosts by service
hypervisor-list List hypervisors
network-list Print a list of available networks
image-create Create a new image by taking a snapshot of a running server.
interface-attach Attach a network interface to a server
interface-dettach Detach a network interface to a server
migrate Migrate a server. The new host will be selected by the scheduler
stop Start the server(s)
start Start the server(s)
get-vnc-console Get a vnc console to a server

38. The Virtualization API–Libvirt
§ A toolkit to interact with the virtualization capabilities of recent
versions of Linux
§ Libvirt supports the following hypervisors:
o The KVM/QEMU Linux hypervisor
o The Xen hypervisor on Linux and Solaris hosts.
o The LXC Linux container system
o The VirtualBox hypervisor
o The VMware ESX and GSX hypervisors
o The Microsoft Hyper-V hypervisor
o The IBM PowerVM hypervisor

39. The Virtualization API–Libvirt
§ Provides a unified API called virsh to manage virtual machines
across different types of hypervisors
• Common commands in the libvirt domain:
virsh list Provides a list of running virtual machines
virsh dumpxml Provides virtual machine configuration file
virsh capabilities Returns capabilities of hypervisor / driver
virsh migrate Performs migration between hypervisors
virsh destroy Destroy an existing virtual machine
virsh create Creates a new virtual machine

40. Network–Neutron
§ Provides “networking as a service”
§ Provides Rich Topologies
§ Technology Agnostic
§ Extensible
§ Advanced Services Support
o LBaaS, VPNaaS, FWaaS

41. Features-Neutron
§ Support for Overlapping Ips
o Tenant A: 192.168.0.0/24
o Tenant B: 192.168.0.0/24
§ Configuration
§ DHCP/Metadata
§ Floating Ips
§ Vlan GRE VXLAN Tunneling

42. Concepts-Neutron
§ Ports
§ Networks
§ Subnet
§ VIF (Virtual Interface)
§ Floating IP (NAT)
§ External network
§ Provider Network
§ Tenenat network

43. Architecture–Neutron

44. Topology–Neutron

45. Log Files–Neutron
§ All neutron log files are stored under /var/log/neutron/
o dhcp-agent.log
o l3-agent.log
o openvswitch-agent.log
o ovs-cleanup.log
o server.log

46. Commands–Neutron
net-list List networks that belong to a given tenant
port-list List ports that belong to a given tenant
router-list List routers that belong to a given tenant.
subnet-list List subnets that belong to a given tenant
router-create Create a router for a given tenant
subnet-create Create a subnet for a given tenant
firewall-create Create a firewall for a given tenant
net-create Create a network for a given tenant
net-gateway-create Create a network gateway.

47. Authentication–Keystone
“ Keystone is an OpenStack project that
provides Identity, Token, Catalog and Policy
services for use specifically by projects in the
OpenStack family ”

48. Authentication–Keystone
§ Provides authentication and authorization for all openstack components
§ Interacts with every service in the OpenStack domain and provide
access validation for each service using token validation

49. Internals–Keystone
§ User: has account credentials, is associated with one or more tenants
§ Tenant: unit of ownership in openstack, contains one or more users
§ Role: a first-class piece of metadata associated with many user-tenant pairs.
§ Token: identifying credential associated with a user or user and tenant
§ Extras: bucket of key-value metadata associated with a user-tenant pair.
§ Rule: describes a set

50. Token & Access–Keystone
§ Initial access – User Password (get a token)
§ All access to OpenStack end-points are done using tokens
§ Token format:

51. Token & Access–Keystone
§ Access to keystone API is done via the following command:
§ Or using the following command:
[root@rdo-cloud ~(keystone_admin)]# cat ~/keystonerc_admin
unset OS_SERVICE_TOKEN
export OS_USERNAME=admin
export OS_PASSWORD=admin
export OS_AUTH_URL=http://10.82.55.64:5000/v2.0
export PS1='[u@h W(keystone_admin)]$ '
export OS_TENANT_NAME=admin
export OS_REGION_NAME=RegionOne
[root@rdo-cloud ~(keystone_admin)]# keystone --os-username=admin --os-password=admin --os-auth-
url=http://rdo-cloud:35357/v2.0 token-get
+-----------+---------------------------------------+
| Property | Value |
+-----------+---------------------------------------+
| expires | 2016-03-12T20:42:53Z |
| id | 2c4fe2b486234c7bb23a5b58734a3ca2 |
| tenant_id | 2817825172314573b4be9b5c36943b65 |
| user_id | a7fe5bc8b48b49fc973fe592ff1effdd |
+-----------+---------------------------------------+

52. Auth Flow–Keystone

53. Nova Auth Flow–Keystone

54. Log Files–Keystone
§ All log files are stored under /var/log/keystone
o keystone.log

55. Commands–Keystone
role-list List all roles
service-list List all services in Service Catalog
user-get Display user details
user-list List users
tenant-list List all tenants
endpoint-list List configured service endpoints
tenant-create Create new tenant
tenant-delete Delete tenant

56. Dashboard–Horizon
“ The OpenStack Dashboard (Horizon) provides a
baseline user interface for managing OpenStack
services."

57. Dashboard-Horizon
§ Is "Stateless"
§ Error handling is delegated to back-end
§ Doesn't support all API functions
§ Can use memcached or database to store
§ Sessions
§ Gets updated via nova-api polling

58. Dashboard-Horizon
§ Django based application that provides access to OpenStack services
§ Typically deployed as an Apache WSGI application
§ Leverage well known existing technologies
§ Bootstrap, jQuery, Underscore.js, Angular, D3.js

59. Internals-Horizon
§ 2 subprojects
o horizon - generic Django libraries and components to work with REST-
based back-end
o Openstack-dashboard - web app itself, with styles, locale, etc.
§ Dashboard for each entity (like instances or images) - nested Django app

60. Dashboard-Horizon

61. Logs-Horizon
§ Log files are stored under Apache folder /var/log/httpd/
o access_log
o error_log
o horizon_access.log
o horizon_error.log

62. Configuration Files-OpenStack
§ Configuration files are usually stored under /etc/
§ Example – Cinder configuration files are stored under /etc/cinder/
§ System service must be reloaded for configuration change to take affect
[root@rdo-cloud ~(keystone_admin)]# ll /etc/cinder
total 136
-rw-r----- 1 root cinder 2018 Mar 9 21:32 api-paste.ini
-rw-r----- 1 root cinder 121028 Mar 10 11:03 cinder.conf
-rw-r----- 1 root cinder 4623 Dec 22 14:59 policy.json
-rw-r----- 1 root cinder 973 Dec 22 14:59 rootwrap.conf
drwxr-xr-x 2 root root 29 Mar 9 21:32 rootwrap.d
drwxr-xr-x 2 cinder root 6 Dec 22 17:54 volumes

63. Service Management-OpenStack
§ Services in RHEL Fedora Centos are managed by systemctl
§ 2 dedicated tools:
o openstack-service – control OpenStack service states
o openstack-status – validate OpenStack service state
[root@rdo-cloud ~(keystone_admin)]# openstack-service status neutron
MainPID=10533 Id=neutron-dhcp-agent.service ActiveState=active
MainPID=10534 Id=neutron-l3-agent.service ActiveState=active
MainPID=10535 Id=neutron-metadata-agent.service ActiveState=active
MainPID=10536 Id=neutron-openvswitch-agent.service ActiveState=active
MainPID=10537 Id=neutron-server.service ActiveState=active

64. Troubleshooting OpenStack ?

65. Troubleshooting Guidelines
§ Start with the following questions:
o What triggered the failure ?
o Which of the components are involved ?
o Is the issue repeatable or its a one time occurrence ?
o Which logs should I start with ?
o Search the web forums chat rooms mailing lists

66. Useful Resources
§ RDO - https://www.rdoproject.org/
§ Miranits - https://www.mirantis.com/
§ OpenStack official site - http://www.openstack.org/
o Docs
o Training
o Learning
o Events
§ Ubuntu OpenStack - http://www.ubuntu.com/cloud/openstack

67. Q & A
Q & A

68. The End (part 4)