This HowTo is about how to manage Public- and Hybrid Cloud deployments with openQRM. As the deployment manager for Amazon EC2 and its API compatible derivatives (e.g Eucalyptus) openQRM is capable to fully automate Instance provisioning and to add additional value by attaching automated application deployment via Puppet, automated monitoring via Nagios and also highavailability on Infrastructure-Level to the providers cloud features. The whole workflow of Instance-deployment in openQRM is exactly the same as for local resources in the internal IT-environment.
Navigating the Deluge_ Dubai Floods and the Resilience of Dubai International...
Automated Amazon EC2 Cloud deployments with openQRM
1. Automated Amazon EC2 Cloud deployments with openQRM
5.1 on Debian Wheezy
This HowTo is about how to manage Public- and Hybrid Cloud deployments with openQRM. As the deployment manager for Amazon EC2 and its API
compatible derivatives (e.g Eucalyptus) openQRM is capable to fully automate Instance provisioning and to add additional value by attaching automated
application deployment via Puppet, automated monitoring via Nagios and also highavailability on Infrastructure-Level to the providers cloud features.
The whole workflow of Instance-deployment in openQRM is exactly the same as for local resources in the internal IT-environment.
Requirements
One physical Server. Alternatively the installation can be also done within a Virtual Machine
at least 1 GB of Memory
at least 100 GB of Diskspace
optional VT (Virtualization Technology) enabled in the Systems BIOS so that the openQRM Server can run KVM Virtual Machines later
Install openQRM 5.1 on Debian Wheezy
Install a minimal Debian Wheezy on a physical Server
Install and initialize openQRM 5.1
A detailed Howto about the above initial starting point is available at "Install openQRM 5.1 on Debian Wheezy (resources/documentation-
howtos/howtos/install-openqrm-51-on-debian-wheezy.html)"
For this howto we have used the same openQRM server as for the howto about 'Virtualization with KVM and openQRM 5.1 on Debian Wheezy'.
That means with this howto we are going to add functionality to an existing openQRM setup. This is to show that openQRM manages all different
virtualization and deployment types seamlessly.
Actually this means you can use either use the "Install openQRM 5.1 on Debian Wheezy (resources/documentation-howtos/howtos/install-openqrm-
51-on-debian-wheezy.html)" or "Virtualization with KVM and openQRM 5.1 on Debian Wheezy (resources/documentation-howtos/howtos/virtualization-
with-kvm-and-openqrm-51-on-debian-wheezy.html)" howto as starting point.
Set a custom Domain name
As the first step after the openQRM installation and initialization it is recommended to configure a custom domain name for the openQRM
management network.
In this Use-Case the openQRM Server has the private Class C IP address 192.168.178.5/255.255.255.0 based on the previous "Howto install
openQRM 5.1 on Debian Wheezy (resources/documentation-howtos/howtos/install-openqrm-51-on-debian-wheezy.html)". Since the openQRM
management network is a private one any syntactically correct domain name can be used e.g. 'my123cloud.net'.
The default domain name pre-configured in the DNS plugin is "oqnet.org".
Best practice is to use the 'openqrm' commandline util to setup the domain name for the DNS plugin. Please login to the openQRM Server system
and run the following command as 'root' in a terminal:
/usr/share/openqrm/bin/openqrm boot-service configure -n dns -a default -k OPENQRM_SERVER_DOMAIN -v my123cloud.net
The output of the above command will look like
2. root@debian:~# /usr/share/openqrm/bin/openqrm boot-service configure -n dns -a default -k OPENQRM_SERVER_DOMAIN -v my123cloud.net
Setting up default Boot-Service Konfiguration of plugin dns
root@debian:~#
To (re)view the current configuration of the DNS plugin please run:
/usr/share/openqrm/bin/openqrm boot-service view -n dns -a default
(/fileadmin/Images/Documentation/Automated_Amazon_EC2_Cloud_deployments_with_openQRM_5.1_on_Debian_Wheezy/01-openqrm-ec2.png)
Enabling Plugins
For this HowTo please enable and start the following plugins in the sequence below:
dns plugin - type Networking
dhcpd plugin - type Networking
tftpd plugin - type Networking
device-manager plugin - type Management
nfs-storage - type Storage
lvm-storage - type Storage
nagios3 - type Monitoring
puppet - type Deployment
sshterm plugin - type Management
hybrid-cloud - type Deployment
Hint: You can use the filter in the plugin list to find plugins by their type easily!
Install the latest Amazon EC2 Tools
Go to Plugins -> Deployment -> Hybrid-Cloud -> About
3. (/fileadmin/Images/Documentation/Automated_Amazon_EC2_Cloud_deployments_with_openQRM_5.1_on_Debian_Wheezy/02-openqrm-ec2.png)
There you can find the URLs and informations about the latest Amazon EC2 API- and AMI-Tools.
Here the steps to install the Amazon EC2 Tools. Please SSH-login to the openQRM server as 'root' and run the following commands:
wget s3.amazonaws.com/ec2-downloads/ec2-api-tools.zip
wget s3.amazonaws.com/ec2-downloads/ec2-ami-tools.zip
unzip ec2-api-tools.zip
unzip ec2-ami-tools.zip
mkdir /usr/local/ec2
cp -r ec2-ami-tools-1.4.0.9/* /usr/local/ec2/
cp -r ec2-api-tools-1.6.8.1/* /usr/local/ec2/
apt-get update && apt-get install default-jdk
Please notice: The version numbers may be different when newer EC2 Tools gets available!
Then please add the following to the system wide profile /etc/profile
# EC2 Tools
export EC2_HOME=/usr/local/ec2
export PATH=$PATH:$EC2_HOME/bin
export JAVA_HOME=/usr
The EC2 API- and AMI Tools are now installed and available in the system path.
Please notice: Now please logout of the openQRM server and re-login. This is to activate the new profile settings in the environment. After re-login
please restart the openQRM server to also activate the profile in its environment by running:
/etc/init.d/openqrm restart
To re-check the configuration please run:
ec2-describe-regions -O [your-aws-access-key] -W [your-aws-secret-key]
The output of the above command looks like:
root@debian:~# ec2-describe-regions -O XXXXXXXXXXXXXXXXXXXXX -W YYYYYYYYYYYYYYYYYYYYYYYY
REGION eu-west-1 ec2.eu-west-1.amazonaws.com
REGION sa-east-1 ec2.sa-east-1.amazonaws.com
4. REGION us-east-1 ec2.us-east-1.amazonaws.com
REGION ap-northeast-1 ec2.ap-northeast-1.amazonaws.com
REGION us-west-2 ec2.us-west-2.amazonaws.com
REGION us-west-1 ec2.us-west-1.amazonaws.com
REGION ap-southeast-1 ec2.ap-southeast-1.amazonaws.com
REGION ap-southeast-2 ec2.ap-southeast-2.amazonaws.com
root@debian:~#
Configure which Amazon EC2 regions to use
Best practice is to use the 'openqrm' commandline util to setup which Amazon regions to use for the hybrid-cloud plugin. Please login to the
openQRM Server system and run the following command as 'root' in a terminal:
/usr/share/openqrm/bin/openqrm boot-service configure -n hybrid-cloud -a default -k OPENQRM_PLUGIN_HYBRID_CLOUD_REGIONS -v "eu-west-1,
us-west-1"
To (re)view the current configuration of the Hybrid-Cloud plugin please run:
/usr/share/openqrm/bin/openqrm boot-service view -n hybrid-cloud -a default
(/fileadmin/Images/Documentation/Automated_Amazon_EC2_Cloud_deployments_with_openQRM_5.1_on_Debian_Wheezy/03-openqrm-ec2.png)
Create a Hybrid-Cloud Account
Go to Plugins -> Deployment -> Hybrid-Cloud -> Actions and click on 'Add new Account'
Provide an account name and the AWS Access and Secret Key plus a description for the account. Then click on submit.
8. (/fileadmin/Images/Documentation/Automated_Amazon_EC2_Cloud_deployments_with_openQRM_5.1_on_Debian_Wheezy/09-openqrm-ec2.png)
(/fileadmin/Images/Documentation/Automated_Amazon_EC2_Cloud_deployments_with_openQRM_5.1_on_Debian_Wheezy/10-openqrm-ec2.png)
Create a custom auto-configuration script to the EC2 Instance on S3
The integration with Amazon EC2 in openQRM allows to attach a custom script to a starting Instance. The Instance is then running this script on
system startup. This can be used in combination with the Puppet integration to fully pre-configure an Instance in EC2. The easiest way to create such
a custom auto-configuration script is to use the S3 action in the account overview. This provides you with a File-Manager for S3 and allows to easily
upload files to S3. Those files, if set to 'public-read' permission is directly available via http. As an example we create a small bash-script which actually
just outputs some text to a file.
On your Desktop create a new file named 'my-custom-auto-configure.sh' with the following content:
#!/bin/bash
echo "Here custom commands are running on instance startup" > /tmp/my-custom-auto-configure.log
Now go to Plugins -> Deployment -> Hybrid-Cloud -> Actions -> S3 and create a new S3 bucket.
Click on 'Files in bucket' to list the files in the bucket.
26. (/fileadmin/Images/Documentation/Automated_Amazon_EC2_Cloud_deployments_with_openQRM_5.1_on_Debian_Wheezy/46-openqrm-ec2.png)
(/fileadmin/Images/Documentation/Automated_Amazon_EC2_Cloud_deployments_with_openQRM_5.1_on_Debian_Wheezy/47-openqrm-ec2.png)
Also please re-check /tmp/my-custom-auto-configure.log on the Instance to see you custom script got executed.
And here the Datacenter Dashboard after we have created the Amazon EC2 Instance
You can now fully automate your Amazon EC2 deployment with openQRM 5.1.
Hope you enjoyed this Howto!
Add more functionalities to your openQRM Setup
To continue and further enhance your openQRM KVM Virtualization Setup there are several things to do:
Enable the highavailability plugin to automatically gain HA for your server
Enable the cloud plugin for a complete Self-Service deployment of your Server and Software stack to end-users
Enable further Virtualization plugins and integrate remote Virtulization hosts for a fully distributed Cloud environment
Enable further Storage and Deployment plugins to automatically provision your Virtualization Hosts and other physical systems
... and more.
Links
27. openQRM Community: http://www.openqrm.com/ (http://www.openqrm.com/)
openQRM Project at sourceforge: http://sourceforge.net/projects/openqrm/ (http://sourceforge.net/projects/openqrm/)
openQRM Enterprise: http://www.openqrm-enterprise.com/ (http://www.openqrm-enterprise.com/)
openQRM at Twitter: https://twitter.com/openQRM (https://twitter.com/openQRM)
openQRM at Facebook: https://www.facebook.com/pages/openQRM-Enterprise/324904179687
(https://www.facebook.com/pages/openQRM-Enterprise/324904179687)
Amazon EC2: http://aws.amazon.com/ec2/http://linuxcoe.sourceforge.net/
(http://aws.amazon.com/ec2/http://linuxcoe.sourceforge.net/)