How to create your own private cloud?

BANGALORE, INDIA: Most discussions around Cloud Computing nowadays are either focused on the security issues associated with it, or providing basic definitions of the three different types of Cloud based services i.e SaaS, PaaS and IaaS. A third thing is of course the loud noise created by all the vendors around their own cloud based offerings.

There's hardly anybody talking about actual usage of cloud computing. Which public cloud service should you choose? What's a private cloud and how to get started on it? What are the tools that would let me create my own private cloud? There are very few discussions on these aspects.

So while the public cloud based services have been covered elsewhere in our cover story, in this story we'll focus on how to build your own private IaaS (Infrastructure as a Service) based private cloud. For this, we've used Ubuntu's 10.04 Server Edition, which is completely free. We've given it on this month's DVD, else you can download it from ubuntu.com/cloud/private.

The benefits of setting this up are many. It has the elasticity to grow or shrink your compute capacity based on your application's needs. You can rapidly deploy new applications whenever required on your existing IT infrastructure. This way, you don't end up investing in additional hardware resources. The Ubuntu Enterprise Cloud provides the same APIs as the popular EC2 cloud from Amazon, so if you've worked with that, this should be a piece of cake. What you create and run on Ubuntu, you can also run on EC2.

Let's Implement

You need at least two servers for deploying the Ubuntu Cloud. One (Machine A) would act as the cloud, cluster, warehouse, and storage controller; while the other (Machine B) would be the node controller. One thing to keep in mind is that the node controller should support virtualization as virtual machines would be running on it.

By default a single virtual machine would run per core, therefore having a multi-core node controller is highly recommended. We configured Machine A on a Core2duo X6800 processor based machine with 2 GB DDR 2 RAM and 80 GB HDD. Machine B was running on an AMD Phenome II X4 965 processor with 4 GB DDR 3 RAM and 250 GB HDD.

{#PageBreak#}

?

You'll need to burn the Ubuntu server ISO on a CD/DVD and boot machine A from it. From the first screen that pops up, choose 'Install Ubuntu Enterprise Cloud'. The installer would automatically check for Eucalyptus components on the network and recommend that you install the required components. From the screen with 'Select Cloud installation mode' heading, select Cloud, cluster, warehouse, and storage controller. Other Cloud specific inputs required during installation are cluster name (cluster1 in our case) and range of public IPs that can be assigned to instances. Once Machine A is up and running, you'll need to setup the Node Controller. For this, boot Machine B with the Ubuntu Enterprise Server CD/DVD and? select 'Install Ubuntu Enterprise Cloud'. This would automatically detect the cluster1 and pre-select 'node controller' in 'Select Cloud installation mode'. Once the installation is done, reboot machine B and we are done.

{#PageBreak#}

Configuring your Private Cloud

Before you can start using the nodes, they have to be registered. In our case, registration is done automatically as we are installing Ubuntu Enterprise Cloud using the ISO. To verify this registration, run following command on Machine A:

cat /var/log/eucalyptus/registration.log

Next you need to? get credentials of the users. This can be done either through a web browser or by using the command line. Just visit the IP address of your Cloud controller (in our case, 192.168.2.10), login using 'admin' as both user name and password. This action would prompt you to change your password. Once done with these changes, click on 'Credentials' tab on the top, and then click on 'Download Credentials' button on the top. Save this zipped file to '~/.euca', then unzip this file using:

unzip -d ~/.euca mycreds.zip

?

{#PageBreak#}

?

{#PageBreak#}

The other way of achieving credentials is by running the following commands on the cloud controller: 

mkdir -p ~/.euca

chmod 700 ~/.euca

cd ~/.euca

sudo euca_conf --get-credentials mycreds.zip

unzip mycreds.zip

ln -s ~/.euca/eucarc ~/.eucarc

cd ?

To use your credentials, install the required cloud user tools as follows:

sudo apt-get install euca2ools

And to check if everything is running properly

. ~/.euca/eucarc

euca-describe-availability-zones verbose

 Installing and running an image

We have created our infrastructure. Now we show you how to run applications on top of this. You can either create your own image or get the sample images from the image store on web interface. Simply access Machine A from a web browser by typing its IP address, and login as admin. From this interface, click on 'Store' and then click on 'Search'. Now simply click on install button presented in front of the available images. Once installed, the image would show 'How to run?' link in front of it. This link displays commands to run the pre-installed image. All installed images can be seen under the Images tab.

{#PageBreak#}

{#PageBreak#}

Using Command line to run image

To run images go to Machine A and run following commands:

if < ! -e ~/.euca/mykey.priv >; then

mkdir -p -m 700 ~/.euca

touch ~/.euca/mykey.priv

chmod 0600 ~/.euca/mykey.priv

euca-add-keypair mykey > ~/.euca/mykey.priv

fi

These commands are to create a keypair (ssh key) that you can use to log into your instance as root, once it boots. As the key is stored, you only need to create them once. You also need to allow access to port 22 of instance. This can be done using:

euca-authorize default -P tcp -p 22 -s 0.0.0.0/0

Then create instances of your registered images as follows:

euca-run-instances $EMI -k mykey emi-E1461079 -t m1.small ?addressing private

You can get an image id from the web interface of the Cloud controller by clicking on 'How to run?' under the 'Store' tab. You can monitor state of instance using the following:

watch -n5 euca-describe-instances

Output of this command shows information about the instance, including its state. While first-time caching is being performed, the instance's state will be 'pending'. When the instance is fully started, the above state will become 'running'. Look at the IP address assigned to your instance in the output and then connect to it:

IPADDR=$(euca-describe-instances | grep $EMI | grep running | tail -n1 | awk '{print $4}')

ssh -i ~/.euca/mykey.priv ubuntu@$IPADDR

To terminate instance run

INSTANCEID=$(euca-describe-instances | grep $EMI | grep running | tail -n1 | awk '{print $2}')

euca-terminate-instances $INSTANCEID

{#PageBreak#}

Using Hybridfox extension to run Images

Besides using command line, one easy and efficient way to manage your Ubuntu private cloud is via Firefox extension known as Hybridfox. Download this extension from http://code.google.com/p/hybridfox/. Once installed, you need to connect it to Ubuntu private Cloud. Click on  'Region' on the top. This would pop up a window named 'Manage EC2 Regions' asking for region name and endpoint URL. Name region (India, in our case) and put 'http://192.168.2.10:8773/services/Eucalyptus' where '192.168.2.10' is the IP address of Machine A. Second important step to be carried out is to add credentials to hybridfox. Click on 'Credentials' tab on the top; this would pop up 'Manage EC2 Credentials' window. Get value of 'AWS Access Key' and 'AWS Secret Access Key' from '192.168.2.10', click on 'Credentials' tab and then click on 'Show keys' button under 'Query interface credentials'. Here 'Query ID' is 'AWS Access Key' and 'Secret Key' is 'AWS Secret Access Key'. Now refresh your page and you should get all the installed images under 'Images' tab. Simply select the listed instance and click on 'Start Instances(s)' button on the top (green start icon). This would imitate instance and assign Elastic IP to it. You can see the different states of imitated Instance; initially it starts as 'pending' then changes to 'running' and when terminated changes to 'terminated'.  To get list of available Elastic IP's, simply click on 'Elastic IP's' tab on the top. Elastic IP's are nothing but public IP's that were defined during installation on Machine A. These are assigned to instances for access by users.

Allocate Resource on the fly

To summarize IaaS in plain English, it gives the capability of adding/ subtracting resources to your application on the fly. Suppose you had anticipated that x number of CPU's and y amount of memory would be enough to run your application, but later you found  that extra memory/storage/processing power was needed, you might be in a fix. However, if you are using Cloud setup, it is just a matter of few clicks that would add those extra resources needed by your application and that too on the fly. Here, I show you how to create a virtual storage unit and then attach this storage unit to running instance on the fly. In Hybridfox, go to 'Volume and Snapshots' tab on the top. Here you can create volumes, click on green plus sign and fill in amount of storage required plus put available zone (Cluster1, in our case). Once volume is created, it would be assigned a volume ID, now select this volume and attach it to running instance by clicking on green button with tick mark. In 'Attach Volume' pop up window, select Instance and add device name '/dev/sdb'. With this, you have successfully created and attached volume to your running instance. You can also detach volumes by clicking on red button with cross image.

All these steps show how easy it is to take the first step in creation of your own private Cloud and that too with minimal investment.