Jupiter

Jupiter is designed to expedite the deployment of new virtual machines either by cloning an existing virtual machine, even if it is running, or by using a pre-built template. The goal was to allow developers to provision a new VM in only a few minutes without the need to bother system admins unnecessarily. Jupiter can also perform a few different virtual machine administration tasks documented below.

Limitations

Currently Jupiter is only tested on vSphere 5 using Ubuntu 12.04.2 LTS 64-bit virtual machines. That being said, most of the vSphere calls are via the VMware API or standard UNIX commands and should be consistent with later versions... in a perfect world that is. Also the VM templates have a few requirements to be fully functional with Jupiter. Namely a single LVM partitioning and a tweaking of the sudoers file documented in more detail below.

Also, even though Jupiter is destined to be a gem, it is currently controlled only via command-line options. Programatic control will be implimented in the future.

Document Conventions

The terms 'Host', 'vSphere' and 'ESXi' are used interchangeably in the document. All refer to the VMWare Host Server that is being managed.

The terms 'Guest', 'VM' and 'Virtual Machine' are all used to refer to the virtual machine that is running on the ESXi server.

Installation

Add this line to your application's Gemfile:

gem 'jupiter'

And then execute:

$ bundle

Or install it yourself as:

$ gem install jupiter

Or just pull down the git repository and use as a command-line application.

Configuration

The god Jupiter desires the following sacrifices to function properly:

  1. Root user access to the vSphere ESXi host server
  2. Adding an ssh key to the ESXi server so the templates can be scp'd over without a need to enter a password
  3. A common 'deploy' user on the VM templates with sudoer privileges and sudoer file entries (Jupiter can revoke the privileges when done setting up the new VM)

All the configuration information used by Jupiter is stored in the config.yaml file. It is broken up into four sections.

The vmusers section stores the deploy user who has sudoer privileges on the virtual machines.

:vmusers:
  :deploy_user: deploy_users_name
  :deploy_pass: deploy_users_pass

The hosts section stores the vSphere server information. You can have as many servers listed here as you wish. The desired sever is selected when Jupiter is called via the --host= switch. We use the symbolic link path for vmpath instead of the crazy actual string created by the ESXi server.

:hosts:
  -
    :name:      ESXi-SERVER01
    :vmpath:    /vmfs/volumes/VMDATA
    :host:      192.168.100.100
    :sshport:   22
    :sshuser:   root
    :sshpass:   roots_password
    :dcname:    ha-datacenter

  -
    :name:      ESXi-SERVER02
    :vmpath:    /vmdata
    :host:      192.168.100.209
    :sshport:   2500
    :sshuser:   root
    :sshpass:   roots_password
    :dcname:    ha-datacenter

The templates section listed where your pre-made template files will be stored and what user has privileges to transmit them to the ESXi server.

:templates:
  :host:      192.168.100.23
  :sshport:   2500
  :sshuser:   template_admin_user
  :sshpass:   template_admin_user_pass
  :location:  /vm_templates

The Nagios section is needed if you wish to generate a Nagios configuration file for your virtual machine and upload it to the nagios server via Jupiter. This section can have null values if you wish to only generate the config and not upload it. The sshuser must have write privileges to the Nagios object location directory in order to successfully upload the config.

:nagios:
  :host:      192.168.1.222
  :sshport:   2500
  :sshuser:   franky
  :sshpass:   frankiespassword
  :location:  /usr/local/nagios/etc/objects
  :url:       http://192.168.1.222:8000/nagios/

Template Virtual Machine

Templates are by far the quickest way to have a virtual machine up and running. However you must create the template first. The templates Jupiter was designed with are based on Ubuntu Server LTS 64-bit but Ubuntu's conventions are pretty consistant from version to version. (Although not tested yet, this is probably true for Debian also.) Here are the considerations when building your template:

Hardware Setup

The VMware hardware defaults seem to work just fine. The only considerations are ram and hardisk size. Ram can be whatever size you like but consider what is appropiate for the corresponding swap disk size. Hardisk sizes can be any size also because Jupiter can resize them for you once the VM is created. However, this fuctionality is only possible if you setup your partitioning using LVM as described below. For template deployment speed it is best to have a smaller initial hardisk. Ubuntu seems to want to have at least 2 GBs of diskspace to install a basic system, so a good starting HD size for the template would be 3 GBs. 1 GB for swap space and 2 GBs for the root partition.

HD Partitioning

TODO: Make this clearer by numbering steps (Review actual installation to make sure this is correct)

This is not meant as a definitive source for installation instructions. There are many reputable sources for information on installing Linux that you can review, so we will only touch on the specific needed for the Jupiter template VM.

Follow Ubuntu's installation steps until you get to the Partition disks step. Select Manual as the partition method.

Because we want to later resize the partition on the template, we need the swap partition to be first on the disk. This leaves any room we add to the drive after our LVM parition so that it can grow to fill the unused space. If a swap partition is at the end of the disk we have a roadblock.

Select 'sda' since we only created one disk. Create a new partition and set the size to 1GB or however big you want your swap space to be. Set it to 'Primary', 'Beginning' and format 'swap'.

Next, create another logical parition (it should be sda2). Then change "Use as" to "physical volume for LVM" then "Done setting up the partition". Now select "Configure the Logical Volume Manager" at the top, and choose "Yes" to write the changes to disk. Then use the LVM Configuration menu to setup first a 'Volume Group' then a 'Logical Volume'. Name them something short and simple like 'VolGrp1' and 'LogVol1'. Choose "Finish" and you should be back at the main "Partition Disks" screen.

Now add a filesystem to the new LVM. Select the partition under "LVM VG VolGrp1, LV LoVol1", or whatever name you have chosen. Setup a file system as ext4 selecting "/" as the mount point. Once done, select "Done setting up the partition".

Adding Deploy User

Ubuntu doesn't like people (or Jupiter) logging in as root, thus the vmusers entry in the config file above. So when building a virtual machine it is necessary to add the deploy user with sudo privileges. This can be done during the installation or afterwards. Please note that Jupiter has an option to revoke the deploy users sudoer privileges. Make sure you have at least one other user with sudoer powers before doing this.

Software

You may wish to have other software on your template but Jupiter only requires a functional OpenSSH server and vmware-tools. Installing only the base system and the OpenSSH server will keep your template small but vmware-tools is needed for many of Jupiter's features to work properly.

Post Install Template Configuration

We have a couple of tasks to perform on our template before sending it to our template server. First we need to log into our running template machine as our deploy user and edit out sudoer file.

To edit the sudoer file we use visudo.

sudo visudo

Then add the following line to the end of the file (Changing the deploy_user_name of course):

deploy_user_name ALL = (root) NOPASSWD: /bin/hostname, /bin/chmod, /bin/echo, /bin/sed, /etc/init.d/hostname, /usr/bin/gpasswd, /sbin/fdisk, /sbin/pvs, /sbin/pvresize, /sbin/vgs, /sbin/lvextend, /sbin/resize2fs

Now we shutdown our template VM and compress that baby into a tar file.

Once the template is shutdown log onto the host server it was created on and cd to its parent directory. Example:

cd /vmfs/volumes/VMDATA/

Once there list the directory and note your templates name.

ls -lah

drwxr-xr-t    1 root     root        1.6K Mar 26 10:39 .
drwxr-xr-x    1 root     root         512 Mar 26 15:10 ..
-r--------    1 root     root        9.3M Dec  6 11:27 .fbb.sf
-r--------    1 root     root      254.7M Dec  6 11:27 .fdc.sf
-r--------    1 root     root        1.1M Dec  6 11:27 .pb2.sf
-r--------    1 root     root      256.0M Dec  6 11:27 .pbc.sf
-r--------    1 root     root      250.6M Dec  6 11:27 .sbc.sf
-r--------    1 root     root        4.0M Dec  6 11:27 .vh.sf
drwxr-xr-x    1 root     root        2.1K Feb 28 15:58 Continuous-Deployment
drwxr-xr-x    1 root     root        2.1K Mar 26 12:00 Puppet-Server
drwxr-xr-x    1 root     root        1.4K Mar 22 15:30 Ubuntu64-Template
drwxr-xr-x    1 root     root        2.3K Feb  7 15:07 Resque-Server

In this example we called it 'Ubuntu64-Template'.

Now lets tar and gzip that bad boy up with the following 2 commands. Note this may take a few minutes to run.

tar -cf Ubuntu64-Template.tar Ubuntu64-Template
gzip Ubuntu64-Template.tar

Once this is done you will have a file called 'Ubuntu64-Template.tar.gz' which will act as your template. We will move this file over to your template server below. (Any machine with space and ssh access will do. Possibly even your local workstation.)

Template Server

The template server currently can be any system that has SSH access and SCP capabilities. Create a user that has permission to access a directory that will hold the template tar.gz files. We are going to grant this user password-less access to the ESXi (Host) server by adding a ssh key to the host. This is necessary because ESXi servers do not seem to have a functional SCP command.

Security Concerns: It is not ideal to setup password-less access to any *nix system but this seemed the easiest solution for the moment. Please make sure the Template server is a secured system and that the template user is not used for any other services on that system.

Log into the Template server as the user you have configured and run the following commands:

# First lets create a password less ssh key for the template server user:
cd ~/.ssh
ssh-keygen -t rsa
# Do not enter any passphrase when asked so we can have password-less access.
# Now we will add our new key to the host server like this:
cat .ssh/id_rsa.pub | ssh [email protected] "cat >> .ssh/authorized_keys"
# Where 192.168.100.100 is the IP of the host ESXi server. Enter the root users password
# when asked. Note that you will have to run the above cat command for every ESXi host
# server you wish to manage with Jupiter.

The Template server will now have ssh root access to the Host server without a password.

Now to move the Ubuntu64-Template.tar.gz template we created above to the Template server:

# Create the template directory that is referenced in the template section of the yaml file. Example:
mkdir /vm_templates
# Them download the tar.gz file from the Host:
scp [email protected]:/vmfs/volumes/VMDATA/Ubuntu64-Template.tar.gz /vm_templates/
# '192.168.100.100' being the IP of the Host server
# '/vmfs/volumes/VMDATA' being the vmpath
# 'Ubuntu64-Template.tar.gz' being the name of the template you created
# '/vm_templates/' being the template directory you created above

After the file is done downloading it is safe to remove from the ESXi server but not necessary. The first time Jupiter uses that template the file on the Host server will be removed anyway.

Notes on template file names

The template file name must match the template folder name and file names used in the vm. Jupiter changes the names as needed when deploying a new VM. Keep this in mind when backing up your compressed templates. They will not work properly until you change their name back to thier original name. Just renaming all the files the same name will not work because of the references to files used inside the VM image itself.

Example:

Template-Franks-Awesome-VM.tar.gz
  |
  V
Folder: Template-Franks-Awesome-VM
          |
          V
        Template-Franks-Awesome-VM-flat.vmdk
        Template-Franks-Awesome-VM.nvram
        Template-Franks-Awesome-VM.vmdk
        Template-Franks-Awesome-VM.vmsd
        Template-Franks-Awesome-VM.vmx
        Template-Franks-Awesome-VM.vmxf

If I rename this file to Template-Franks-Awesome-VM-Version2.tar.gz it will still show up in the usable template list but not function properly.

Stating the obvious... Best practice would be have a basic template file that has a properly configured LVM partition and keep using it to make more templates that are specialized. If one wanted to make a Nginx, MySQL template, one would use Jupiter to make a new virtual machine using the basic template, name it to something like "nginx-mysql-template" and then install nginx and mysql. Shutdown the new VM and tar/gzip that bad boy back into another template. Then upload it to the template server. Now both "basic-template.tar.gz" and "nginx-mysql-template.tar.gz" will be available as templates through Jupiter.

TODO: Finish instructions for Host and Template servers

BASIC USAGE

To run Jupiter from the command-line there are a number of options

jupiter ip --guest=GUEST --host=HOST          # Shows IP address for supplied VM
jupiter manage --host=HOST                    # Provision a new or manage an existing VM
jupiter register --guest=GUEST --host=HOST    # Registers the named VM on this ESXi server
jupiter state --guest=GUEST --host=HOST       # Shows power state for supplied VM
jupiter unregister --guest=GUEST --host=HOST  # Unregisters VM from supplied ESXi server

Most of the time you will use the 'manage' option and pass it the name of the entry for the desired ESXi server in the yaml config file.

Example:

jupiter manage --host=ESX-SERVER77

Jupiter will ask if you would like to create a new VM or work with an existing VM registered on this ESXi server.

################################################################################
############################## WELCOME TO JUPITER ##############################
################################################################################
## We hope the god of sky and thunder will make your day most good and stuff. ##
################################################################################

 Would you like to create a new virtual machine or work with an existing one? 

 1. Clone / Create a Virtual Machine      
 2. Work with an existing Virtual Machine 

Please select a number:

Option 1 will give you a list of directories on the ESXi server in the VM images location path. (These VMs are not necessarily registered on the server.) You can choose to either clone one of these VMs or to create a new VM from a template.

################################################################################
######### These are the currect virtual machines found on ESX-SERVER77 #########
################################################################################

* Continuous-Deployment
* Puppet-Master
* nagios-openfire
* resque

################################################################################
 Would you like to clone one of these virtual machines or use a template? 

 1. Clone Virtual Machine 
 2. Use a Template 

Please select a number:

If you choose to clone a VM you will be asked to which one and what the new name for the VM should be. This process can take quite a while since large files are copied over during the cloning process. This will all depend on the size of the VM and speed of the server.

If you choose to create a VM from a template Jupiter will list the tar.gz files you have on your template server to choose from. Although this process is much faster than cloning, it still needs to copied the archived VM over to the server. You will also be prompted to enter a new name for the VM. Jupiter will take care of renaming all the appropriate files as needed.

Restrictions: When naming a new virtual machine please only use a valid file name format. (No spaces and no special characters with the exception of "-" and "_") Also please avoid using consecutive zeros in the name as these are also used in naming snapshot files.

Option 2 will ask you to select a virtual machine and then present you with a number of tasks that can be preformed on that VM.

################################################################################
############### Please select a task to perform on XXXXXXXXXXXXX ###############
################################################################################

  1. Change hostname
  2. Reboot (graceful restart)
  3. Shutdown (graceful stop)
  4. Cycle Power (non-graceful restart)
  5. Kill Power (non-graceful)
  6. Remove deploy user from sudoers
  7. Display current IP address of XXXXXXXXXXXXX
  8. Increase the alocated HD space for XXXXXXXXXXXXXX (VM must be off)
  9. Grow the LVM partition to fill the HD on XXXXXXXXXXXXXX
 10. Generate Nagios config file for XXXXXXXXXXXXXX

Most of these options will only work if you have correctly built you VM as described above and have vmware-tools installed on the guest.

  1. Will change the hostname of the Linux VM by updating the name in the hosts and hostname files and restarting the hostname service.
  2. Will send a graceful reboot command via the vSphere API.
  3. Will send a graceful halt command via the vSphere API.
  4. Will kill the power (hard reset) via vSphere API and reboot the VM. (Used for frozen systems.)
  5. Will cut the power to the VM via the vSphere API.
  6. Removes all sudoer privileges for the deploy user in the yaml file. Only use this if you wish to no longer manage the VM with non API commands. (Resizing HD, Changing Hostname) Do not use this option if plan on cloning or using this VM as a template in the future.
  7. Shows IP address of VM via vSphere API.
  8. This option will resize the HD space allocated to the VM by the ESXi server. The VM must be off for this to function. This only increases the amount of space on the virtual harddisk. To access the new space you must either grow it using Jupiters 'Grow LVM Partition' option or use standard linux tools to allocate the space.
  9. Once the virtual hardisk has been made larger you can automatically grow the root LVM partition to fill all of the new space. This is only possible if you properly partitioned the VM as described in the template creation section above.
  10. Will display a valid Nagios config file for this VM or upload one to the Nagios server in the yaml file if desired.

Contributing

  1. Fork it
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create new Pull Request