MyLinuxTips

The biggest reason most of us switch from instance store to EBS backed instance is that S3 backed (instance-store) images are not persistent i.e. they can loose the changes when rebooted whereas EBS provides “block level storage volumes for use with Amazon EC2 instances.”.  The other basic differences are:

1. instance-store AMIs cannot be “stopped,” only rebooted or terminated
2. Rebooting an instance-store AMI reverted the instance back to the AMI defaults

Since EBS is a high-performance attachable block device to a given instance, booting off of EBS AMIs has shown to be faster than the traditional instance-store boot as well.

Another benefit is that EC2 instances booted off of EBS volumes can be stopped, which effectively equates to shutting down a machine in the real world.  You are still responsible for the charges incurred while this instance is reserved, but all of the changes made to said instance will persist after you start it again.

The last major benefit of booting off the EBS volumes is that AWS has made it easy for you to create a new EBS AMI from a running EBS AMI.  In the Console, right clicking on an EBS instance will yield a new option, “Create Image (EBS AMI).”  This will basically shut down your instance, and proceed to generate a new EBS AMI from the contents on the disk of your instance.  This command seems to have a failure rate of ~40%, which can be a little frustrating.  I’ve found that if you put an instance into ‘stopped’ state before creating the EBS AMI, the process has a higher chance of success, but will still take anywhere from 20-45 minutes.

Now let’s start to focus on converting an instance-store AMI into an EBS AMI.

In order to perform this conversion, you will need to have an instance-store AMI that is the base OS you’d like to run. We are using CentOS 6 for the conversion here. The stuff I did in the console will be suffixed with [console],  and the stuff from CLI will be prefixed with #. Another thing is, I am assuming that EC2 API tools are already setup on your instance.

1. Booting an instance-store AMI – I executed the following to get a list of the images that fit my criteria (64bit, CentOS6, base install):

# ec2-describe-images –region ap-southeast-1 –all | grep -i CentOS6 | grep x86_64

IMAGE   ami-xxxxxxxx    BucketName/ImageName.manifest.xml
IMAGE   ami-xxxxxxxx    BucketName/ImageName.manifest.xml

Once you have the AMI, boot an instance with this AMI:

# ec2-run –region ap-southeast-1 -k $keypair  ami-xxxxxxxx

Note: $keypair in this case is the name of keypair used to SSH into the server

2. Customizing the EBS volume – After the instance is up and running, look to see which availability zone the instance is in.  If the region is ap-southeast-1, the availability zone is going to be either ap-southeast-1a, or ap-southeast-1b.  In either case, find out which availability zone your instance is in, and then create a 10gb EBS volume is the same zone.

Why 10gb?  10gb is the maximum size for an S3-backed AMI, which makes a 10gb volume the largest any instance-store AMI will be.  Obviously EBS AMIs can exist on larger volumes (all the way up to 1tb in size), and you can easily do so once you have an EBS-backed AMI.

After the EBS volume has been created, attach it to the running instance [console].  Remember what you chose as the device name the volume identified itself as (/dev/sdf for example).

In a root shell on the instance:

Create filesystem on fresh volume.

# mkfs.ext4 /dev/xvdf

Create a folder for mounting the volume & mount it there.

# mkdir /mnt/target && mount /dev/xvdf /mnt/target

Sync all files/folders under root (/).

# rsync -avHx / /mnt/target

Sync /dev

# rsync -avHx /dev /mnt/target

Flush all pending write operations and unmount the EBS volume

# sync;sync;sync;sync && umount /mnt/target

3. Creating the AMI – At this point, you should have a 10gb EBS volume that shows available [console].  Simply right-click on the volume and create a snapshot for the volume [console].  Once the snapshot has completed, select from the list of available kernels on ec2 with the following command:

# ec2-describe-images –region ap-southeast-1 | grep -i CentOS6

Store the AKI for the kernel you want to use in the environment variable AKI:

# export AKI=aki-xxxxxxxx

Up to this point, we have booted an instance-store AMI, created an EBS volume, synchronized the instance-store filesystem with the EBS volume, and created a snapshot of the EBS volume.  The only thing we need to do now is associate an AKI with the snapshot, and register the end result as an AMI in the EC2 repository.

# ec2-register –region ap-southeast-1 -s $SNAP –name $NAME –description “$DESC” –architecture $ARCH –root-device-name /dev/sda1

Where $SNAP is the ID of the snapshot, $NAME is the name of your AMI, $DESC is a description of the AMI, and $ARCH is either i386 (for 32-bit) or x86_64 (for 64-bit).  The command will return an AMI, which will be yours to boot from once it finishes!

Wait for a minute to get the image registered. Once you get the AMI ID you can use it for running any number of EBS backed instances.

Now that you have an EBS-backed AMI, any further customizations you make to this image can be preserved forever by simply right-clicking on the instance [console], and clicking “Create Image (EBS AMI).”

Cheers!!!