Converting EC2 S3/instance-store image to EBS

Amazon’s instance-store images are convenient, but ephemeral in nature. Once you shut them down, they’re history. If you want persistence of data, you want to use an EBS instance that can be stopped and started at will without losing your info. Here’s the process I went through to convert a Wowza image to EBS for a client to use with a reserved instance. I’m going to assume no configuration changes for Wowza Media Server, as the default startup package is fairly full-featured. This process works for any other instance-store AMI, just ignore the Wowza bits if that’s your situation.

Boot up a 64-bit Wowza lickey instance. I was working in us-east-1, so I used ami-e6e4418f, which was the latest as of this blog post.

Once it’s booted up, log in.

Elevate yourself to root. You deserve it:

sudo su -

Stop the Wowza service:

service WowzaMediaServer stop

delete the Server.guid file. This will cause new instances to regenerate their GUID.

rm /usr/local/WowzaMediaServer/conf/Server.guid

Go into the AWS management console and create a blank EBS volume in the same zone as your instance.

Attach that volume to your instance (I’m going to assume /dev/sdf here)

Create a filesystem on it (note: while the console refers to it as /dev/sdf, Amazon Linux uses the Xen virtual disk notation /dev/xvdf):

mkfs.ext4 /dev/xvdf

Create a mount point for it, and mount the volume:

mkdir /mnt/ebs
mount /dev/xvdf /mnt/ebs

Sync the root and dev filesystems to the EBS disk:

rsync -avHx / /mnt/ebs
rsync -avHx /dev /mnt/ebs

Label the disk:

tune2fs -L '/' /dev/xvdf

Flush all writes and unmount the disk:

sync;sync;sync;sync && umount /mnt/ebs

Using the web console, create a snapshot of the EBS volume. Make a note of the snapshot ID.

Still in the web console, go to the instances and make a note of the kernel ID your instance is using. This will be aki-something. In this case, it was aki-88aa75e1.

For the next step, you’ll need an EC2 X.509 certificate and private key. You get these through the web console’s “Security Credentials” area. This is NOT the private key you use to SSH into an instance. You can have as many as you want, just keep track of the private key because Amazon doesn’t keep it for you. If you lose it, it’s gone for good. Download both the private key and certificate. You can either upload them to the instance or open them in a text editor and copy the text, and paste it into a file. Best place to do this is in /root. Once you have the files, set some environment variables to make it easy:

export EC2_CERT=`pwd`/cert-*.pem
export EC2_PRIVATE_KEY=`pwd`/pk-*.pem

once this is done, you’ll need to register the snapshot as an AMI. It’s important here to specify the root device name as well as map out the ephemeral storage as Wowza uses those for content and logs. Ephemeral storage will persist through a reboot, but not a termination. If you have data that needs to persist through termination, use an additional EBS volume.

ec2-register --snapshot [snapshot ID] --description "Descriptive Text" --name "Unique-Name" --kernel [kernel ID] --block-device-mapping /dev/sdb=ephemeral0 --block-device-mapping /dev/sdc=ephemeral1 --block-device-mapping /dev/sdd=ephemeral2 --block-device-mapping /dev/sde=ephemeral3 --architecture x86_64 --root-device-name /dev/sda1

Once it’s registered, you should be able to boot it up and customize to your heart’s content. Once you have a configuration you like, right-click on the instance in the AWS web console and select “Create Image (EBS AMI)” to save to a new AMI.

Note: As of right now, I don’t think startup packages are working with the EBS AMI. I don’t know if they’re supposed to or not. 

8 Comments On “Converting EC2 S3/instance-store image to EBS”

  1. Thanks Ian for this, it is a helpful reference. One thing to note for some who may be getting snapshot not found errors, make sure your region is specified when your snapshots not in eastern us. In my case I had to specify –region us-west-2. 

    Reply

  2. Hi, excellent and helped me a lot (I had troubles with the startup packages.

    Two things
    – the aws webinterface allows to the last part very easy now
    – the password to the new  image stays also the same (standard the name of the instance i-xxxxxxx)

    Reply

  3. Thnx for the post!

    small note: because the EBS is only available in the zone of creation i had to add
    –region eu-west-1

    to the ec2-register command

    Reply

  4. Amazing article thank you.

    Reply

  5. Pingback: Ready to go Wowza 3.5 EBS-Backed AMI | Hire Alek

  6. Hi Ian.

    I have an annoying error at the very end of the procedure.
    File not found: ‘/root/pk-*.pem’
    for the life of me, I cant understand what does it want from me. both Cert and private keys are in the /root
    Hope you can help.
    Thanks
    Svyatko

    Reply

  7. Just want to say thank you, this worked great for me!

    Reply

Leave a Reply

Your email address will not be published.