Zougg Blog

So I was told the other day that I had 48 hours to migrate one of my previous AMI (Amazon Machine Image) instances, because the instance was running on degraded hardware (which apparently had some hardware failure). Amazon was pro-active about it and had stopped billing for this particular instance. In any case, I was able to login to this instance and retrieve files that are specific to this instance.

One great thing about EC2 is that there is a wide selection of public AMIs available. This means one can experiment with different system configurations, from the different Linux distributions (e.g. Debian, Fedora, Gentoo) to the software installed (e.g. a full LAMP stack). Since my old instance has been running the Amazon-provided Fedora4, I figure it’s time to try something else and also document the steps.

1. Getting the certificate and private key ready
Amazon provides a set of command-line tools (written in Java) for working with EC2. These tools use HTTPS to communicate with the web service and therefore requires X.509 certificates and private keys to be around. Once you are in the EC2 program (currently Beta at aws.amazon.com), you can generate the required certificate and keys. Since I already have these generated before, there’s little to do here. Amazon recommends putting these credential files in, say, ~/.ec2, like so:

$ ls -alkn ~/.ec2
total 12
drwx------ 5 501 501 170 Jul 14 15:05 .
drwxr-xr-x 53 501 501 1802 Jul 14 14:11 ..
-rw-r--r-- 1 501 501 689 Jul 14 14:11 cert-A6O5VGEIFPYKTCNTXVK4D2XE5ESNCB7U.pem
-rw-r--r-- 1 501 501 721 Jul 14 14:11 pk-A6O5VGEIFPYKTCNTXVK4D2XE5ESNCB7U.pem


2. Install EC2 Tools
The command line tools are available from the Developer Connection site (here). These tools also expect Java to be installed on your machine. Once installed, the tools are ready for use after setting a few environment variables. For example, the downloaded zip file unzips to directory foo, and the environment variables are then set as follows:

export EC2_HOME=~/projects/ec2/ec2-api-2007-03-01/ec2-api-tools
export PATH=$EC2_HOME/bin:$PATH
export EC2_PRIVATE_KEY=~/.ec2/pk-A6O5VGEIFPYKTCNTXVK4D2XE5ESNCB7U.pem
export EC2_CERT=~/.ec2/cert-A6O5VGEIFPYKTCNTXVK4D2XE5ESNCB7U.pem

That’s it!

3. Finding public AMIs, checking instances
Several commands are useful:

ec2-describe-images -a This shows a list of available AMIs. This has grown to be a pretty long list from the early days of EC2. This information is also available at http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=101

Since this new instance will be serving web pages with a Java and S3 backend, an AMI that is small, optimized for Xen, would be ideal. For this, I decided to try out the Gentoo distribution (ami-8b8a6fe2). This particular AMI has the basics like openssh and Apache2 and that’s about it.

ec2-describe-instances basically shows any currently running instances. There is also a Firefox plugin that simplifies much of these tasks by providing a nice UI. Still, command line tools are scriptable…

4. Running a new AMI instance
With the tools in place and the cert/keys ready, starting up a new instance is pretty trivial. First, we need to generate a keypair for the new instance. An instance of a public AMI has no password and logging in via ssh requires public/private keypair. Since this is specific to my instantiations of the AMI, a keypair need to be generated so that one half of the keypair is embedded in this AMI allowing me to login later on with the other half of the keypair. To do this, simply use the command:

ec2-add-keypair ami-8b8a6fe2-gentoo-base-eminent

The key name ami-8b8a6fe2-gentoo-base-eminent is created following a simple convention that denotes which AMI (ami-8b8a6fe2) and the configuration (gentoo-base-eminent). This key is important in later starting up new instances of this AMI. This command prints out the private RSA key necessary for ssh login (via the -i option) later:

-----BEGIN RSA PRIVATE KEY-----
MIIEpgIBAAKCAQEAmmVOcPrBRXgGbo3XtvKxld/Glmuqi9gGKLNzyfUspKCuSjwmgHB91y7e8aH+
tGyHdbYnHPC/nNbh15F3jjdneM5W1GphcUJu4m2HylAklgTOC8pYVdS8XacKiGSBaUXvZimXCsH/
Uzcm3rxfxwNESwWpsg9aPXYi//T0quqM1xvZNFXO1s1s5ZJfKugCUUJrq365afaOR1hiipx+02U5
zKSTYZc9XWKbbaNSSeIDCPh8CZTxEH/FEuutaMxisMJ26uAqD0plnc1sj+mv8NNCl+/XgTlPLzVg
...
-----END RSA PRIVATE KEY-----

Since this output needs to be captured in a file, we can just do this:

ec2-add-keypair ami-8b8a6fe2-gentoo-base-eminent > ~/.ec2/ami-8b8a6fe2-gentoo-base-eminent.id

Change the permission of the id file (chmod 600 ~/.ec2/ami-8b8a6fe2-gentoo-base-eminent.id) or ssh won’t like it! Now that we have the keypair, start up a new instance:

ec2-run-instances ami-8b8a6fe2 -k ami-8b8a6fe2-gentoo-base-eminent

This here starts up one instance of the Gentoo AMI (ami-8b8a6fe2) identified by the keypair (ami-8b8a6fe2-gentoo-base-eminent) . This instance is started with the default group. In order to gain ssh access, we need to authorize port access (for the default group):

ec2-authorize default -p 22 for ssh, and ec2-authorize default -p 80 for HTTP.

5. Connecting to the new AMI instance
Now to connect to the new instance, we first find out how to get to it:

$ ec2-describe-instances
...
INSTANCE i-8d688be4 ami-8b8a6fe2 ec2-72-44-51-245.z-1.compute-1.amazonaws.com ...

To connect to it, simply ssh to the hostname listed above:

ssh -i ~/.ec2/ami-8b8a6fe2-gentoo-base-eminent.id root@ec2-72-44-51-245.z-1.compute-1.amazonaws.com


That’s it!