Main Page
From Ec2
Amazon Ec2
Getting Started
Also see Amazon Ec2 getting started guide.
Sign Up
Create/Download X.509 certs
This is done on the amazon web site under Web Services Account
Download and Configure Command Line Tools
Download the command line tools
Configure the following environment variables:
export EC2_HOME=/opt/ec2 export PATH=$PATH:$EC2_HOME/bin export JAVA_HOME=/usr/lib/jvm/java-1.5.0-sun-1.5.0.14/ export EC2_PRIVATE_KEY=/path/to/pk-DV6QP65RY47SLP473YSIDFDDGDXDBPAFO.pem export EC2_CERT=/path/to/cert-DV6QP65RY47SLP4DFFSIKKDGDXDBPAFO.pem
Verify command line tools are installed/configured by issuing this command:
ec2-describe-images -x all
Add Keypair
Add a keypair so that we can access images from our environment
ec2-add-keypair foo-keypair
Returns private key. Copy/paste this to a file called foo-keypair. Append a newline to end of file, to workaround putty (ssh client) bug.
Start Image Instance
ec2-run-instances ami-78b1452323 -k foo-keypair
which will return
RESERVATION r-aec225c7 183822812427 default INSTANCE i-ccdf3ca4 ami-15b6534c pending foo-keypair 0
where ami-78b1452323 is a valid instance listed when running ec2-describe-images -x all
If the keypair is not provided, there will be no way to ssh into it.
This can take up to 10 minutes to initialize.
To check on the instance to see if its up and running
ec2-describe-instances i-ccdf3ca4
Open Firewall Ports
Open up firewall ports for port 80 (webserver) and port 22 (ssh)
ec2-authorize default -p 22 ec2-authorize default -p 80
default means "use the default security group"
More examples:
ec2-authorize default -P udp -p 1000-32768
To show the firewall ports that are currently open
ec2-describe-group default
SSH into instance
Via OpenSSH client
ssh -i foo-keypair root@domU-12-34-31-00-00-05.usma1.compute.amazonaws.com
replacing domU-12-34-31-00-00-05.usma1.compute.amazonaws.com with the hostname for your particular instance.
Via Putty
Convert Keypair
Putty cannot use the foo-keypair we generated earlier, so we need to convert it via puttygen.
- Go to conversions/import key
- Import foo-keypair
- Optionally add a passphrase (password) to protect it
- File/Save private key as putty-foo-keypair.ppk
Run Putty
- Go to Auth node in LHS config tree
- Where it says "Private key for authentication", browse for putty-foo-keypair.ppk
Find Hostname
ec2-describe-images
At this point, you can paste the hostname into your browser and you will see a default web page from Apache.
Login
- Tell putty to ssh into hostname found above
- Login as root
- No password needed (all in the certificate)
Command Line Tools
Show Available Shared Images
ec2-describe-images -x all
or
ec2-describe-images -o self -o amazon
Show Running Instances
ec2-describe-instances
This will show any instances you have running, as well as the hostname for ssh'ing in.
Terminate an instance
ec2-terminate-instances i-823424ce
View Console Output
ec2-get-console-output i-10b35d79
Disk Storage
By default, your running instance only has access to a temporary storage space. If you run df you may see 9 gigs free on the root file system, but if you terminate your instance, anything you had written in the first instance will have disappeared. If you reboot the instance, or the instance self-reboots, or the host is rebooted, your data will still be there.
For packages, the solution is to rebundle a new AMI.
For persisting data, there are many different solutions. One particularly attractive solution for data is too use the Amazon S3 storage system for your persistent data.
Mounting S3
s3fs
http://code.google.com/p/s3fs/wiki/FuseOverAmazon
S3InfiDisk for EC2
This is a commericial product, details here.
Mounting Other
GlusterFS
This is an open source product, GlusterFS. It does not have S3 support, but this is planned for version 1.5. According to this testimonial, it is already possible to get it working on Amazon Ec2.
HadoopDFS
http://lucene.apache.org/hadoop
OpenAFS
Samba
Kosmix
Synchronizing to Persistent Store
Another option is to periodically synchronize to another server that has a persistent store available. For databases that have built-in replication options, that is one option. Another possibility is to dump the database to a file and rsync the file to another server, or into S3.
Articles
API Access
Python
Libs
Articles
Building AMI's
First, check out the official Amazon documentation on EC2. Also check the full list of public AMI's
Rebundling Tutorial
Rebundling is the act of creating a new image from within an existing image, rather than creating one from scratch.
For more details, see official amazon tutorial
Upload Keys
Scp your .pem keys up to the instance
$ scp -i your-keypair *.pem root@ec2-75-44-43-79.z-2.compute-1.amazonaws.com:/root
Bundle Volume
Example ec2-bundle-vol command
ec2-bundle-vol -d /mnt -k pk-DV6QW65RY47SLP473YSIKKWGDXDBPAFO.pem -c cert-DV6QP23RY47SLP473HSIKKDGDXDBPAFO.pem -u 293622712127 -s 8000
where -u is your account id (this is not your aws username -- see faq) and -s specifies the size, in this example approx 8G
Upload to s3
ec2-upload-bundle -b my_foo_ami -m /mnt/image.manifest.xml -a 1MF54KFA8X88DEJQG2 -s AkFGGDFJHDJFHDNTFvWUZitt9yXj0R2Z
where my_foo_ami is a globally unique "bucket name" to be used for S3. -a is the aws username, and -s is the aws secret password. If another user already used this bucket name it'll fail.
Register AMI
This step can be done on your workstation, since your instance may not have the command line tools installed.
$ ec2-register my_foo_ami/image.manifest.xml
result:
IMAGE ami-5bae4b32
Unregister AMI
If you later decide to remove the AMI, you can unregister it via:
$ ec2dereg ami-accd18c4
NOTE: this probably does not delete the storage from S3, which will need to be done seperately.
Tutorials
HOWTO Building a self-bundling Debian AMI by Paul Morris
How to create a Fedora 6 Instance for EC2
How to create a Fedora 7 Instance for EC2
Scripts
Sharing AMI's
Tutorial on how to share AMI's
Rebundling AMI's
If the AMI-Tools are installed on the image, you can rebundle the running image as required.
This is explained here.
Tips
When inter-communicating between instances, use internal ip's
FAQ
Q: Can I ssh into it?
Yes
Q: Can I install new packages to an image?
Yes, but if you terminate the instance and start a new instance with a the same AMI, your newly installed packages will be gone.
OTOH, if you just reboot the instance, the data will survive.
Actually, it's better than "It just won't be there if the instance reboots". It *will* still be there after an instance reboot. It can even be there after a host reboot. The only time it will be gone is if the instance is terminated, whether as result of the user issueing a terminate instance call, the instance itself doing a shutdown, or as result of some malfunction.
Q: Does it have local storage? How much?
Here is the output from a "df . -h" command
[root@domU-12-31-36-00-3D-11 ~]# df . -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 9.9G 829M 8.6G 9% /
So there is approximately 9G free.
Not sure .. check out http://www.architected.info/blog/using-s3-as-a-file-system
Q: How do I get www.mydomain.com to point to my Ec2?
- Find the hostname of your instance using ec2-describe-images
- Modify the DNS server which is acting as the nameserver for your domain, and create a CNAME record that points to the instance hostname
Q: What is my AWS account ID?
Login and go to My Web Services Account / Account Activity. Along the top you will see a 12 digit account id.
Q: How do I get the kernel source?
wget http://s3.amazonaws.com/ec2-downloads/linux-2.6.16-ec2.tgz
Q: How do install ami bundling tools on Fedora Core 7?
yum install ruby wget http://s3.amazonaws.com/ec2-downloads/ec2-ami-tools.noarch.rpm rpm -i ec2-ami-tools.noarch.rpm
Q: Can I run FreeBSD kernels?
Not yet, but soon. OpenSolaris is currently avaiable, though.
Errors
Client.InvalidSecurity: Request has expired
Check the current date and time of the PC/server from which the command is being issued. This message is received when the time/date with the request is incorrect. See forum thread
Resources
Amazon EC2 Resource Center
http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=84
Shared Images Directory
http://developer.amazonwebservices.com/connect/kbcategory.jspa?categoryID=101
