🖥️ EC2 From Scratch: A Server You Can Trust

Launching an Amazon EC2 server on purpose: a key pair you made yourself, a security group that opens one door, first-boot configuration checked before you use it, IMDSv2, backups with snapshots, and the clean-up

The decisions

An EC2 instance is a virtual server. Launching one takes a minute, and each choice you click through is a decision you will live with. The ones that matter, in the order of this page:

1. A key pair: how you log in.
2. A security group: which doors are open.
3. First-boot configuration (user data): what the machine sets up by itself.
4. An identity (an instance role): how it reaches other AWS services without keys.
5. The metadata service setting (IMDSv2).
6. Backups: snapshots of the disk.
7. A way out: stopping, terminating and cleaning up.

The size (instance type) and the image (AMI) matter too, and the free VM guide covers choosing ones that suit a free account. Pick a current Ubuntu or Amazon Linux image from the official list and a small instance type, and revisit them when the workload tells you to.

1. The key pair

A key pair is how you prove who you are when you connect over SSH. From AWS's documentation: it has a public key and a private key; for Linux, EC2 puts the public key into ~/.ssh/authorized_keys on the instance when it first boots, and you keep the private key. Two sentences from AWS to remember: "anyone who possesses your private key can connect to your instances that use the key pair", and EC2 does not keep a copy of your private key, so a lost one cannot be recovered (there are recovery procedures, but plan on not needing them). As an alternative to key pairs, AWS offers Systems Manager Session Manager, a browser or CLI shell that needs no SSH key (and no open port 22).

You can let AWS create the pair in the console (you download the private key once), or make your own and give AWS only the public half, which is better, because the private key then never exists anywhere but your machine. Making an ED25519 pair, as run:

user@lab:~$ ssh-keygen -t ed25519 -N '' -C 'lab-key' -f ~/.ssh/lab-key
Generating public/private ed25519 key pair.
Your identification has been saved in ~/.ssh/lab-key
Your public key has been saved in ~/.ssh/lab-key.pub
The key fingerprint is:
user@lab:~$ ls -l ~/.ssh
total 8
-rw------- 1 user user 399 Oct  6 18:49 lab-key
-rw-r--r-- 1 user user  89 Oct  6 18:49 lab-key.pub
user@lab:~$ cut -c1-60 ~/.ssh/lab-key.pub
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPd3Ovjw4dglg4fqqn8bDyBg
user@lab:~$ ssh-keygen -lf ~/.ssh/lab-key.pub
256 SHA256:eEehMv0z5x1Qq6txH0z1UBXxdLyUfY68QAeGXaNtXm0 lab-key (ED25519)
user@lab:~$ chmod 400 ~/.ssh/lab-key; ls -l ~/.ssh/lab-key
-r-------- 1 user user 399 Oct  6 18:49 ~/.ssh/lab-key

• ssh-keygen -t ed25519 -f ~/.ssh/lab-key made two files: lab-key (private) and lab-key.pub (public). The private key is created mode 600 (-rw-------), the public one mode 644.
• The public key is one line: the key type (ssh-ed25519), the key itself and the comment lab-key. This is the only thing AWS needs.
• ssh-keygen -lf prints the fingerprint (SHA256:...): a short summary that you can compare against the one the console shows after you import the key.
• The usual advice is to make the private key read-only for you (chmod 400), and SSH itself refuses a key file that others can read.
• Here I used no passphrase (-N '') to keep the demonstration non-interactive. On a real laptop, set a passphrase (or use an agent) so a stolen file is not enough.

To import your public key into AWS (from the documentation, not run here; check the supported key types for your case on the key pair page):

aws ec2 import-key-pair --key-name lab-key --public-key-material fileb://~/.ssh/lab-key.pub

An SSH config entry, and how to check it

You will connect often, so give the server a short name. The default user depends on the image: ubuntu on Ubuntu images, ec2-user on Amazon Linux. A file ~/.ssh/config with an entry, and the effective settings printed by ssh -G (which does not connect to anything):

user@lab:~$ cat ~/.ssh/config
Host lab-ec2
    HostName 203.0.113.25
    User ubuntu
    IdentityFile ~/.ssh/lab-key
    IdentitiesOnly yes
user@lab:~$ ssh -F ~/.ssh/config -G lab-ec2 | grep -E '^(hostname|user|identityfile|identitiesonly) '
user ubuntu
hostname 203.0.113.25
identitiesonly yes
identityfile ~/.ssh/lab-key

ssh -G lab-ec2 shows what SSH would use: user ubuntu, the address, the identity file, and identitiesonly yes, which stops SSH from offering every key in your agent (a long key list can hit the server's authentication-attempt limit and fail with "Too many authentication failures"). Connecting is then ssh lab-ec2. (The address 203.0.113.25 is from a range reserved for documentation: use your instance's real public address, or its name if you have one.)

2. The security group: one door, not all of them

A security group is the firewall of an instance. From AWS's documentation: it controls the traffic allowed to reach and leave the resources it is attached to, it is stateful (replies to allowed traffic are allowed automatically), and there is no additional charge. A server with no rule for SSH cannot be reached with SSH: only the traffic that a rule allows gets in. The best practices on AWS's own page:

• For ports 22 (SSH) and 3389 (RDP), authorize only specific address ranges: 0.0.0.0/0 (and ::/0) "enables anyone to access your instances from any IP address using the specified protocol".
• Do not open large port ranges.
• Create the minimum number of security groups, one for each kind of server.
• Limit who may create and change security groups (the IAM page).
• Security groups do not filter some traffic: Amazon DNS, DHCP, the instance metadata service, the time service and a few others.

For a small web server, the rules are two inbound: TCP 22 from your own address (a single /32), and TCP 80 and 443 from anywhere, since the web is public. Everything else is closed. (A web server's database or cache, if any, would accept connections from this security group only, by naming the group as the source: the networking page explains that idea.) From the documentation, not run here:

aws ec2 create-security-group --group-name web-sg --description "web server" --vpc-id VPC_ID
aws ec2 authorize-security-group-ingress --group-id SG_ID --protocol tcp --port 22 --cidr YOUR.IP.ADDRESS/32
aws ec2 authorize-security-group-ingress --group-id SG_ID --protocol tcp --port 443 --cidr 0.0.0.0/0

Your own address changes if you are on a home connection: check it before you rely on the rule, and update the rule rather than opening port 22 to everyone when it changes.

3. First-boot configuration: user data

User data is text that you hand to the instance at launch, and that it runs on its first boot. On Linux it is one of two kinds: a shell script (starting with #!) or cloud-init directives (starting with the line #cloud-config), the YAML format that most cloud images understand (and which the FreeBSD Corner's cloud page used with nuageinit). Facts from AWS's documentation:

• By default it runs only on the first boot, not on later restarts.
• It is limited to 16 KB before encoding. The AWS CLI base64-encodes it for you with run-instances.
• A script runs as root, so do not use sudo in it, and it cannot ask questions: use -y flags.
• The output goes to /var/log/cloud-init-output.log: the first place to look when a new server is not what you expected.
• The script is kept in /var/lib/cloud/instances/INSTANCE-ID/ and not deleted: remove it before you make an image from the instance.
• If the script calls AWS APIs, the instance needs an instance profile (a role: section 4), not keys.
• User data is not a safe place for secrets. Treat it as readable by anyone who can read the instance or its settings.

A cloud-init file for a simple web server (this is the file I validated):

#cloud-config
hostname: web1
package_update: true
package_upgrade: true
packages:
  - nginx
  - unattended-upgrades
ssh_pwauth: false
write_files:
  - path: /var/www/html/index.html
    owner: root:root
    permissions: "0644"
    content: |
      <h1>web1 is up</h1>
runcmd:
  - systemctl enable --now nginx

What it does: sets the host name, updates and upgrades the packages, installs nginx and unattended-upgrades (automatic security updates), turns off SSH password logins, writes a small web page with a file mode set on purpose, and enables the service.

Check it before you launch

A mistake in user data is discovered after the instance has booted, minutes later, in a log. Cloud-init can check the file first, offline. This ran for real, on the good file and on a deliberately broken one:

user@lab:~$ cloud-init --version
26.1-0ubuntu1~24.04.1
user@lab:~$ cloud-init schema --config-file user-data.yaml
Valid schema user-data.yaml
user@lab:~$ echo $?
0
user@lab:~$ cloud-init schema --config-file bad-user-data.yaml
Error: Cloud config schema errors: package_updat: Additional properties are not allowed ('package_updat' was unexpected), packages: 'nginx' is not of type 'array', write_files.0.permissions: 644 is not of type 'string'

Error: Invalid schema: user-data

Cloud config schema deprecations: ssh_pwauth:  Changed in version 22.3. Use of non-boolean values for this field is deprecated.
Invalid user-data bad-user-data.yaml
user@lab:~$ echo $?
1

The valid file prints "Valid schema" and exits 0. The broken file (a misspelled key package_updat, packages: nginx as a string instead of a list, and permissions: 644 as a number) was caught: the validator names all three problems, says "Invalid schema" and exits 1. It also warned that ssh_pwauth: "no" (the string) is deprecated: use a real boolean. (Without root, cloud-init printed warnings about files it could not read on this machine; they are not about the user data and are left out.) Run cloud-init schema --config-file FILE on every cloud-config before you paste it into a launch, and keep the file in version control.

Using it at launch (from the documentation, not run here):

aws ec2 run-instances --image-id AMI_ID --instance-type t3.micro --key-name lab-key \
    --subnet-id SUBNET_ID --security-group-ids SG_ID --user-data file://user-data.yaml \
    --iam-instance-profile Name=web-role --metadata-options HttpTokens=required \
    --tag-specifications 'ResourceType=instance,Tags=[{Key=Name,Value=web1}]'

(--user-data file://... is the pattern in AWS's documentation. The other options are the ones this page recommends: check aws ec2 run-instances help for the exact current syntax.)

4. An identity for the server

If the server needs to read an S3 bucket or write to a queue, give it an IAM role through an instance profile, and the AWS SDKs and CLI on the machine find temporary credentials by themselves. No access keys on the server. The IAM page shows the trust policy and the permissions policy, and says why: a key left on a machine is a key that can leak, a role's credentials expire.

5. The metadata service: require IMDSv2

Every instance can ask a local address, 169.254.169.254, about itself (its ID, its role credentials, its user data). There are two versions, and AWS's documentation explains the difference:

• IMDSv1: a plain request and response.
• IMDSv2: session-oriented. The program first sends a PUT request to get a token (valid for up to six hours), then sends it with each GET. A request without a valid token gets 401 Unauthorized when tokens are required. The token is specific to the instance, and by default the response has an IP hop limit of 1, so it does not travel beyond the machine.

By default an instance accepts both. You can configure it to accept only IMDSv2, so that v1 calls fail, and AWS's page links to its security blog on "defense in depth against open firewalls, reverse proxies and SSRF vulnerabilities": a flaw in a web application that lets an attacker make the server fetch a URL could otherwise be used to read the role credentials from the metadata address with one plain GET. The documented way to ask for it at launch is the metadata options (HttpTokens=required above), or modify-instance-metadata-options on a running instance. Getting a value from the instance, with IMDSv2, from AWS's examples (run on the instance, not here):

TOKEN=`curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"`
curl -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-id

One catch from the same page: programs need a recent AWS SDK to speak IMDSv2 (all the latest ones do), and containers on the instance may need a larger hop limit.

6. Backups: snapshots

AWS does not back up the data on your EBS volumes for you. The documentation puts it in an "Important" box: it is your responsibility to take snapshots regularly, or to automate them with Data Lifecycle Manager or AWS Backup. What a snapshot is:

• A point-in-time copy of an EBS volume, and incremental: after the first, only changed blocks are saved.
• Stored in S3 that you cannot access directly, and replicated across the zones of the region, so you can restore into any zone of that region.
• Restoring means creating a new volume from the snapshot (it begins as an exact replica).
• You pay for the data stored. Because snapshots are incremental, deleting one may not reduce the cost, since blocks used by other snapshots are kept.

aws ec2 create-snapshot --volume-id VOLUME_ID --description "web1 before upgrade"
aws ec2 describe-snapshots --owner-ids self --output table

(From the documentation, not run here. Find the volume ID with aws ec2 describe-volumes.) Two habits: take a snapshot before a risky change, and test a restore once (create a volume from a snapshot, attach it to a test instance, and look at the files): the same rule as the ZFS backups page, where the restore test was the point. And remember that a snapshot protects the disk, not the design: if you can rebuild the server from your user data file and your code, you lose nothing by losing it.

7. The way out: stop, terminate, clean up

From AWS's documentation on stopping instances:

• Stop an instance from the console, CLI or API, not only from inside the system (a stop that way is clean and is recorded in CloudTrail).
• A stopped instance is not deleted: its EBS disks stay (and bill), and when you start it again it is typically given a new public IPv4 address.
• Data on instance store volumes is erased when you stop, so do not keep anything there that you need.
• To remove it for good, terminate it.
• Public IPv4 addresses are billed by the hour (since February 2024: about $0.005 per address per hour) whether the instance is busy or not, and an idle Elastic IP is billed too (the services map has the details).
• To find forgotten instances in every region at once, use EC2 Global View.

aws ec2 stop-instances --instance-ids i-0123456789abcdef0
aws ec2 terminate-instances --instance-ids i-0123456789abcdef0
aws ec2 describe-snapshots --owner-ids self --output table

Clean-up checklist: terminate the instance, delete the volumes that stay (a volume is deleted with the instance only if it was set to), delete snapshots you do not need, release Elastic IPs, delete the security group and the key pair if the project is over, and check Cost Explorer the next day. For a repeatable lab, build the whole thing with Terraform and use terraform destroy.

After the first login

A short list for a server that will stay, in the spirit of the site's Hardening tab (hardening):

• Patch. The user data above upgrades once and installs automatic security updates: also schedule a reboot after kernel updates, and know how you will find out a server is behind.
• No password logins (done above) and no root login over SSH.
• Keep the security group tight, and review it when something changes.
• Send logs somewhere that survives the server (the CloudWatch agent, or your own collector), and watch with the ideas of the monitoring page.
• Do not store long-lived credentials on it: use the role.
• Write down the instance's purpose, its owner and the date to review it (tags are a good place: Name, owner, project).

Common mistakes

• Port 22 open to 0.0.0.0/0 (the commonest, and AWS names it in its own best practices).
• Losing the private key with no other way in (set up Session Manager, or a second key, early).
• A cloud-config file with a typo that you find out about 10 minutes after launch (run the validator).
• Secrets in user data.
• Leaving IMDSv1 on for a server that runs a web application.
• No snapshots, or snapshots never tested.
• Believing stop is free: disks and addresses keep billing.
• Forgetting what you launched, in a region you no longer look at.
• Making an AMI from a server whose user data script is still in /var/lib/cloud/instances/.

What this page did not cover

Choosing the instance type and the AMI in depth, Elastic IPs and DNS names, load balancers and Auto Scaling, EBS volume types and encryption settings, Systems Manager in detail, Windows instances, spot instances and savings plans. The services map and the EC2 user guide cover them.

The cheat sheet

• Key pair: make your own (ssh-keygen -t ed25519), import only the .pub (aws ec2 import-key-pair), private key mode 600 (400), a passphrase on a laptop. Login: ssh -i KEY ubuntu@IP (ec2-user on Amazon Linux). Check the effective settings with ssh -G HOST.
• Security group: stateful, allow-only. SSH from your own /32, web from the world, nothing else. Never 0.0.0.0/0 on 22 or 3389.
• User data: cloud-config (#cloud-config) or a script. Runs once, as root, 16 KB, not for secrets. cloud-init schema --config-file FILE before launch. Logs: /var/log/cloud-init-output.log.
• Identity: an instance profile (role), no keys on the server.
• IMDSv2: token with PUT, then GET. Require it: --metadata-options HttpTokens=required.
• Backups: aws ec2 create-snapshot, incremental, you pay for stored data, restore means a new volume, automate with Data Lifecycle Manager or AWS Backup, test a restore.
• Stop and clean up: stop keeps disks and bills for them, new public IP on start. Terminate to remove. Delete volumes, snapshots, Elastic IPs. EC2 Global View finds strays.

← Kubernetes on the Cloud: A Managed Cluster and kubectl · ↑ All tutorials · Cloud Cost Control: Stop Surprises Before They Start →