Secure AWS EC2s & GCP VMs with Terraform SSH Keys!

This blog post will help you to understand -

1. How to manage SSH Keys for your EC2 Instance on AWS
2. How to manage SSH Keys for Virtual Machine on GCP

When you work on cloud services such as AWS, Google Cloud you use Terraform to setup EC2 and VMs. But to access those instances(EC2, VMS) you need to setup the SSH Keys.

As all the cloud provider does not provide password for the root user. You have to use SSH key-pair(public key, private key) for authentication.

In this blog post, we will go through the following steps -

(Note - Step-1: Generation of key-pair is common for both the cloud environment)

AWS

1. Generating SSH key-pairs(public key, private key) using ssh keygen
2. Use public key to setup EC2 instance
3. Use the private key to SSH into EC2 instance
4. Implementing Security Groups for EC2
5. Generate pem file from AWS console and use the file to SSH into EC2
   • 5.1 Verify the EC2 instance

GCP(Google Cloud)

6. Setup Google Provider
   • 6.1 Generating SSH Key Pairs
   • 6.2 Provisioning Google Compute Instances along with metadata SSH Keys
   • 6.3 Setup google_compute_firewall rule to allow ssh
   • 6.4 Initialize and apply terraform configuration
7. Setup google_compute_instance with metadata ssh-keys
   

(Note - The first 3 steps in which we are going to generate the ssh key-pair manually and in the 4th step we will generate the key pair from AWS console.If you do not want to generate the SSH keys manually then you can refer to this blog post where I described on how to use terraform's tls_private_key module to generate and upload private, public key for AWS)

1. Generating SSH key-pairs(public key, private key) using ssh keygen

1.1 Generate the public key and private key

Before you start playing with AWS console and terraform script we need to first generate the key-pair(public key, private key) using ssh-keygen.

Later we are going to associate both public and private keys with AWS EC2 instances.

Let us generate the key pair using the following command

1ssh-keygen -t rsa -b 2048 

By default, the above command will generate the public as well as private key at location '/home//.ssh'

But we can override the end destination with a custom path. (I have assigned my custom path /home/rahul/Jhooq/keys/aws followed my key name .i.e. aws_key )

Here is the output along with a screenshot my terminal-

 1Generating public/private RSA key pair.
 2Enter file in which to save the key (/home/rahul/.ssh/id_rsa): /home/rahul/Jhooq/keys/aws/aws_key 
 3Enter passphrase (empty for no passphrase): 
 4Enter the same passphrase again: 
 5Your identification has been saved in /home/rahul/Jhooq/keys/aws/aws_key
 6Your public key has been saved in /home/rahul/Jhooq/keys/aws/aws_key.pub
 7The key fingerprint is:
 8SHA256:sAOjXyvJc2gnMrvxXA+qiaU9pUEvwl5ZG9Y2kZqRf5M rahul@rahul-HP-ZBook-15-G2
 9The key's randomart image is:
10+---[RSA 2048]----+
11|      . .        |
12|     o o         |
13|    o B . .      |
14|  .. O B E       |
15|....+ B S .      |
16|..o=o= o         |
17|..*=X *          |
18| *oX O o         |
19|o *++   .        |
20+----[SHA256]-----+ 

1.2 Verify the generated public key and private key

In the previous step, we have generated the key-pair which we are going to use for provisioning the EC2 instance. But let us take a look at the keys and how it looks.

If you remember in the previous step we have generated the keys at path /home/rahul/Jhooq/keys/aws we should see two key files over there -

1. aws_key (private key)
2. aws_key.pub (public key)

We are going to use public key aws_key.pub inside the terraform file to provision/start the ec2 instance.

1.2.1 public key aws_key.pub

Here is the content of the public key aws_key.pub(you can open the file in any editor of your choice) -

1ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbvRN/gvQBhFe+dE8p3Q865T/xTKgjqTjj56p1IIKbq8SDyOybE8ia0rMPcBLAKds+wjePIYpTtRxT9UsUbZJTgF+SGSG2dC6+ohCQpi6F3xM7ryL9fy3BNCT5aPrwbR862jcOIfv7R1xVfH8OS0WZa8DpVy5kTeutsuH5FMAmEgba4KhYLTzIdhM7UKJvNoUMRBaxAqIAThqH9Vt/iR1WpXgazoPw6dyPssa7ye6tUPRipmPTZukfpxcPlsqytXWlXm7R89xAY9OXkdPPVsrQA0XFQnY8aFb9XaZP8cm7EOVRdxMsA1DyWMVZOTjhBwCHfEIGoePAS3jFMqQjGWQd rahul@rahul-HP-ZBook-15-G2

1.2.2 private key aws_key

Here is the content of the private key aws_key(you can open the file in any editor of your choice) -

 1-----BEGIN OPENSSH PRIVATE KEY-----
 2b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABFwAAAAdzc2gtcn
 3NhAAAAAwEAAQAAAQEA270Tf4L0AYRXvnRPKd0POuU/8UyoI6k44+eqdSCCm6vEg8jsmxPI
 4mtKzD3ASwCnbPsI3jyGKU7UcU/VLFG2SU4BfkhkhtnQuvqIQkKYuhd8TO68i/X8twTQk+W
 5j68G0fOto3DiH7+0dcVXx/DktFmWvA6VcuZE3rrbLh+RTAJhIG2uCoWC08yHYTO1CibzaF
 6DEQWsQKiAE4ah/Vbf4kdVqV4Gs6D8Oncj7LGu8nurVD0YqZj02bpH6cXD5bKsrV1pV5u0f
 7PcQGPTl5HTz1bK0ANFxUJ2PGhW/V2mT/HJuxDlUXcTLANQ8ljFWTk44QcAh3xCBqHjwEt4
 8xTKkIxlkHQAAA9CsiPUKrIj1CgAAAAdzc2gtcnNhAAABAQDbvRN/gvQBhFe+dE8p3Q865T
 9/xTKgjqTjj56p1IIKbq8SDyOybE8ia0rMPcBLAKds+wjePIYpTtRxT9UsUbZJTgF+SGSG2
10dC6+ohCQpi6F3xM7ryL9fy3BNCT5aPrwbR862jcOIfv7R1xVfH8OS0WZa8DpVy5kTeutsu
11H5FMAmEgba4KhYLTzIdhM7UKJvNoUMRBaxAqIAThqH9Vt/iR1WpXgazoPw6dyPssa7ye6t
12UPRipmPTZukfpxcPlsqytXWlXm7R89xAY9OXkdPPVsrQA0XFQnY8aFb9XaZP8cm7EOVRdx
13MsA1DyWMVZOTjhBwCHfEIGoePAS3jFMqQjGWQdAAAAAwEAAQAAAQB83JvPwSHWGtWhK4Yw
14S6Tz2oDTJLQGT4o8Ns/tbmPJAXnRSMLp+/vpvgBxrUV6XE5xAvt/IZfwqOFH9AKNwRV8zV
152BLzaw7qQBPyYai9Ozzmana4V+dl4RgwffkX/GTruIPac7KKR+zLXy/aNVBACwhUJBVYDP
16Dlf1g8hUOS5WcqpRpdzf3PWjAosC2sZUNtoU9wmMTOQMb96haWgdqQrdulzhBxMb7/hJQ5
17q11gGdTboFM2l8yKMbRsba1OG/0JR2cmSNoK2PDL8GxskolwveGjvzsYPUS0iQsvjqAWQX
18F3cW7lBq0eL23qkLFxAJxFQU4mB9yH5KCunU0K9am18hAAAAgAxkrreETLmI0snIX3uX+T
19ySTNTb0G9kya0G3TGpkjvZT8Ump/L8ysleidqK2VYVEPre6vfTECqzWhBL1BVT/howuacG
205LeEkgS/F8oQJ19iEKT4LXxTumlFJqIj7+9jjnD5Z+gxi/ttJVTWvN/L4Ho9ZbZ6SoUBIB
21hnb+tDP0+mAAAAgQDwFBx6E7dARqwOeN8byrmzsYmnr9a2jx4eXR1iDX7ZzPLjYoOCFiJP
223ahHhXSzPx5pJK4k8MRYUFNTjtl2K+cKgCZC75Dj5mfI/8l0pvmUrZ2e2GN1+ATUcQO7Dy
230j8124Bke0OkItBTfwQrsIOSyYzySOll88odoISNB2BdZsewAAAIEA6k+ljxLX1sfn8wOg
24l9zgXqoT64tiQIJ720WSHuc5xoslgHVdjeunVy1eAaZMDURn8sbedlGLaebaGdVZwidR04
25gpodkAZdkBm+tju2NRIxIhuDU02ddJFEGy/8lp+XqEm+YfbUpHrCSNOjYBDkdAp6umQFVq
26jQi2RBtpIsNFikcAAAAacmFodWxAcmFodWwtSFAtWkJvb2stMTUtRzIB
27-----END OPENSSH PRIVATE KEY-----

2. Use public key to setup EC2 instance

Alright now we have the public key and the private key with us, let us create our terraform configuration file using the public key .i.e. aws_key.pub

(*Note - If this is your first time with terraform and you have not worked before then I would recommend you to go through an article on How to install terraform and How to create EC2 instance on AWS)

Here is the main.tf -

 1provider "aws" {
 2   region     = "eu-central-1"
 3   access_key = ""
 4   secret_key = ""
 5   
 6}
 7
 8resource "aws_instance" "ec2_example" {
 9
10    ami = "ami-0767046d1677be5a0"  
11    instance_type = "t2.micro" 
12    key_name= "aws_key"
13    vpc_security_group_ids = [aws_security_group.main.id]
14
15  provisioner "remote-exec" {
16    inline = [
17      "touch hello.txt",
18      "echo helloworld remote provisioner >> hello.txt",
19    ]
20  }
21  connection {
22      type        = "ssh"
23      host        = self.public_ip
24      user        = "ubuntu"
25      private_key = file("/home/rahul/Jhooq/keys/aws/aws_key")
26      timeout     = "4m"
27   }
28}
29
30resource "aws_security_group" "main" {
31  egress = [
32    {
33      cidr_blocks      = [ "0.0.0.0/0", ]
34      description      = ""
35      from_port        = 0
36      ipv6_cidr_blocks = []
37      prefix_list_ids  = []
38      protocol         = "-1"
39      security_groups  = []
40      self             = false
41      to_port          = 0
42    }
43  ]
44 ingress                = [
45   {
46     cidr_blocks      = [ "0.0.0.0/0", ]
47     description      = ""
48     from_port        = 22
49     ipv6_cidr_blocks = []
50     prefix_list_ids  = []
51     protocol         = "tcp"
52     security_groups  = []
53     self             = false
54     to_port          = 22
55  }
56  ]
57}
58
59
60resource "aws_key_pair" "deployer" {
61  key_name   = "aws_key"
62  public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbvRN/gvQBhFe+dE8p3Q865T/xTKgjqTjj56p1IIKbq8SDyOybE8ia0rMPcBLAKds+wjePIYpTtRxT9UsUbZJTgF+SGSG2dC6+ohCQpi6F3xM7ryL9fy3BNCT5aPrwbR862jcOIfv7R1xVfH8OS0WZa8DpVy5kTeutsuH5suehdngba4KhYLTzIdhM7UKJvNoUMRBaxAqIAThqH9Vt/iR1WpXgazoPw6dyPssa7ye6tUPRipmPTZukfpxcPlsqytXWlXm7R89xAY9OXkdPPVsrQdkdfhnY8aFb9XaZP8cm7EOVRdxMsA1DyWMVZOTjhBwCHfEIGoePAS3jFMqQjGWQd rahul@rahul-HP-ZBook-15-G2"
63}

You can first verify the terraform configuration using the terraform plan and then finally you can apply it using terraform apply.

After applying the configuration you can verify the instance by going into the AWS console -

3. Use private key 'aws_key' to SSH into EC2 instance

In the previous step, we have started the EC2 instance, now we need to connect to EC2 instance using the private key.

You can find the connect command from the aws console -

Here is the SSH connect command for your reference-

(By default in the command you will see .pem extension in the private key file name but since we have created private with the name aws_key so we need to remove .pem extension from the file)

1ssh -i "aws_key" ubuntu@ec2-18-185-22-181.eu-central-1.compute.amazonaws.com

4. Implementing Security Groups for EC2

AWS EC2 instances have a virtual fence called a security group that controls data coming in and going out. To keep unwanted people from getting into your instances, you must set up your security groups correctly. In this case, we'll make a security group that lets SSH access to your computer.

Along with ssh-keys you also need to set-up the Security Groups so that SSH Port(22) is open for the EC2 instance.

 1# Security Group for EC2
 2
 3resource "aws_security_group" "main" {
 4  egress = [
 5    {
 6      cidr_blocks      = [ "0.0.0.0/0", ]
 7      description      = ""
 8      from_port        = 0
 9      ipv6_cidr_blocks = []
10      prefix_list_ids  = []
11      protocol         = "-1"
12      security_groups  = []
13      self             = false
14      to_port          = 0
15    }
16  ]
17
18# Port 22 for SSH
19 
20 ingress                = [
21   {
22     cidr_blocks      = [ "0.0.0.0/0", ]
23     description      = ""
24     from_port        = 22
25     ipv6_cidr_blocks = []
26     prefix_list_ids  = []
27     protocol         = "tcp"
28     security_groups  = []
29     self             = false
30     to_port          = 22
31  }
32  ]
33}
34 

By implementing this security group, you enable robust protection for your AWS EC2 instance while still permitting secure SSH access. Remember to adjust the cidr_blocks attribute to limit access to specific IP addresses or ranges as needed.

5. Generate pem file from AWS console and use the file to SSH into EC2

In this step, we are going to AWS's key pair utility to generate the keys for us.

Login to your AWS console and in the search bar type is key pair

Click on the Create Key pair

Now you need to supply the key name. You can choose the private key format .pem

As you will click on Create key pair you will be able to download the key and save it somewhere onto your disk.

Now you can simply use the key aws_key.pem by specifying the key name inside your terraform file.

Here is an example of terraform script -

 1provider "aws" {
 2   region     = "eu-central-1"
 3   access_key = "AKIATQ37NXB2BYDxxxxx"
 4   secret_key = "JzZKiCia2vjbq4zGGGewdbOhnacm2QIMgcBxxxxx"
 5   
 6}
 7
 8resource "aws_instance" "ec2_example" {
 9
10    ami = "ami-0767046d1677be5a0"  
11    instance_type = "t2.micro" 
12    key_name= "aws_key"
13    vpc_security_group_ids = [aws_security_group.main.id]
14
15}
16
17resource "aws_security_group" "main" {
18  egress = [
19    {
20      cidr_blocks      = [ "0.0.0.0/0", ]
21      description      = ""
22      from_port        = 0
23      ipv6_cidr_blocks = []
24      prefix_list_ids  = []
25      protocol         = "-1"
26      security_groups  = []
27      self             = false
28      to_port          = 0
29    }
30  ]
31 ingress                = [
32   {
33     cidr_blocks      = [ "0.0.0.0/0", ]
34     description      = ""
35     from_port        = 22
36     ipv6_cidr_blocks = []
37     prefix_list_ids  = []
38     protocol         = "tcp"
39     security_groups  = []
40     self             = false
41     to_port          = 22
42  }
43  ]
44}

Few points to pay attention for -

1. You need to open port 22 for SSH
2. For opening port 22 you need to create a security group and attach the security group to aws instance

Apply the Terraform configuration You can simply run the following commands in the same sequence one after another -

1. terraform init
2. terraform plan
3. terraform apply

5.1 Verify the EC2 instance

After provisioning/starting the EC2 instance, we need to verify two things -

1. EC2 instance running state
2. SSH into EC2 instance using the aws_key.pem key

1. Verify EC2 instance running state

Go to your AWS console and look for the number of EC2 instances running. If you successfully started your EC2 instance then you will following under the EC2 dashboard of AWS -

2. SSH into EC2 instance using the aws_key.pem key

So till now we have created the key as well started the AWS ec2 instance which is also up and running.

Now we need to ssh into the ec2 instance so that we can verify that the keys which we have generated are working fine.

Let's first goto the AWS console and select the EC2 instance which we have created. Then click on connect.

After you click on the connection you will be prompted with different options to connect -

1. EC2 Instance connect
2. Session Manager
3. SSH Client
4. EC2 Serial Console

We are going to choose the option SSH Client

Open your terminal and go to the location where you have saved aws_key.pem key file. Because we need that private key file to connect with the EC2 instance.

Run the following command to change the permission of the file -

1chmod 400 aws_key.pem

Now use the following command to connect with the EC2 instance -

1ssh -i "aws_key.pem" ubuntu@ec2-52-58-111-83.eu-central-1.compute.amazonaws.com

After making the successful SSH connection you should see the following on you terminal -

Hope this blog post helps you to solve the issue Can't SSH into EC2 instance created with Terraform.

6. The Ultimate SSH Key Setup for Google Compute

Google Cloud Compute Engine is an Infrastructure as a Service (IaaS) offering that provides scalable and flexible virtual machines (VMs) in the cloud.

Secure Shell (SSH) is a network system that lets a client and a server talk to each other in a safe way. By using SSH along with Google Cloud Compute Engine, you can view and control your cloud resources in a safe way.

Here are the steps which you need to follow -

6.1 Generating SSH Key Pairs

For generating the SSH Key pair please refer to the Step-1 in this blog post. Once you complete the SSH Key generation then you will end up with two SSH Keys -

• Public Key
• Private key

6.2 Provisioning Google Compute Instances along with metadata SSH Keys

Once you have generated your SSH keys, you can provision Google Compute instances using the following terraform code -

 1# Provision the compute instance
 2
 3resource "google_compute_instance" "terraform-managed-e2-small" {
 4  name         = "terraform"
 5  machine_type = "e2-small"
 6  boot_disk {
 7    initialize_params {
 8      image = "debian-cloud/debian-11"
 9      #image = "centos-cloud/centos-stream-8"
10    }
11  }
12
13  network_interface {
14    network = "default"
15    access_config {
16    }
17  }
18  
19  # Public Key : Please replace following string with your public key
20  metadata = {
21    ssh-keys = "rahulwagh:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDwmgMHFJE7J4qepIzAZL3/yC6J0zsEAb/oHYL+WBBDNUjSH4TeHUnHVNe9b/pyPcub+O/HNvlGrzSxUp0xT0b3O7kkTtgBKiG8NaBbonj+c7byfOGER80DYxc5adlBltuIDd8StFe7OMzbYyUSr1mdxDTIWm/OoE39G/fu3hTqUGkykv072GAy8nMFejITRw9pf+53B9ziE5rsdOUH4uqBiQa/Ng/qKo7h9MtJGcloRATYiObXwAgrHtt3sDrtvkq2ZceT906/BJm1Twlm+BHlQecHV18Ak3bzm/6HzlsA/q+yORsoB+VxSUxvVy0nXTc1X8vJAD4KSYVL5DTrpisdnQAIcuqAbea+LMku2o4sdnrrIlUi8/8BXeVbI4TNNGd0+sWpCVcDEhb4gyA/XXTvloQyjTYrL4+am/9XEY6NGdsrPK74sjvtpUZPUrmzTJ/mJWG5ncGY88GAj+YZAsY5pnAqh2CkR2TUpglugldnWyrppbe2QyC9iQkgUGSkBTs= rahulwagh@Rahuls-MacBook-Pro.local"
22  }
23}

6.3 Setup google_compute_firewall rule to allow ssh

Along with setting up the google_compute_instance and SSH Keys you also need to enable the firewall rule so that you can allow the SSH on port 22.

Use the following Terraform code to setup the google_compute_firewall rule -

 1resource "google_compute_firewall" "allow_ssh" {
 2  name    = "allow-ssh"
 3  network = "default"
 4
 5  allow {
 6    protocol = "tcp"
 7    ports    = ["22"]
 8  }
 9
10  source_ranges = ["0.0.0.0/0"]
11} 

This code creates a firewall rule named allow-ssh that allows inbound traffic on port 22 (the default SSH port) from any IP address (0.0.0.0/0).

You can customize the source_ranges attribute to limit access to specific IP addresses or ranges.

6.4 Initialize and apply terraform configuration

Run the following terraform command to apply the above configuration -

1#Run the following commands 
2
3$ terraform init
4$ terraform apply 

7. Best Practices for Managing SSH Keys with AWS and Google Cloud

Using the best ways to manage SSH keys with AWS and Google Cloud can help make sure that your cloud system is safe and works well. Here are some important things you should do:

1. Limit access to certain IP addresses or ranges: Set up your security groups or firewall rules to limit SSH access to certain IP addresses or ranges. This makes it harder for people who shouldn't be there to get into your instances.

2. Use strong, unique SSH keys: For each server or project, make a strong SSH key with at least 2048 bits. This makes sure that even if one key is stolen, other projects or cases will still be safe.

3. Change your SSH keys on a regular basis: If you change your SSH keys on a regular basis, it will be less likely that a stolen key will be used to get illegal access. Make a plan for changing the keys and stick to it.

4. Store private keys safely: Keep your private SSH keys in a safe place, like a secured folder or a key management system that requires a password. This keeps people from getting into your instances who shouldn't be able to.

5. Limit the number of people who can use SSH: Only give SSH access to the people who need it and limit the number of people who can access your servers. This makes it harder for illegal people to get in.

6. Use a pair of keys for each person, not each instance: Instead of giving each user the same SSH key pair, you should give each person their own key pair. This makes it easy to control access and take it away when necessary.

7. watch and audit SSH activity: You should regularly watch and audit SSH activity on your servers to find security problems or attempts to get in without permission. To keep track of SSH access, you can use cloud tracking and logging services like AWS CloudTrail or Google Cloud Operations Suite.

8. Set up two-factor authentication (2FA): Set up two-factor authentication to make your SSH login more secure. Use tools like Google Authenticator or Duo to make your account even safer.

9. Turn off password authentication: Turn off password authentication for SSH and only use key authentication. This makes security stronger and makes brute-force attacks less likely.

10. Don't stop updating software: Make sure that your instances' software, including the SSH server, is always up to date. This makes security better and helps protect against known flaws.

Read More - Terragrunt -

1. How to use Terragrunt?