Terraform and AWS credentials handling?
If you are new to terraform or maybe you have been using the terraform for quite some time then might have felt the need to manage your AWS Credentials inside your terraform file.
In this blog we will go through the different ways to set up your AWS Credentials so that you do not have to re-configure the AWS Credentials every time, also we will pay attention what are the most secure way to handle your AWS Credentials
Here are three different ways of handling AWS Credentials -
- Hard coding AWS Credentials(ACCESS_KEY, SECRET_KEY) inside terraform file (*Not recommended)
- Using /.aws/credentials file along with terraform's shared_credentials_file settings
- Configure AWS Credentials as environment variables
AWS Credentials always consists of aws_access_key_id, aws_secret_access_key and same credentials are used by Terraform to communicate with Amazon Cloud.
If you do not store those credentials securely then you might compromise the security of your infrastructure by exposing AWS Root Account credentials. So it is always recommended to follow good security practices for handling the credentials. In this guide, we will go through all the possible ways to handle your AWS Credentials but also what are the recommended ways to handle it.
1. Hard coding AWS Credentials(ACCESS_KEY, SECRET_KEY) inside terraform file (*Not recommended)
The easiest way to use the AWS credentials is to hard code the AWS's access_key and secret_key inside your terraform file.
(It is not recommended to hard code the credentials inside terraform file)
Here is an Example-
1provider "aws" {
2 profile = "test"
3 region = "eu-central-1"
4 access_key = "A0LPA99DFKJ74HW0CPAGH0FNM3"
5 secret_key = "a1toUHEBHSU49348JjdssNicCwcT61DfzHtHtkWJYhgnN"
6}
Problem with the above approach
- Your AWS Credentials are stored in plain text format
- If you are using version control(GitHub, BitBucket, etc..) then your AWS credentials might end up in a public code repository where anyone can use those credentials and can impose a great risk to your cloud infrastructure.
- Hard-coding credentials in one place make it hard to re-use those credentials in other terraform projects or other terraform modules. So if you are planning to create another terraform project using the same credentials then you can not re-use the same piece of code.
When to Hard Code AWS Credentials
- If you are doing some development work and your terraform code is only accessible within your local development environment.
- Doing some POC(Proof of Concept) tasks with terraform limiting it only to the local development environment.
2. Using /.aws/credentials file along with terraform's shared_credentials_file settings
The second approach would be to store your AWS Credentials inside /.aws/credentials
file. You need to install
aws cli before you start storing those credentials inside /.aws/credentials. Refer to this aws cli
installation guide so that you have /.aws/credentials created onto your machine.
After installing the aws cli you will have file credentials
present at /.aws/credetials
Here is screenshot of /.aws/credetials-
Use terraform's shared_credentials_file settings
Now "How to make terraform to read AWS Credentials file?" - For that, we are going to use terraform setting
shared_credentials_file
. This setting allows you to refer /.aws/credentials present in your machine after
installing the AWS CLI.
Example-
1provider "aws" {
2 profile = "test"
3 region = "eu-central-1"
4 shared_credentials_file = "/home/vagrant/.aws/credentials"
5}
Now you can compare both the screenshot of /.aws/credentials and the example code snippet where it is referring to the same /.aws/credentials file.
Pros of using /.aws/credentials and shared_credentials_file
- You can re-use the AWS Credentials with other terraform projects or terraform modules
- You do not need to repeat or hard code AWS Credentials everywhere
- AWS credentials are stored at the machine level, so they will not end up in the public code repository
- If you limit the access of /.aws/credentials to root user then you have improved the security over the credentials sharing
Cons
- If you do not set permission over the /.aws/credentials file then anyone with access to the machine can view the AWS credentials.
3. Configure AWS Credentials as environment variables
The third option to handle the AWS Credentials is to export the variables as environment variables.
How to Export? -
1export AWS_ACCESS_KEY_ID="A0LPA509NDFKJ74HW0CPAGH0FNM3"
1export AWS_SECRET_ACCESS_KEY="a1toUHEBHSU404823JjdssNicCwcT61DfzHtHtkWJYhgnN"
Run the above command to export AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY variables into the environment variables. After exporting the above variables you do not need to hard code or refer to /.aws/credentials file.
You can simply write you are terraforming configuration and terraform will fetch the AWS Credentials from the environment variables
Here is a sample Terraform code for EC2 using environment variables-
1terraform {
2 required_providers {
3 aws = {
4 source = "hashicorp/aws"
5 version = "3.74.0"
6 }
7 }
8}
9
10provider "aws" {
11 region = "eu-central-1"
12}
13
14resource "aws_instance" "ec2_example" {
15
16 ami = "ami-0767046d1677be5a0"
17 instance_type = "t2.micro"
18
19 tags = {
20 Name = "Terraform EC2"
21 }
22}
If you look carefully in the provider block then we have only defined region
while the credentials are being
fetched from environment variables.
Benefits-
- The AWS Credentials are separate from the code.
- Development team can change or update the AWS Credentials externally without the need to change the code
- The credentials are stored locally at the machine level, so they will not end up in a public code repository with broader access.
- Reducing the risk of messing up environment variables in production, stage, or test
Cons
- Anyone with access to the machine can update those environment variables.
- If the machine needs re-installation then you have to maintain the backup of environment variables otherwise you might end up losing those environment variables.
4.Conclusion
Handling the AWS Credentials should be on the top of the list for improving infrastructure security. Although /.aws/credentials file as well as environment variables will improve the security of AWS Credentials but still it is not sufficient from the production's point of view. The most preferred and secure way would be to dynamically generate short-lived AWS credentials and store them inside the credentials vault for better management. I will write a detailed post on generating AWS credentials dynamically using the Hashicorp vault soon.
Update - Here is the detailed blog post on - Securing AWS secrets using HashiCorp Vault with Terraform
Read More - Terragrunt -
Posts in this Series
- Securing Sensitive Data in Terraform
- Boost Your AWS Security with Terraform : A Step-by-Step Guide
- How to Load Input Data from a File in Terraform?
- Can Terraform be used to provision on-premises infrastructure?
- Fixing the Terraform Error creating IAM Role. MalformedPolicyDocument Has prohibited field Resource
- In terraform how to handle null value with default value?
- Terraform use module output variables as inputs for another module?
- How to Reference a Resource Created by a Terraform Module?
- Understanding Terraform Escape Sequences
- How to fix private-dns-enabled cannot be set because there is already a conflicting DNS domain?
- Use Terraform to manage AWS IAM Policies, Roles and Users
- How to split Your Terraform main.tf File into Multiple Files
- How to use Terraform variable within variable
- Mastering the Terraform Lookup Function for Dynamic Keys
- Copy files to EC2 and S3 bucket using Terraform
- Troubleshooting Error creating EC2 Subnet InvalidSubnet Range The CIDR is Invalid
- Troubleshooting InvalidParameter Security group and subnet belong to different networks
- Managing strings in Terraform: A comprehensive guide
- How to use terraform depends_on meta argument?
- What is user_data in Terraform?
- Why you should not store terraform state file(.tfstate) inside Git Repository?
- How to import existing resource using terraform import comand?
- Terraform - A detailed guide on setting up ALB(Application Load Balancer) and SSL?
- Testing Infrastructure as Code with Terraform?
- How to remove a resource from Terraform state?
- What is Terraform null Resource?
- In terraform how to skip creation of resource if the resource already exist?
- How to setup Virtual machine on Google Cloud Platform
- How to use Terraform locals?
- Terraform Guide - Docker Containers & AWS ECR(elastic container registry)?
- How to generate SSH key in Terraform using tls_private_key?
- How to fix-Terraform Error acquiring the state lock ConditionalCheckFiledException?
- Terraform Template - A complete guide?
- How to use Terragrunt?
- Terraform and AWS Multi account Setup?
- Terraform and AWS credentials handling?
- How to fix-error configuring S3 Backend no valid credential sources for S3 Backend found?
- Terraform state locking using DynamoDB (aws_dynamodb_table)?
- Managing Terraform states?
- Securing AWS secrets using HashiCorp Vault with Terraform?
- How to use Workspaces in Terraform?
- How to run specific terraform resource, module, target?
- How Terraform modules works?
- Secure AWS EC2s & GCP VMs with Terraform SSH Keys!
- What is terraform provisioner?
- Is terraform destroy needed before terraform apply?
- How to fix terraform error Your query returned no results. Please change your search criteria and try again?
- How to use Terraform Data sources?
- How to use Terraform resource meta arguments?
- How to use Terraform Dynamic blocks?
- Terraform - How to nuke AWS resources and save additional AWS infrastructure cost?
- Understanding terraform count, for_each and for loop?
- How to use Terraform output values?
- How to fix error configuring Terraform AWS Provider error validating provider credentials error calling sts GetCallerIdentity SignatureDoesNotMatch?
- How to fix Invalid function argument on line in provider credentials file google Invalid value for path parameter no file exists
- How to fix error value for undeclared variable a variable named was assigned on the command line?
- What is variable.tf and terraform.tfvars?
- How to use Terraform Variables - Locals,Input,Output
- Terraform create EC2 Instance on AWS
- How to fix Error creating service account googleapi Error 403 Identity and Access Management (IAM) API has not been used in project before or it is disabled
- Install terraform on Ubuntu 20.04, CentOS 8, MacOS, Windows 10, Fedora 33, Red hat 8 and Solaris 11