Terraform state locking using DynamoDB (aws_dynamodb_table)?
Terraform state is the backbone of your terraform project for provisioning your cloud infrastructure. When you work on large infrastructure provisioning with Terraform then it is always more than one developer or team of a developer working on the same terraform project.
The problems arise when two developers try to update the same terraform state file which is stored remotely(S3 Bucket). Ideally, the update on the state file should be done incrementally so that when one developer finishes pushing its state file changes another developer can push their changes after taking the update.
But because of the agile working environment, we can not guarantee that incremental updates on terraform state files will be performed one after another. Any developer can update and push terraform state file at any point in time, so there should be some provision to prevent a developer from writing or updating terraform file when it is already being used by another developer.
Why Terraform State Locking is important?- It prevents Terraform state file(terraform.tfstate) from accidental updates by putting a lock on file so that the current update can be finished before processing the new change. The feature of Terraform state locking is supported by AWS S3 and Dynamo DB.
In this blog, we will address the topic on How to implement Terraform state file(terraform.tfstate) locking using AWS S3 bucket and DynamoDB?
Table of Content
- How Store Terraform state file remotely on S3?
- Create DynamoDB table on AWS
- Add AWS DynamoDB Table reference to Backend S3 remote state?
- Spin one more EC2 instance with same terraform state file
- Conclusion
1. How Store Terraform state to file remotely on S3?
Before we implement the Dynamo DB locking feature first we need to store the Terraform state file(terraform.tfstate) file remotely on AWS S3 bucket.
I am gonna take a very simple example in which we are going to provision an AWS EC2 machine and store the terraform state file remotely.
Let's start by creating main.tf and we will add the following resource blocks to it -
- Provider Block
- AWS Instance resource block(aws_instance) for EC2
- Backend S3 block
- Execute terraform script
- Verify the remote state file
1.1 Provider Block
As we are working on the AWS environment so we will be using AWS provider. So add the following block to your main. tf -
1provider "aws" {
2 region = "eu-central-1"
3 access_key = var.access_key
4 secret_key = var.secret_key
5}
1.2 AWS Instance resource block(aws_instance) for EC2
After adding the provider block let's add the aws_instance resource block in which we are going to set up the EC2 the machine of type t2.micro -
1provider "aws" {
2 region = "eu-central-1"
3 access_key = var.access_key
4 secret_key = var.secret_key
5}
6
7resource "aws_instance" "ec2_example" {
8 ami = "ami-0767046d1677be5a0"
9 instance_type = "t2.micro"
10 tags = {
11 Name = "EC2 Instance with remote state"
12 }
13}
1.3 Backend S3 block
(*Note - I have already created an S3 bucket with the name jhooq-terraform-s3-bucket, so make sure to create one for you as well.)
Now after adding the provider and aws_instance block let's add the backend S3 block to my main.tf -
1provider "aws" {
2 region = "eu-central-1"
3 access_key = var.access_key
4 secret_key = var.secret_key
5}
6
7resource "aws_dynamodb_table" "state_locking" {
8 hash_key = "LockID"
9 name = "dynamodb-state-locking"
10 attribute {
11 name = "LockID"
12 type = "S"
13 }
14 billing_mode = "PAY_PER_REQUEST"
15}
16
17resource "aws_instance" "ec2_example" {
18 ami = "ami-0767046d1677be5a0"
19 instance_type = "t2.micro"
20 tags = {
21 Name = "EC2 Instance with remote state"
22 }
23}
24
25terraform {
26 backend "s3" {
27 bucket = "jhooq-terraform-s3-bucket"
28 key = "jhooq/terraform/remote/s3/terraform.tfstate"
29 region = "eu-central-1"
30 }
31}
2. Create DynamoDB table on AWS
Now for implementing the state locking we need to create a DynamoDB table.
- Goto your AWS management console and search for DynamoDB onto the search bar.
-
Click on the DynamoDB
-
From the left navigation panel click on Tables
- Click on Create Table
- Enter the Table name - "dynamodb-state-locking" and Partition Key - "LockID"
- Click on Create Table and you can verify the table after the creation
3. Add AWS DynamoDB Table reference to Backend S3 remote state?
After creating the DynamoDB table in the previous step, let's add the reference of DynamoDB table name (dynamodb-state-locking) to backend S3 sate.
1terraform {
2 backend "s3" {
3 bucket = "jhooq-terraform-s3-bucket"
4 key = "jhooq/terraform/remote/s3/terraform.tfstate"
5 region = "eu-central-1"
6 dynamodb_table = "dynamodb-state-locking"
7 }
8}
Your final Terraform main.tf should look like this -
1provider "aws" {
2 region = "eu-central-1"
3 access_key = var.access_key
4 secret_key = var.secret_key
5}
6
7resource "aws_dynamodb_table" "state_locking" {
8 hash_key = "LockID"
9 name = "dynamodb-state-locking"
10 attribute {
11 name = "LockID"
12 type = "S"
13 }
14 billing_mode = "PAY_PER_REQUEST"
15}
16
17resource "aws_instance" "ec2_example" {
18 ami = "ami-0767046d1677be5a0"
19 instance_type = "t2.micro"
20 tags = {
21 Name = "EC2 Instance with remote state"
22 }
23}
24
25terraform {
26 backend "s3" {
27 bucket = "jhooq-terraform-s3-bucket"
28 key = "jhooq/terraform/remote/s3/terraform.tfstate"
29 region = "eu-central-1"
30 dynamodb_table = "dynamodb-state-locking"
31 }
32}
3.1 Apply the above terraform configuration with DynamoDB table
- The first command we are gonna run is terraform init
- Now the run the terraform plan command
- Finally, the terraform apply command
- Verify the DynamoDB LockID by going into the AWS management console -
4. Spin one more EC2 instance with the same Terraform state file
(*Note- To simulate the locking scenario I am creating another main.tf with the same configuration. I would encourage you to create one main.tf and save the file in some other directory)
To test terraform state locking I will provision one more EC2 machine using the same Terraform state file (jhooq/terraform/remote/s3/terraform.tfstate) stored in my S3 bucket along with the same DynamoDB table (dynamodb-state-locking).
Keep in mind we are still using following two components from previous main.tf
- S3 Bucket - jhooq-terraform-s3-bucket
- DynamoDB Table - dynamodb-state-locking
- Terraform state file - jhooq/terraform/remote/s3/terraform.tfstate
Here is my another main.tf file -
1provider "aws" {
2 region = "eu-central-1"
3 access_key = var.access_key
4 secret_key = var.secret_key
5}
6
7resource "aws_instance" "ec2_example" {
8 ami = "ami-0767046d1677be5a0"
9 instance_type = "t2.micro"
10 tags = {
11 Name = "EC2 Instance with remote state"
12 }
13}
14
15terraform {
16 backend "s3" {
17 bucket = "jhooq-terraform-s3-bucket"
18 key = "jhooq/terraform/remote/s3/terraform.tfstate"
19 encrypt = true
20 region = "eu-central-1"
21 dynamodb_table = "dynamodb-state-locking"
22 }
23}
4.1 Run both the terraform files at the same time to simulate the Locking on terraforming state file
On the left side of the screen, you will see the first terraform file(main.tf) which I have created in the Step-1 and on the right-hand side, you will see the terraform file(main.tf) from the Step-4.
**How did I simulate the remote state locking scenario? **
-
I have executed terraform apply on terraform file present on the right-hand side but did not let it finish. While executing terraform apply command I did not type yes when it asks for Do you want to perform these actions? so basically terraform apply command is still running and holding a lock on the remote state file.
-
At the same time I executed the terraform apply on main.tf from Step-4 which you can see on the right side of the screenshot. Since the second main.tf file also referring the same remote state as well as same dynamo db table it will throw en error - Error: Error acquiring the state lock Error message: ConditionalCheckFailedException: The conditional request failed Lock Info ID: 8f014160-8894-868e-529d-0f16e42af405
5. Conclusion
Terraform state file locking is one of the most valuable features offered by terraform for managing the Terraform state file. If you are using the AWS S3 and Dynamo DB then terraform state locking can improve your state management and save your time from unforeseen issues.
Read More - Terragrunt -
Posts in this Series
- Use Terraform to manage AWS IAM Policies, Roles and Users
- How to split Your Terraform main.tf File into Multiple Files
- How to use Terraform variable within variable
- Mastering the Terraform Lookup Function for Dynamic Keys
- Copy files to EC2 and S3 bucket using Terraform
- Troubleshooting Error creating EC2 Subnet InvalidSubnet Range The CIDR is Invalid
- Troubleshooting InvalidParameter Security group and subnet belong to different networks
- Managing strings in Terraform: A comprehensive guide
- How to use terraform depends_on meta argument?
- What is user_data in Terraform?
- Why you should not store terraform state file(.tfstate) inside Git Repository?
- How to import existing resource using terraform import comand?
- Terraform - A detailed guide on setting up ALB(Application Load Balancer) and SSL?
- Testing Infrastructure as Code with Terraform?
- How to remove a resource from Terraform state?
- What is Terraform null Resource?
- In terraform how to skip creation of resource if the resource already exist?
- How to setup Virtual machine on Google Cloud Platform
- How to use Terraform locals?
- Terraform Guide - Docker Containers & AWS ECR(elastic container registry)?
- How to generate SSH key in Terraform using tls_private_key?
- How to fix-Terraform Error acquiring the state lock ConditionalCheckFiledException?
- Terraform Template - A complete guide?
- How to use Terragrunt?
- Terraform and AWS Multi account Setup?
- Terraform and AWS credentials handling?
- How to fix-error configuring S3 Backend no valid credential sources for S3 Backend found?
- Terraform state locking using DynamoDB (aws_dynamodb_table)?
- Managing Terraform states?
- Securing AWS secrets using HashiCorp Vault with Terraform?
- How to use Workspaces in Terraform?
- How to run specific terraform resource, module, target?
- How Terraform modules works?
- Secure AWS EC2s & GCP VMs with Terraform SSH Keys!
- What is terraform provisioner?
- Is terraform destroy needed before terraform apply?
- How to fix terraform error Your query returned no results. Please change your search criteria and try again?
- How to use Terraform Data sources?
- How to use Terraform resource meta arguments?
- How to use Terraform Dynamic blocks?
- Terraform - How to nuke AWS resources and save additional AWS infrastructure cost?
- Understanding terraform count, for_each and for loop?
- How to use Terraform output values?
- How to fix error configuring Terraform AWS Provider error validating provider credentials error calling sts GetCallerIdentity SignatureDoesNotMatch?
- How to fix Invalid function argument on line in provider credentials file google Invalid value for path parameter no file exists
- How to fix error value for undeclared variable a variable named was assigned on the command line?
- What is variable.tf and terraform.tfvars?
- How to use Terraform Variables - Locals,Input,Output
- Terraform create EC2 Instance on AWS
- How to fix Error creating service account googleapi Error 403 Identity and Access Management (IAM) API has not been used in project before or it is disabled
- Install terraform on Ubuntu 20.04, CentOS 8, MacOS, Windows 10, Fedora 33, Red hat 8 and Solaris 11