Terraform State File Explained: terraform.tfstate, Remote State on S3, Locking, Pull and Push

Terraform has its backbone known as the terraform.tfstate file. Any change you make to your infrastructure will have its presence in the terraform.tfstate file. So when you work with Terraform for managing and provisioning your infrastructure, Terraform will always create a terraform.tfstate file for you.

I have been using Terraform since the 0.12 days and I can tell you that 9 out of 10 Terraform problems I have seen in real projects were state file problems - a state file committed to git, two developers applying at the same time, a lost state file, or a state file that no longer matches what is running in the cloud.

So in this blog post we will go through everything about the Terraform state file -

1. What is Terraform State(terraform.tfstate) file?
2. What is the Purpose of Terraform state file?
3. Terraform state file Metadata
4. Terraform State Performance and Caching
5. What is terraform_remote_state Data source?
6. Terraform State Storage, Manual state pull/push and Locking
7. How to store Terraform state file remotely on AWS S3 (step by step)
8. Terraform workspaces and state file
9. Common Terraform state errors and how to fix them
10. Conclusion

1. What is Terraform State(terraform.tfstate) file?

Terraform state file acts as a recorder for your infrastructure setup. Terraform state file(terraform.tfstate) is created from the very beginning when you run your first terraform apply command.

Here is a screenshot from my terraform workspace containing the terraform.tfstate file -

How does Terraform state file store information?

Terraform state file stores the information(metadata) about the infrastructure resources. Any change you plan to make or have already made to your infrastructure, that changed information will be stored inside the Terraform state file.

It is a plain JSON file. You can open it with any text editor, but you should never edit it by hand - we will see the proper commands for that in Section 6.

How to save terraform state file?

1. Saving Locally - If you are the only developer working with Terraform and managing your infrastructure then you do not have to do anything special. The terraform apply command will generate the terraform state file for your workspace and save it inside your working directory.

2. Saving Remotely - If you are working in a team where multiple developers are writing the terraform code to manage the infrastructure, then it is highly recommended to store the terraform state file remotely(AWS S3 Bucket) in a central location, so that with every infrastructure change your terraform state file is up to date and in sync with the others.

My rule of thumb - Local state is only for learning and throwaway experiments. The moment there is a second person, a CI pipeline, or anything you cannot afford to lose, move the state to a remote backend. I have written the exact S3 steps in Section 7.

2. What is the Purpose of Terraform state file?

Any change which you make to your resource will be first reflected into the terraform state file before updating it onto the cloud infrastructure. So the terraform state file will store each action which you take on the terraform resource.

Before you use the terraform apply or terraform destroy command, terraform always validates your resource configuration against the terraform state file, and if the changes seem to be valid then your terraform state file is first going to be updated and then your actual resource will be updated.

So the question comes how can terraform differentiate the multiple resources of the same type running in the cloud environment?

Terraform has its own local database known as Terraform state file where each resource is tagged with its own unique id and this unique id is stored in the state file in the form of metadata.

Here are the three jobs the state file does for you -

1. Mapping - It maps the resource block in your .tf file (aws_instance.ec2_example) to the real resource id in the cloud (i-0abc123...).
2. Dependency tracking - It remembers the order in which resources were created, so that terraform destroy can delete them in the reverse order.
3. Performance - It is a local cache of the attributes of your resources, so terraform plan does not have to query every single resource from the cloud API every time.

3. Terraform state file Metadata

Any information which is stored inside the terraform state file is known as metadata. As we discussed in the previous points all the infrastructure changes are stored in the form of Metadata but terraform also maintains order for managing the Metadata inside the Terraform state file.

Let's take an example -

1. Step-1: You have written a resource block resource "aws_instance" "ec2_example"
2. Step-2: Terraform has generated a terraform.tfstate file to store metadata about ec2_example
3. Step-3: You have added an S3 bucket manually from the AWS Console without making any changes to the terraform file.
4. Step-4: Now you have run the terraform destroy command.
5. End Result: Terraform only deletes the EC2 instance

Conclusion - It can only find the EC2 instance resource metadata in the terraform state file and there is no reference to the S3 bucket because you have created the bucket manually, and the terraform state file has no clue about the S3 Bucket.

This is how Terraform maintains a dependency order for maintaining the infrastructure inventory. It is highly recommended to always provision or update your infrastructure only with terraform. (And if someone did create something by hand, you can bring it under Terraform control with terraform import.)

Here is a sample copy of metadata from my terraform state file -

 1{
 2  "version": 4,
 3  "terraform_version": "1.16.2",
 4  "serial": 1,
 5  "lineage": "b7b8eea8-c581-4c38-85eb-f2c78f1e91c7",
 6  "outputs": {},
 7  "resources": [
 8    {
 9      "mode": "managed",
10      "type": "aws_instance",
11      "name": "ec2_example",
12      "provider": "provider[\"registry.terraform.io/hashicorp/aws\"]",
13      "instances": [
14       ............
15      ]
16    }
17  ]
18}

Let me explain the four fields at the top, because you will see them again when we talk about state push -

• version - The state file format version. It has been 4 for a long time now, you do not need to touch it.
• terraform_version - The Terraform CLI version that last wrote the state. Terraform will refuse to work with a state written by a newer Terraform than the one you are running, so keep the team on the same version.
• serial - A counter which goes up by one every time the state is written.
• lineage - A unique id generated when the state file was first created. Two state files with different lineage are two different infrastructures as far as Terraform is concerned.

4. Terraform State Performance and Caching

The next benefit of the Terraform state file is Caching. As the terraform state file acts as a local database for your terraform code, before making any request to the cloud infrastructure it first checks the local terraform state file to show the current status of the cloud infrastructure.

Imagine you have a really large infrastructure and you wanna make a query to get the status of each resource running in the cloud infrastructure. It could take really long and would not be a very effective way to know the state of your cloud infrastructure.

terraform plan - The terraform plan command is a really good example of caching and information returned from the terraform state file. Whenever we run the terraform plan command it returns 3 pieces of information -

1. Resources to be added
2. Resources to be changed
3. Resources to be destroyed

Here is a sample screenshot of the terraform plan command stating how many resources it is going to add, change and destroy -

Note - By default terraform plan still refreshes the real resources from the cloud before comparing. On a very large project you can skip that with terraform plan -refresh=false to get a much faster plan purely from the cached state. Just remember that it will not notice anything that was changed by hand in the console.

5. What is terraform_remote_state Data source?

Let's assume you are a team of multiple developers working on the same Terraform project to set up and maintain your cloud infrastructure.

Considering the team size all the developers will maintain a single terraform state file so that every change which they make is always in sync with the other teammates.

But is there a way to retrieve the metadata information from the remote state file?

Well yes, you can use the terraform_remote_state data source to retrieve the outputs of another state file. A very common real-world use case is - your network team manages the VPC in one Terraform project, and your application team needs the subnet_id from it in a different Terraform project.

S3 backend -

Here is an example code snippet of terraform_remote_state reading from a state file stored on S3 -

 1data "terraform_remote_state" "network" {
 2  backend = "s3"
 3
 4  config = {
 5    bucket = "jhooq-terraform-state"
 6    key    = "network/terraform.tfstate"
 7    region = "eu-central-1"
 8  }
 9}
10
11resource "aws_instance" "jhooq-test" {
12  ami           = "ami-0a1b2c3d4e5f67890"
13  instance_type = "t3.micro"
14  subnet_id     = data.terraform_remote_state.network.outputs.subnet_id
15}

Local Backend -

 1data "terraform_remote_state" "local_state" {
 2  backend = "local"
 3
 4  config = {
 5    path = "/home/rwagh/terraform/terraform.tfstate"
 6  }
 7}
 8
 9resource "aws_instance" "jhooq-test" {
10  # ...
11  subnet_id = data.terraform_remote_state.local_state.outputs.subnet_id
12}

Important - terraform_remote_state can only read the values which the other project has declared in an output block. If subnet_id is not exported as an output in the network project, you will not be able to read it. Also, whoever runs this code needs read access to the whole state file, which may contain sensitive values - so for a bigger organisation consider reading the values from AWS SSM Parameter Store or a data source instead.

6. Terraform State Storage, Manual state pull/push and Locking

6.1 Storage

Terraform state files can be stored locally as well as remotely, it depends on your need. The storage of the terraform state file is defined by the backend block inside the terraform {} block.

Local Storage - This is the default. If you do not write any backend block, Terraform uses the local backend and saves the state file on your local disk -

1terraform {
2  backend "local" {
3    path = "terraform.tfstate"
4  }
5}

1. When using local storage terraform will persist/save the state file on the local disk
2. If you are using local storage then you have to manually push and pull the Terraform state file (not recommended, we will see why in a moment)

Remote Storage - Here is an example where we have defined the storage as remote on an AWS S3 bucket -

1terraform {
2  backend "s3" {
3    bucket       = "jhooq-terraform-state"
4    key          = "dev/terraform.tfstate"
5    region       = "eu-central-1"
6    encrypt      = true
7    use_lockfile = true
8  }
9}

1. When using remote storage terraform will not persist/save the state file on the local disk.
2. Remote storage of the Terraform state file is really beneficial for teams with multiple developers
3. The remote backend can also provide locking (see 6.3) and versioning, which the local backend cannot.

Apart from S3, the other popular backends are azurerm (Azure Storage), gcs (Google Cloud Storage), kubernetes and the cloud block for HCP Terraform (which was called Terraform Cloud earlier). If you are coming from the old backend "remote" block, the cloud block is its replacement -

1terraform {
2  cloud {
3    organization = "jhooq"
4    workspaces {
5      name = "jhooq-test"
6    }
7  }
8}

6.2 Manual state pull/push

In the previous point, we discussed how to store state files locally and remotely. But apart from storing the state file there are two more important operations -

1. Pull terraform state file
2. Push terraform state file

1. Pull terraform state file - It is quite often for a developer to take a break from work and during the break if the developer's terraform project is out of sync then it is always recommended to get in sync with all the updates which have happened.

So apart from doing the normal git pull it is also good to do the terraform state pull, so that all the latest changes which have been made to the terraform state file will be retrieved -

1terraform state pull

terraform state pull prints the complete state as JSON to your terminal. The most common reason I use it is to take a backup before doing anything risky -

1terraform state pull > backup-$(date +%F).tfstate

2. Push terraform state file - It is a dangerous command to work with. It can potentially lead you to an inconsistent state where you might override other team members' changes to the terraform state file. It is often discouraged to use the terraform state push command.

1terraform state push backup-2026-10-09.tfstate

How does terraform prevent us from an accidental state push?

1. Differing Lineage - Terraform has a safety mechanism on the terraform state file. Each time you create a terraform state file it will assign a lineage ID to the state file. If you are attempting to push a terraform state file with a different lineage ID then terraform will not allow it.

2. Higher Serial Number - Apart from the unique lineage ID terraform also assigns a serial number to the terraform state file. If you are attempting to push a terraform state file which has a lower serial number than the remote one, Terraform will not allow you to push it.

How to force push the terraform state file?

Considering all the safety mechanisms (lineage, higher serial number) provided by Terraform, if you still wanna override those safety settings then you can use the -force flag to push your Terraform state file.

1terraform state push -force backup-2026-10-09.tfstate

Other state commands you should know -

Here are the other terraform state sub-commands which I use almost every week -

 1# List every resource tracked in the state file
 2terraform state list
 3
 4# Show all the attributes of one resource
 5terraform state show aws_instance.ec2_example
 6
 7# Rename a resource in state without destroying it (e.g. after refactoring into a module)
 8terraform state mv aws_instance.ec2_example module.ec2.aws_instance.this
 9
10# Stop tracking a resource without destroying it
11terraform state rm aws_instance.ec2_example

I have written separate guides on terraform state rm and how to run terraform apply on a specific resource if you want to go deeper.

6.3 Terraform State Locking

Terraform has one more safety mechanism known as locking. By default terraform puts a lock on the terraform state file (if the backend supports locking).

Terraform state locking will prevent any other developer from updating the terraform state file while your terraform apply is running. Terraform locking always happens behind the scenes but if terraform locking fails then terraform will not let you continue -

1Error: Error acquiring the state lock
2
3Error message: operation error S3: PutObject, https response error StatusCode: 412, PreconditionFailed
4Lock Info:
5  ID:        3e2f9b1c-5d7a-4a9e-8f0c-1b2c3d4e5f60
6  Path:      jhooq-terraform-state/dev/terraform.tfstate
7  Operation: OperationTypeApply
8  Who:       rwagh@jhooq-laptop
9  Created:   2026-10-09 06:41:12.123456 +0000 UTC

How to disable the terraform state locking?

Terraform state locking can be disabled by passing the flag -lock=false but it is not at all recommended -

1terraform apply -lock=false

A much better option, if you just want to wait for your colleague's apply to finish, is -lock-timeout -

1terraform apply -lock-timeout=5m

How to Force Unlock the Terraform state file?

If someone has already acquired a lock on the Terraform state file (for example a CI job that was killed halfway) then you can use the force-unlock command with the lock ID from the error message above -

1terraform force-unlock 3e2f9b1c-5d7a-4a9e-8f0c-1b2c3d4e5f60

But there is a potential risk while using force-unlock - if the other apply is actually still running, two developers can perform the write operation on the terraform state file which can lead to a corrupted state. Always check with your team first.

7. How to store Terraform state file remotely on AWS S3 (step by step)

Alright, this is the part most of you came here for. Let's move a local state file to an S3 bucket. The S3 backend is the most popular remote backend for AWS users and since Terraform 1.10 it supports native state locking with the use_lockfile option, so you no longer need a separate DynamoDB table for locking.

7.1 Create the S3 bucket for the state

I prefer to create the state bucket with the AWS CLI (or by hand in the console) and not with Terraform, because of the chicken and egg problem - the bucket has to exist before Terraform can store its state in it.

 1# 1. Create the bucket (bucket names are global, so pick your own name)
 2aws s3api create-bucket \
 3  --bucket jhooq-terraform-state \
 4  --region eu-central-1 \
 5  --create-bucket-configuration LocationConstraint=eu-central-1
 6
 7# 2. Turn on versioning, so you can recover an older state if something goes wrong
 8aws s3api put-bucket-versioning \
 9  --bucket jhooq-terraform-state \
10  --versioning-configuration Status=Enabled
11
12# 3. Block all public access - the state file can contain secrets
13aws s3api put-public-access-block \
14  --bucket jhooq-terraform-state \
15  --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

7.2 Add the backend block

Add the following backend "s3" block to your main.tf (or a separate backend.tf) -

 1terraform {
 2  required_version = ">= 1.10"
 3
 4  backend "s3" {
 5    bucket       = "jhooq-terraform-state"
 6    key          = "dev/terraform.tfstate"
 7    region       = "eu-central-1"
 8    encrypt      = true
 9    use_lockfile = true
10  }
11}

• bucket - The S3 bucket we just created.
• key - The path of the state file inside the bucket. Use a different key for every project/environment.
• encrypt - Server-side encryption of the state object.
• use_lockfile - Enables the native S3 locking. Terraform creates a dev/terraform.tfstate.tflock object while an apply is running and deletes it afterwards.

What about dynamodb_table? If you have seen older tutorials (including my own video above) using a DynamoDB table for locking, that still works, but the dynamodb_table argument is deprecated since Terraform 1.11 and use_lockfile = true is the recommended way now. You can keep both during a transition and remove the DynamoDB table once everybody is on Terraform 1.10 or newer.

7.3 Migrate the existing local state

Now run terraform init. Because you already have a local terraform.tfstate, Terraform will ask whether you want to copy it to the new backend -

1terraform init -migrate-state

 1Initializing the backend...
 2Do you want to copy existing state to the new backend?
 3  Pre-existing state was found while migrating the previous "local" backend to the
 4  newly configured "s3" backend. No existing state was found in the newly
 5  configured "s3" backend. Do you want to copy this state to the new "s3"
 6  backend? Enter "yes" to copy and "no" to start with an empty state.
 7
 8  Enter a value: yes
 9
10Successfully configured the backend "s3"! Terraform will automatically
11use this backend unless the backend configuration changes.

Type yes. After this, the local terraform.tfstate is no longer used - you can delete it (and make sure *.tfstate and *.tfstate.backup are in your .gitignore, I have explained why in this post).

7.4 Verify

1aws s3 ls s3://jhooq-terraform-state/dev/

You should see terraform.tfstate in the bucket. Run a terraform plan and watch for the .tflock object appearing and disappearing while it runs -

1terraform plan

That's it - from now on every teammate who runs terraform init with the same backend block will read and write the same state file, with locking.

8. Terraform workspaces and state file

Terraform workspaces are a little special because terraform always creates a separate state file for each workspace.

With the local backend the state files of the non-default workspaces live inside a directory called terraform.tfstate.d -

1.
2├── main.tf
3├── terraform.tfstate            # "default" workspace
4└── terraform.tfstate.d
5    ├── dev
6    │   └── terraform.tfstate    # "dev" workspace
7    └── prod
8        └── terraform.tfstate    # "prod" workspace

With the S3 backend Terraform puts the workspace name in the object key automatically, under an env:/ prefix -

1s3://jhooq-terraform-state/dev/terraform.tfstate              # default workspace
2s3://jhooq-terraform-state/env:/staging/dev/terraform.tfstate # "staging" workspace

The rest of the Terraform state file handling (locking, pull, push) is still the same.

9. Common Terraform state errors and how to fix them

Here are the state related errors which people ask me about most often in the YouTube comments -

1. "Backend configuration changed" / "Backend initialization required" - You changed something in the backend block (or switched from remote to cloud). Run terraform init -migrate-state to move the state, or terraform init -reconfigure if you intentionally want to start pointing at a different, already existing state.

2. "type": "cloud" inside .terraform/terraform.tfstate - This confuses a lot of people. The file .terraform/terraform.tfstate is not your infrastructure state, it is just a small file where Terraform remembers which backend you initialised ("type": "cloud" means the HCP Terraform cloud block). Your real state lives in the backend. If this file is stale or you are switching backends, delete the whole .terraform directory and run terraform init again.

3. "Error acquiring the state lock" - Somebody else (or a dead CI job) holds the lock. Wait, use -lock-timeout, or terraform force-unlock <ID> as described in 6.3.

4. "state snapshot was created by Terraform v1.x, which is newer than current v1.y" - A teammate (or your CI) wrote the state with a newer Terraform. Upgrade your CLI; never downgrade the state by hand.

5. "Resource already exists" after a lost state file - Your state is gone but the resources are still in the cloud. Restore the state from S3 versioning (that is why we enabled it in 7.1), or re-adopt the resources one by one with terraform import.

10. Conclusion

The Terraform state file is always the heart of a terraform project. You must treat the terraform state file with care, otherwise you might end up in a situation where your terraform project is full of mess and hard to manage.

The best practices for using the terraform state file which I can recommend are -

1. If you have more than one developer working on the Terraform project then always use a remote backend (S3 with use_lockfile = true, or HCP Terraform) to store your state file remotely.
2. Enable versioning and encryption on the state bucket, and block public access. The state file is plain JSON and will contain things like database passwords.
3. Never commit terraform.tfstate to git - read why you should not store tfstate in git.
4. Take a terraform state pull > backup.tfstate before any state mv, state rm or force-unlock.
5. Do not force push the terraform state file unless you fully understand the lineage and serial, it will always lead you to inconsistent behavior.
6. Use terraform workspaces and terraform modules more often to effectively manage the Terraform state file.

Read More - Terragrunt -

1. How to use Terragrunt?