Understanding terraform count, for_each and for loop?
When working with "collection variables" in Terrafrom, you must understand "loops with count," "loops with for each," and "for loop." If you don't, it will be very hard to go through collections like list, map, and set.
This article is mostly about how loops work. We'll look at an example of each idea in more depth:
Table of Content
1. Loops with count
1.1 Iterate List using count
As the name suggests we need to use count but to use the count first we need to declare collections inside our terraform file.
Let's create a collection variable of type list(string) -
1variable "user_names" {
2 description = "IAM usernames"
3 type = list(string)
4 default = ["user1", "user2", "user3"]
5}
Here is the pictorial representation of the above list variable -
In the above collection, we have created a list of type string which contains usernames and these usernames we are going to use for creating aws_iam_user.
The code snippet shows how we are going to iterate over the list(string) -
1resource "aws_iam_user" "example" {
2 count = length(var.user_names)
3 name = var.user_names[count.index]
4}
Here is the complete terraform file -
1provider "aws" {
2 region = "eu-central-1"
3 access_key = "XXXXXXXXXXXXXXXX"
4 secret_key = "XXXXXXXXXXXXXXXX"
5}
6resource "aws_instance" "ec2_example" {
7
8 ami = "ami-0767046d1677be5a0"
9 instance_type = "t2.micro"
10 count = 1
11
12 tags = {
13 Name = "Terraform EC2"
14 }
15
16}
17
18resource "aws_iam_user" "example" {
19 count = length(var.user_names)
20 name = var.user_names[count.index]
21}
22
23variable "user_names" {
24 description = "IAM usernames"
25 type = list(string)
26 default = ["user1", "user2", "user3"]
27}
Once you apply this terraform configuration using the terraform apply command, it will do the following on aws-
- Create one ec2 instance
- Create three IAM users - user1, user2, user3
1.2 Iterate Set using count
Here is one more example in which I will be iterating the set using the count meta-argument, although you can not use the count meta argument directly over the set, first you need to convert the set to the list.
Here is an example -
1#Step 1: Create a set
2variable "my_set" {
3 type = set(string)
4 default = ["value1", "value2", "value3"]
5}
6
7#Step 2: Convert set to list
8locals {
9 my_list = tolist(var.my_set)
10}
11
12#Step 3: Use count to iterate
13resource "my_resource" "example" {
14 count = length(local.my_list)
15
16 name = local.my_list[count.index]
17 # Additional resource configuration...
18}
1.3 Iterate map using count
Ideally the count meta-argument are not meant to be used for iterating over the map or set but there are some alternates by which you can iterate over the map using the count meta-argument.
Here is how it works -
- Create terraform variable map
- Create local and use keys to get all the keys
- Then use count and keys to iterate over the map
1
2#Step 1: Create a map variable
3variable "my_map" {
4 type = map(string)
5 default = {
6 key1 = "value1"
7 key2 = "value2"
8 key3 = "value3"
9 }
10}
11
12#Step 2: Fetch keys of map
13locals {
14 my_keys = keys(var.my_map)
15}
16
17#Step 3: iterate over map using keys and count.index meta argument
18resource "my_resource" "example" {
19 count = length(local.my_keys)
20
21 name = local.my_keys[count.index]
22 value = var.my_map[local.my_keys[count.index]]
23 # Additional resource configuration...
24}
2. Loops with for_each
2.1 Iterate List using for_each
The for_each is a little special in Terraform and you can not use it on any collection variable.
Note : - It can only be used on set(string) or map(string).
The reason why for_each does not work on list(string) is because a list can contain duplicate values but if you are using set(string) or map(string) then it does not support duplicate values.
How to make for_each work with list? -
- First you need to get rid of the duplicate values inside the list
- You need to convert(cast) list into set using the toset() function
Here is example -
1#Step 1: Create a list varible
2variable "my_list" {
3 type = list(string)
4 default = ["value1", "value2", "value3"]
5}
6
7resource "my_resource" "example" {
8
9 #Step 2: Convert list to set using toset() function
10 for_each = toset(var.my_list)
11
12 #Step 3: Iterate over the list
13 name = each.value
14 # Additional resource configuration...
15}
2.2 Iterate Set using for_each
Let's first create a set(string) variable -
1variable "user_names" {
2 description = "IAM usernames"
3 type = set(string)
4 default = ["user1", "user2", "user3s"]
5}
Now let's iterate over the variable user_names.
1resource "aws_iam_user" "example" {
2 for_each = var.user_names
3 name = each.value
4}
Here is the complete terraform file with implementation of for_each
1provider "aws" {
2 region = "eu-central-1"
3 access_key = "AKIATQ37NXB2NN3D4ARS"
4 secret_key = "3v9mlwZQvmccL3ou1dxiDeEf1bWaG3kccpVlXXXX"
5}
6resource "aws_instance" "ec2_example" {
7
8 ami = "ami-0767046d1677be5a0"
9 instance_type = "t2.micro"
10 count = 1
11
12 tags = {
13 Name = "Terraform EC2"
14 }
15
16}
17
18resource "aws_iam_user" "example" {
19 for_each = var.user_names
20 name = each.value
21}
22
23variable "user_names" {
24 description = "IAM usernames"
25 type = set(string)
26 default = ["user1", "user2", "user3"]
27}
You can apply the above terraform configuration by running the command terraform apply.
Here is the output which you will notice after running the command -
1# aws_iam_user.example["user1"] will be created
2 + resource "aws_iam_user" "example" {
3 + arn = (known after apply)
4 + force_destroy = false
5 + id = (known after apply)
6 + name = "user1"
7 + path = "/"
8 + tags_all = (known after apply)
9 + unique_id = (known after apply)
10 }
11
12 # aws_iam_user.example["user2"] will be created
13 + resource "aws_iam_user" "example" {
14 + arn = (known after apply)
15 + force_destroy = false
16 + id = (known after apply)
17 + name = "user2"
18 + path = "/"
19 + tags_all = (known after apply)
20 + unique_id = (known after apply)
21 }
22
23 # aws_iam_user.example["user3s"] will be created
24 + resource "aws_iam_user" "example" {
25 + arn = (known after apply)
26 + force_destroy = false
27 + id = (known after apply)
28 + name = "user3s"
29 + path = "/"
30 + tags_all = (known after apply)
31 + unique_id = (known after apply)
32 }
2.3 Iterate map using for_each
Let's take a look on how to iterate over the map using the for_each in terraform.
Here is an example -
1
2#Step 1: Create a map
3variable "my_map" {
4 type = map(string)
5 default = {
6 key1 = "value1"
7 key2 = "value2"
8 key3 = "value3"
9 }
10}
11
12#Step 2: Iterate over the map using for_each
13resource "my_resource" "example" {
14 for_each = var.my_map
15
16 name = each.key
17 value = each.value
18 # Additional resource configuration...
19}
3. for loop
The for loop is pretty simple and if you have used any programming language before then I guess you will be pretty much familiar with the for loop.
Only the difference you will notice over here is the syntax in Terraform.
3.1 How to iterate over Set using for?
Here's an example of how you can use a for expression to iterate over a set:
1#Step 1: Create a set
2variable "my_set" {
3 type = set(string)
4 default = ["value1", "value2", "value3"]
5}
6
7#Step 2: Conver it to the list
8locals {
9 my_list = tolist(var.my_set)
10}
11
12#Step 3: Use for loop
13resource "my_resource" "example" {
14 for_each = { for idx, value in local.my_list : idx => value }
15
16 name = each.value
17 # Additional resource configuration...
18}
In the above example, we define a variable my_set of type set(string) and provide a default set with three values. The locals block is used to create a local variable my_list that converts the set into a list using the tolist function.
3.2 How to iterate over List using for?
I am going to take the same example by declaring a list(string) and adding three users to it - user1, user2, user3
1variable "user_names" {
2 description = "IAM usernames"
3 type = list(string)
4 default = ["user1", "user2", "user3"]
5}
You can use the above-declared variable inside your terraform file in a very simple way -
1output "print_the_names" {
2 value = [for name in var.user_names : name]
3}
You apply the above terraform configuration by running the command terraform apply. And if you see the logs where it will show how many users it is going to create -
1# aws_iam_user.example["user1"] will be created
2 + resource "aws_iam_user" "example" {
3 + arn = (known after apply)
4 + force_destroy = false
5 + id = (known after apply)
6 + name = "user1"
7 + path = "/"
8 + tags_all = (known after apply)
9 + unique_id = (known after apply)
10 }
11
12 # aws_iam_user.example["user2"] will be created
13 + resource "aws_iam_user" "example" {
14 + arn = (known after apply)
15 + force_destroy = false
16 + id = (known after apply)
17 + name = "user2"
18 + path = "/"
19 + tags_all = (known after apply)
20 + unique_id = (known after apply)
21 }
22
23 # aws_iam_user.example["user3s"] will be created
24 + resource "aws_iam_user" "example" {
25 + arn = (known after apply)
26 + force_destroy = false
27 + id = (known after apply)
28 + name = "user3s"
29 + path = "/"
30 + tags_all = (known after apply)
31 + unique_id = (known after apply)
32 }
3.3 How to iterate over MAP using for?
We can use a similar approach to iterate over the map also. But always keep in mind you need to specify the type of the map like string or number.
Here is the same example which I have taken but modified a bit for map -
1variable "iam_users" {
2 description = "map"
3 type = map(string)
4 default = {
5 user1 = "normal user"
6 user2 = "admin user"
7 user3 = "root user"
8 }
9}
Now let's iterate over the map
1output "user_with_roles" {
2 value = [for name, role in var.iam_users : "${name} is the ${role}"]
3}
Here is the difference between list and map syntax
For list -
1{for <ITEM> in <LIST> : <OUTPUT_KEY> => <OUTPUT_VALUE>}
For Map -
1{for <KEY>, <VALUE> in <MAP> : <OUTPUT_KEY> => <OUTPUT_VALUE>}
4. What is the difference between for_each and count?
for_each and count are both used to make multiple copies of a resource, but they work in very different ways.
1. count : count is an argument that takes a integer value that tells how many copies of a resource to make. When you use count, each instance gets a number (starting with 0), which you can use to get to that instance.
For example : If you wanted to create three instances of an aws_instance, you might use a configuration like this:
1# Following terraform code will create 3 instances of ec2
2
3resource "aws_instance" "example" {
4 count = 3
5
6 ami = "ami-0c55b159cbfafe1f0"
7 instance_type = "t2.micro"
8}
In the above example, Terraform will create three AWS instances with the same ami and instance_type. You can refer to these instances with aws_instance.example[count.index].
2. for_each : This option can be either a map or a set of strings. For each item in the map or set of strings, a new instance is made. The map or set gives each instance a key that can be used to get to that instance.
1# Iterate over the amis set defined in the variable
2# The followig resource will create two ec2 instances.
3
4resource "aws_instance" "example" {
5 for_each = var.ami_id
6 ami = each.value
7 instance_type = "t2.micro"
8}
9
10variable "ami_id" {
11 description = "IAM usernames"
12 type = set(string)
13 default = ["ami-0c55b159cbfafe1f0", "ami-083ac7c7ecf9bb9b0"]
14}
In the above example, Terraform will create two AWS instances with the ami specified for each in the for_each set.
The main differences between count and for_each are:
-
count works on a list and uses IDs that are whole numbers. for_each works with a map or set, and string keys are used as keys.
-
If you remove an item from the middle of your list using count, every item after it will move down to take its place, and Terraform will make them all over again. With for_each, each instance has a stable identity based on its key, and Terraform won't make new instances if you delete one.
-
for_each gives you more freedom and control, especially when you want to make resources with different traits, because you can map different resource options to values in each map or set.
Read More - Terragrunt -
Posts in this Series
- Securing Sensitive Data in Terraform
- Boost Your AWS Security with Terraform : A Step-by-Step Guide
- How to Load Input Data from a File in Terraform?
- Can Terraform be used to provision on-premises infrastructure?
- Fixing the Terraform Error creating IAM Role. MalformedPolicyDocument Has prohibited field Resource
- In terraform how to handle null value with default value?
- Terraform use module output variables as inputs for another module?
- How to Reference a Resource Created by a Terraform Module?
- Understanding Terraform Escape Sequences
- How to fix private-dns-enabled cannot be set because there is already a conflicting DNS domain?
- Use Terraform to manage AWS IAM Policies, Roles and Users
- How to split Your Terraform main.tf File into Multiple Files
- How to use Terraform variable within variable
- Mastering the Terraform Lookup Function for Dynamic Keys
- Copy files to EC2 and S3 bucket using Terraform
- Troubleshooting Error creating EC2 Subnet InvalidSubnet Range The CIDR is Invalid
- Troubleshooting InvalidParameter Security group and subnet belong to different networks
- Managing strings in Terraform: A comprehensive guide
- How to use terraform depends_on meta argument?
- What is user_data in Terraform?
- Why you should not store terraform state file(.tfstate) inside Git Repository?
- How to import existing resource using terraform import comand?
- Terraform - A detailed guide on setting up ALB(Application Load Balancer) and SSL?
- Testing Infrastructure as Code with Terraform?
- How to remove a resource from Terraform state?
- What is Terraform null Resource?
- In terraform how to skip creation of resource if the resource already exist?
- How to setup Virtual machine on Google Cloud Platform
- How to use Terraform locals?
- Terraform Guide - Docker Containers & AWS ECR(elastic container registry)?
- How to generate SSH key in Terraform using tls_private_key?
- How to fix-Terraform Error acquiring the state lock ConditionalCheckFiledException?
- Terraform Template - A complete guide?
- How to use Terragrunt?
- Terraform and AWS Multi account Setup?
- Terraform and AWS credentials handling?
- How to fix-error configuring S3 Backend no valid credential sources for S3 Backend found?
- Terraform state locking using DynamoDB (aws_dynamodb_table)?
- Managing Terraform states?
- Securing AWS secrets using HashiCorp Vault with Terraform?
- How to use Workspaces in Terraform?
- How to run specific terraform resource, module, target?
- How Terraform modules works?
- Secure AWS EC2s & GCP VMs with Terraform SSH Keys!
- What is terraform provisioner?
- Is terraform destroy needed before terraform apply?
- How to fix terraform error Your query returned no results. Please change your search criteria and try again?
- How to use Terraform Data sources?
- How to use Terraform resource meta arguments?
- How to use Terraform Dynamic blocks?
- Terraform - How to nuke AWS resources and save additional AWS infrastructure cost?
- Understanding terraform count, for_each and for loop?
- How to use Terraform output values?
- How to fix error configuring Terraform AWS Provider error validating provider credentials error calling sts GetCallerIdentity SignatureDoesNotMatch?
- How to fix Invalid function argument on line in provider credentials file google Invalid value for path parameter no file exists
- How to fix error value for undeclared variable a variable named was assigned on the command line?
- What is variable.tf and terraform.tfvars?
- How to use Terraform Variables - Locals,Input,Output
- Terraform create EC2 Instance on AWS
- How to fix Error creating service account googleapi Error 403 Identity and Access Management (IAM) API has not been used in project before or it is disabled
- Install terraform on Ubuntu 20.04, CentOS 8, MacOS, Windows 10, Fedora 33, Red hat 8 and Solaris 11