Terraform count, for_each and for Expressions Explained - Loop Over Lists, Sets and Maps (with the count vs for_each trap)


When working with collection variables in Terraform, you must understand the three ways to loop - count, for_each and the for expression. If you don't, going through a list, map or set of anything (users, subnets, security group rules) is painful, and worse, you will eventually hit the trap where removing one item from a list makes Terraform destroy and recreate resources it should not touch.

This article is about how all three work, with an AWS example for each, and when to use which. I have rewritten it for Terraform 1.16 - the biggest additions since the original are for_each on modules, the exact rules about what for_each may iterate over, and a proper explanation of the count-versus-for_each trap with a diagram.

Table of Content

  1. Loops with count
  2. Loops with for_each
  3. for expressions
  4. What is the difference between for_each and count?
  5. Common errors with count and for_each
  6. Conclusion


All examples use this provider setup, with the credentials coming from aws configure or an IAM role - never from the code. (The first version of this post had an access key pasted into a snippet. If you copied that, rotate it. I have explained the right ways in Terraform and AWS credentials handling.)

 1terraform {
 2  required_providers {
 3    aws = {
 4      source  = "hashicorp/aws"
 5      version = "~> 6.0"
 6    }
 7  }
 8}
 9
10provider "aws" {
11  region = "eu-central-1"
12}

1. Loops with count

1.1 Iterate a List using count

As the name suggests we are going to use count, and to use count we first need a collection. Let's create a collection variable of type list(string) -

1variable "user_names" {
2  description = "IAM usernames"
3  type        = list(string)
4  default     = ["user1", "user2", "user3"]
5}

Here is the pictorial representation of the above list variable -

terraform loop and for_each loop

In the above collection we have created a list of type string which contains usernames, and these usernames we are going to use for creating aws_iam_user resources.

count takes a whole number and creates that many instances of the resource. Inside the block, count.index is the current position, starting at 0 -

1resource "aws_iam_user" "example" {
2  count = length(var.user_names)
3  name  = var.user_names[count.index]
4}

terraform loop and for_each loop

Here is the complete terraform file -

 1resource "aws_iam_user" "example" {
 2  count = length(var.user_names)
 3  name  = var.user_names[count.index]
 4}
 5
 6variable "user_names" {
 7  description = "IAM usernames"
 8  type        = list(string)
 9  default     = ["user1", "user2", "user3"]
10}
11
12output "user_arns" {
13  value = aws_iam_user.example[*].arn   # all instances with the splat operator
14}

Once you apply this configuration with terraform apply, it creates three IAM users - user1, user2, user3 - addressed as aws_iam_user.example[0], [1] and [2].


1.2 Iterate a Set using count

You cannot use count directly on a set, because a set has no order and therefore no index. Convert the set to a list first -

 1#Step 1: Create a set
 2variable "my_set" {
 3  type    = set(string)
 4  default = ["value1", "value2", "value3"]
 5}
 6
 7#Step 2: Convert the set to a list
 8locals {
 9  my_list = tolist(var.my_set)
10}
11
12#Step 3: Use count to iterate
13resource "aws_iam_user" "example" {
14  count = length(local.my_list)
15  name  = local.my_list[count.index]
16}

Be aware that tolist() on a set gives you the elements in sorted order (for strings), not the order you wrote them in. If order matters, declare the variable as a list in the first place.


1.3 Iterate a Map using count

Ideally the count meta-argument is not meant for iterating over a map - that is what for_each is for - but here is how it can be done if you need it -

  1. Create a map variable
  2. Get all the keys with keys() into a local
  3. Use count and the keys list to look up each value
 1#Step 1: Create a map variable
 2variable "my_map" {
 3  type = map(string)
 4  default = {
 5    key1 = "value1"
 6    key2 = "value2"
 7    key3 = "value3"
 8  }
 9}
10
11#Step 2: Fetch the keys of the map (sorted)
12locals {
13  my_keys = keys(var.my_map)
14}
15
16#Step 3: iterate over the map using keys and count.index
17resource "aws_ssm_parameter" "example" {
18  count = length(local.my_keys)
19
20  name  = "/app/${local.my_keys[count.index]}"
21  type  = "String"
22  value = var.my_map[local.my_keys[count.index]]
23}

1.4 count as an on/off switch

The one place where count is clearly the right tool even for a single resource - creating it conditionally -

 1variable "create_bastion" {
 2  type    = bool
 3  default = false
 4}
 5
 6resource "aws_instance" "bastion" {
 7  count = var.create_bastion ? 1 : 0
 8
 9  ami           = data.aws_ami.ubuntu.id
10  instance_type = "t3.micro"
11}
12
13# referencing it elsewhere - it is a list with 0 or 1 element
14output "bastion_ip" {
15  value = var.create_bastion ? aws_instance.bastion[0].public_ip : null
16}

2. Loops with for_each

2.1 Iterate a List using for_each (toset)

The for_each meta-argument is a little special in Terraform - you cannot use it on just any collection.

Note : - It can only be used on a set of strings or a map. The documentation says it plainly - "The for_each meta-argument accepts a map or a set of strings."

The reason why for_each does not work on a list(string) is that a list can contain duplicates and has positions, while Terraform needs a unique key for every instance. A set(string) or a map gives it that - the set element or the map key becomes the instance key.

How to make for_each work with a list - convert it to a set with the toset() function. "Conversion from list to set discards the ordering of the items in the list and removes any duplicate elements."

 1#Step 1: Create a list variable
 2variable "my_list" {
 3  type    = list(string)
 4  default = ["value1", "value2", "value3"]
 5}
 6
 7resource "aws_iam_user" "example" {
 8
 9  #Step 2: Convert the list to a set using the toset() function
10  for_each = toset(var.my_list)
11
12  #Step 3: For a set, each.key and each.value are the same string
13  name = each.value
14}

2.2 Iterate a Set using for_each

Let's first create a set(string) variable - then no conversion is needed at all -

1variable "user_names" {
2  description = "IAM usernames"
3  type        = set(string)
4  default     = ["user1", "user2", "user3"]
5}

Now let's iterate over the variable user_names -

1resource "aws_iam_user" "example" {
2  for_each = var.user_names
3  name     = each.value
4}

Here is the complete terraform file -

 1resource "aws_iam_user" "example" {
 2  for_each = var.user_names
 3  name     = each.value
 4}
 5
 6variable "user_names" {
 7  description = "IAM usernames"
 8  type        = set(string)
 9  default     = ["user1", "user2", "user3"]
10}
11
12output "user_arns" {
13  value = { for k, u in aws_iam_user.example : k => u.arn }
14}

Apply it with terraform apply and notice how the instances are named by key, not by number -

 1  # aws_iam_user.example["user1"] will be created
 2  + resource "aws_iam_user" "example" {
 3      + arn           = (known after apply)
 4      + force_destroy = false
 5      + id            = (known after apply)
 6      + name          = "user1"
 7      + path          = "/"
 8      + tags_all      = (known after apply)
 9      + unique_id     = (known after apply)
10    }
11
12  # aws_iam_user.example["user2"] will be created
13  + resource "aws_iam_user" "example" {
14      ...
15      + name          = "user2"
16    }
17
18  # aws_iam_user.example["user3"] will be created
19  + resource "aws_iam_user" "example" {
20      ...
21      + name          = "user3"
22    }

You reference one instance by its key - aws_iam_user.example["user2"].arn.


2.3 Iterate a Map using for_each

With a map, each.key is the map key and each.value is the value, which gives you two pieces of information per instance -

 1#Step 1: Create a map
 2variable "my_map" {
 3  type = map(string)
 4  default = {
 5    key1 = "value1"
 6    key2 = "value2"
 7    key3 = "value3"
 8  }
 9}
10
11#Step 2: Iterate over the map using for_each
12resource "aws_ssm_parameter" "example" {
13  for_each = var.my_map
14
15  name  = "/app/${each.key}"
16  type  = "String"
17  value = each.value
18}

2.4 for_each with a list of objects

In real projects the input is usually a list of objects - subnets with a name and a CIDR, users with a name and a role. for_each cannot take a list, so convert it into a map keyed by something unique, with a for expression (next section) -

 1variable "subnets" {
 2  type = list(object({
 3    name = string
 4    cidr = string
 5    az   = string
 6  }))
 7  default = [
 8    { name = "public-a",  cidr = "10.0.1.0/24", az = "eu-central-1a" },
 9    { name = "public-b",  cidr = "10.0.2.0/24", az = "eu-central-1b" },
10    { name = "private-a", cidr = "10.0.11.0/24", az = "eu-central-1a" },
11  ]
12}
13
14resource "aws_subnet" "this" {
15  # list of objects -> map keyed by name
16  for_each = { for s in var.subnets : s.name => s }
17
18  vpc_id            = aws_vpc.main.id
19  cidr_block        = each.value.cidr
20  availability_zone = each.value.az
21
22  tags = {
23    Name = each.key
24  }
25}

Now aws_subnet.this["public-a"] is stable forever, no matter where it sits in the list. If you want the input to be a map from the start, declare map(object({...})) and skip the conversion.

2.5 for_each on modules and chaining for_each

Since Terraform 0.13, for_each (and count) work on module blocks too - one module instance per environment, per region, per team -

 1module "bucket" {
 2  source   = "./modules/s3-bucket"
 3  for_each = toset(["logs", "backups", "artifacts"])
 4
 5  name = "jhooq-${each.key}"
 6}
 7
 8output "bucket_arns" {
 9  value = { for k, m in module.bucket : k => m.arn }
10}

And one for_each resource can feed another directly, when the two have a one-to-one relationship - the docs call this chaining. One internet gateway per VPC -

1resource "aws_vpc" "this" {
2  for_each   = var.vpcs            # map of name => cidr
3  cidr_block = each.value
4}
5
6resource "aws_internet_gateway" "this" {
7  for_each = aws_vpc.this           # the resource itself is the map
8  vpc_id   = each.value.id
9}

2.6 The rules - known at plan time, no sensitive values

Three restrictions, straight from the for_each documentation, which explain most of the errors in Section 5 -

  1. Keys must be known at plan time. "All values that the for_each argument iterates over must be known before Terraform performs any remote resource operations." You cannot key on a resource attribute that is only known after apply (an instance id, an ARN). Key on the names in your variables instead.
  2. No sensitive values. "Sensitive values are not allowed because Terraform uses the value in for_each to identify the resource instance" - the key would end up in plan output and state.
  3. No impure functions. "Keys in the for_each argument cannot be the result of or rely on the result of impure functions" - uuid(), timestamp(), bcrypt().

And one more - you cannot use count and for_each on the same block.


3. for expressions

The for expression is not a meta-argument - it is an expression which builds a new list or map from an existing collection. If you have used list comprehensions in Python you will feel at home; only the syntax is different.

List output (square brackets) -

1[for <ITEM> in <COLLECTION> : <OUTPUT_VALUE>]

Map output (curly braces) -

1{for <KEY>, <VALUE> in <COLLECTION> : <OUTPUT_KEY> => <OUTPUT_VALUE>}

3.1 How to iterate over a List using for

Taking the same list(string) of three users - user1, user2, user3 -

1variable "user_names" {
2  description = "IAM usernames"
3  type        = list(string)
4  default     = ["user1", "user2", "user3"]
5}

A for expression can transform every element -

 1output "print_the_names" {
 2  value = [for name in var.user_names : name]              # same list
 3}
 4
 5output "upper_names" {
 6  value = [for name in var.user_names : upper(name)]       # ["USER1", "USER2", "USER3"]
 7}
 8
 9output "names_with_index" {
10  value = [for i, name in var.user_names : "${i}: ${name}"] # ["0: user1", "1: user2", "2: user3"]
11}
 1$ terraform apply
 2...
 3Outputs:
 4
 5names_with_index = [
 6  "0: user1",
 7  "1: user2",
 8  "2: user3",
 9]
10upper_names = [
11  "USER1",
12  "USER2",
13  "USER3",
14]

3.2 How to iterate over a Map using for

The same approach works for a map - you get the key and the value -

1variable "iam_users" {
2  description = "user => role"
3  type        = map(string)
4  default = {
5    user1 = "normal user"
6    user2 = "admin user"
7    user3 = "root user"
8  }
9}

Now let's iterate over the map, producing a list of strings and, separately, a new map -

1output "user_with_roles" {
2  value = [for name, role in var.iam_users : "${name} is the ${role}"]
3}
4
5output "roles_by_user" {
6  value = { for name, role in var.iam_users : name => upper(role) }
7}

3.3 Filtering with if, and list vs map output

The part that makes for expressions really useful - an if clause filters the input. This is how you pick a subset of resources without changing the variable -

 1# only the admins
 2output "admins" {
 3  value = [for name, role in var.iam_users : name if role == "admin user"]
 4}
 5
 6# subnets in one AZ, as a map ready for for_each
 7locals {
 8  subnets_in_a = { for s in var.subnets : s.name => s if s.az == "eu-central-1a" }
 9}
10
11# group by: all user names per role (the ... turns duplicate keys into lists)
12output "users_by_role" {
13  value = { for name, role in var.iam_users : role => name... }
14}

The brackets decide the output type - [ ... ] gives a list (or tuple), { ... => ... } gives a map (or object). That is why for_each = { for s in var.subnets : s.name => s } works but for_each = [for s in var.subnets : s] does not.


3.4 Iterate over a Set using for

A for expression works on a set directly, and it is also the clean way to turn a set or list into a map with an index as the key, when you really need positions with for_each -

 1variable "my_set" {
 2  type    = set(string)
 3  default = ["value1", "value2", "value3"]
 4}
 5
 6locals {
 7  my_list = tolist(var.my_set)
 8}
 9
10resource "aws_ssm_parameter" "example" {
11  for_each = { for idx, value in local.my_list : idx => value }   # keys "0", "1", "2"
12
13  name  = "/app/item-${each.key}"
14  type  = "String"
15  value = each.value
16}

(Keying on the index brings back the count-style shifting problem, so prefer a real name as the key whenever one exists.)



4. What is the difference between for_each and count?

Both create multiple instances of a resource, but they identify those instances differently, and that difference decides what happens on the next change.

Removing user2 from the list: count shifts every index and recreates resources, for_each removes only the one

1. count : takes a whole number; each instance gets an index starting at 0. Three identical instances -

1# The following terraform code will create 3 EC2 instances
2resource "aws_instance" "example" {
3  count = 3
4
5  ami           = data.aws_ami.ubuntu.id
6  instance_type = "t3.micro"
7}

You refer to them as aws_instance.example[0], [1], [2], or all at once with aws_instance.example[*].id.

2. for_each : takes a map or a set of strings; each instance gets a string key -

 1# One EC2 instance per entry in the map, each with its own AMI
 2resource "aws_instance" "example" {
 3  for_each      = var.amis
 4  ami           = each.value
 5  instance_type = "t3.micro"
 6
 7  tags = {
 8    Name = each.key
 9  }
10}
11
12variable "amis" {
13  description = "name => ami id"
14  type        = map(string)
15  default = {
16    web = "ami-0c55b159cbfafe1f0"
17    api = "ami-083ac7c7ecf9bb9b0"
18  }
19}

You refer to them as aws_instance.example["web"] and aws_instance.example["api"].

The differences that matter -

  1. count identifies instances by position; for_each by key.
  2. The trap - remove an item from the middle of a count list and every item after it shifts down one index. Terraform sees [1] changing from user2 to user3 and [2] disappearing, and plans to modify or replace them. With for_each, removing user2 destroys exactly ["user2"] and nothing else. For IAM users that is annoying; for database servers it is a disaster.
  3. for_each gives you per-instance settings for free (each.value.cidr), while with count you end up indexing into parallel lists.
  4. count is simpler for N identical copies and for the on/off switch (count = var.enabled ? 1 : 0).

The HashiCorp guidance matches - use for_each when the instances need distinct values that are not derived from an integer, count when they are nearly identical. If you have an existing count resource and want to switch to for_each without destroying anything, the moved block (Terraform 1.1+) renames the state entries -

1moved {
2  from = aws_iam_user.example[0]
3  to   = aws_iam_user.example["user1"]
4}

5. Common errors with count and for_each

1. Error: Invalid for_each argument ... The given "for_each" argument value is unsuitable: the "for_each" argument must be a map, or set of strings, and you have provided a value of type list of string. - The classic. Wrap it - for_each = toset(var.my_list).

2. Error: Invalid for_each argument ... "for_each" value depends on resource attributes that cannot be determined until apply - Your keys come from something only known after apply (rule 1 in 2.6). Key on names from your variables instead, or as a one-off, terraform apply -target=<the upstream resource> first.

3. Error: Sensitive values in for_each - Rule 2. The collection contains a sensitive value. Restructure so the keys are not secret, or nonsensitive() if they genuinely are not.

4. Error: Duplicate object key in a for expression - Two items produced the same key (two subnets with the same name). Fix the data, or use ... to group.

5. Error: Invalid index ... because it has "for_each" set, its attributes must be accessed on specific instances - You wrote aws_iam_user.example.arn on a resource with for_each. Use aws_iam_user.example["user1"].arn, or values(aws_iam_user.example)[*].arn for all.

6. Error: Missing resource instance key - The same for count - aws_instance.example.id must be aws_instance.example[0].id.

7. The plan wants to destroy and recreate everything after I reordered or removed a list item - The count trap from Section 4. Switch to for_each with moved blocks.

8. Error: Reserved argument name in module block / count not allowed on a module - Terraform older than 0.13. Upgrade; count and for_each on modules have been supported for years.

9. each.value is an object and I get "This object does not have an attribute named ..." - a typo in the attribute, or the object type in the variable does not declare it. Check the object({...}) definition.


6. Conclusion

Three tools, three jobs -

  1. count - N identical copies, or a conditional resource with ? 1 : 0. Instances are numbered, so removing one from the middle shifts the rest.
  2. for_each - one instance per key of a map or set (use toset() on lists, a for expression on lists of objects). Instances are named, stable and independent. The default choice for anything with a name.
  3. for expressions - transform and filter lists and maps into the shape the other two need, with if for filtering and { k => v } for maps.

Remember the three for_each rules - keys known at plan time, no sensitive values, no impure functions - and most of the errors above never happen. The official references are the for_each, count and for expressions pages, and if you are building many similar nested blocks inside one resource, that is the job of a dynamic block, which is the natural next post.


Read More - Terragrunt -

  1. How to use Terragrunt?

Posts in this series