How to split Your Terraform main.tf File into Multiple Files

Table of Content

1. Why someone need to split the terraform main.tf into multiple files?
2. How to split the main.tf file: A Step-by-Step Approach
   • 2.1 Identify Resources and Data Sources
   • 2.2 Create Separate .tf Files for Resources and Data Sources
   • 2.3 Migrate Resource and Data Source Blocks into individual .tf files
   • 2.4 Define Variables and Outputs
3. Run terraform init, terraform validate, terraform plan and terraform apply commands
4. Conclusion

1. Why someone need to split the terraform main.tf into multiple files?

There are several good reasons to split a Terraform main.tf file into different files. Some of the main reasons why you might want to break up a Terraform setup into smaller parts are:

1. Improved Code Organization and Readability - As your Terraform setup gets bigger and more complicated, it can be hard to keep track of and understand.

By splitting the main.tf file into several smaller files, you can group resources, data sources, variables, and outputs that are linked. This makes the code easier to read, understand, and keep up-to-date.

2. Easier collaboration: Multiple developers or system managers may work on the same Terraform project in a team setting.

By dividing the main.tf file into smaller, clearly organized files, team members can work on different parts of the infrastructure at the same time without causing conflicts or confusion.

3. It's easier to reuse code - If you split your configuration into different files, you can easily use common parts like variables, outputs, and resource definitions in more than one project.

This modularity can save you time and work, and it helps keep your system consistent.

4. Easier to keep up: As your system changes over time, you may need to change, add, or take away resources.

Putting your Terraform configuration into various files makes it easier to find and change the parts of code that need to be changed. This simplified method can lead to fewer mistakes and a more efficient way to do repair.

5. Better version control: When using version control systems like Git, splitting your Terraform configuration into different files can make it easier to see what changes were made to which resources or components.

This clarity can help you keep track of changes better and make it easier to find and fix problems.

2. How to split the main.tf file: A Step-by-Step Approach

Follow these steps to successfully split your Terraform main.tf file into different files for better organization:

2.1 Identify Resources and Data Sources

Look at your main.tf file to see what tools and data sources you can separate into different files.

Putting resources and data sources that are similar together helps create logical boundaries and makes it easier to handle files.

Example -

 1# main.tf 
 2# Putting all aws resources in one file 
 3 
 4provider "aws" {
 5  region = "us-west-2"
 6}
 7
 8resource "aws_vpc" "example" {
 9  cidr_block = "10.0.0.0/16"
10}
11
12resource "aws_security_group" "example" {
13  name = "example"
14}
15
16resource "aws_subnet" "example" {
17  vpc_id     = aws_vpc.example.id
18  cidr_block = "10.0.1.0/24"
19} 

2.2 Create Separate .tf Files for Resources and Data Sources

Make a separate .tf file for each group of tools and data sources you found in the previous step. Give these files names that describe what they are and what they are used for.

For Example-

• provider.tf
• aws_vpc.tf
• aws_security_group.tf
• aws_subnet.tf

Here is one of my project structure screenshot which you can clone from the GitHub Repo -

2.3 Migrate Resource and Data Source Blocks into individual .tf files

After making separate .tf files, copy the necessary resource and data source blocks from the main.tf file to the new .tf files.

Example-

1. provider.tf

1provider "aws" {
2  region = "us-west-2"
3}

2. aws_vpc.tf

1resource "aws_vpc" "vpc-jhooq-eu-central-1" {
2  cidr_block = var.vpc_cidr
3  tags = {
4    Name = "VPC: jhooq-eu-central-1"
5  }
6}

3. aws_security_group.tf

 1resource "aws_security_group" "sg_vpc_jhooq_eu_central_1" {
 2  egress = [
 3    {
 4      cidr_blocks      = [ "0.0.0.0/0", ]
 5      description      = ""
 6      from_port        = 0
 7      ipv6_cidr_blocks = []
 8      prefix_list_ids  = []
 9      protocol         = "-1"
10      security_groups  = []
11      self             = false
12      to_port          = 0
13    }
14  ]
15  ingress                = [
16    {
17      cidr_blocks      = [ "0.0.0.0/0", ]
18      description      = ""
19      from_port        = 22
20      ipv6_cidr_blocks = []
21      prefix_list_ids  = []
22      protocol         = "tcp"
23      security_groups  = []
24      self             = false
25      to_port          = 22
26    }
27  ]
28  vpc_id = aws_vpc.vpc-jhooq-eu-central-1.id
29  depends_on = [aws_vpc.vpc-jhooq-eu-central-1]
30  tags = {
31    Name = "SG : vpc-jhooq-eu-central-1 "
32  }
33}

4. aws_subnet.tf

 1# Setup public subnet
 2resource "aws_subnet" "aws_jhooq_public_subnets" {
 3  count      = length(var.cidr_public_subnet)
 4  vpc_id     = aws_vpc.vpc-jhooq-eu-central-1.id
 5  cidr_block = element(var.cidr_public_subnet, count.index)
 6  availability_zone = element(var.eu_availability_zone, count.index)
 7
 8  tags = {
 9    Name = "Subnet-Public : Public Subnet ${count.index + 1}"
10  }
11}
12
13# Setup private subnet
14resource "aws_subnet" "aws_jhooq_private_subnets" {
15  count      = length(var.cidr_private_subnet)
16  vpc_id     = aws_vpc.vpc-jhooq-eu-central-1.id
17  cidr_block = element(var.cidr_private_subnet, count.index)
18  availability_zone = element(var.eu_availability_zone, count.index)
19
20  tags = {
21    Name = "Subnet-Private : Private Subnet ${count.index + 1}"
22  }

2.4 Define Variables and Outputs

Define variables and outputs in different .tf files to keep things consistent and make them easier to use more than once.

Make the variables.tf and outputs.tf files and move the relevant blocks from the main.tf file to these new files.

Example variables.tf:

 1variable "region" {
 2  default = "us-west-2"
 3}
 4
 5variable "vpc_cidr_block" {
 6  default = "10.0.0.0/16"
 7}
 8
 9variable "subnet_cidr_block" {
10  default = "10.0.1.0/24"
11}

Example outputs.tf:

 1output "vpc_id" {
 2  value = aws_vpc.example.id
 3}
 4
 5output "security_group_id" {
 6  value = aws_security_group.example.id
 7}
 8
 9output "subnet_id" {
10  value = aws_subnet.example.id
11 

3. Run terraform init, terraform validate, terraform plan and terraform apply commands

After rearranging your Terraform project, run the terraform init and terraform validate commands to initialize the backend and make sure the new configuration is correct.

If there are any problems or mistakes, you need to fix them before moving on.

Finally, test your updated configuration by running terraform plan and terraform apply commands. These steps will help ensure that your infrastructure changes are executed correctly and that your configuration is functional.

4. Conclusion

In conclusion, dividing your Terraform main.tf file into different files is a good way to make it easier to organize and keep up to date.

By following the step-by-step plan and best practices in this guide, you can make sure that your Terraform configuration stays scalable and easy to handle as your infrastructure grows.