How Terraform modules works?
Terraform is an IaC(infrastructure as Code) framework for managing and provisioning the infrastructure but have you ever thought of creating only a single terraform configuration file for managing the complete cloud infrastructure.
Well, it sounds insane because if you only have a single file for managing the complete infrastructure then it will grow in complexity as well the line of configuration code will be manifold. That is why terraform introduced a concept of module which will help you to organize your terraform configuration so that you can re-use the configuration and keep your terraform code more clean and modular.
In this blog post, we will create two modules module-1 and module-2. Inside each module, we will install the apache httpd server with each module is having its home page.
Click here to clone - GitHub Repo
Steps for creating terraform modules along with best practices and benefits
- Create your first module
- Module structure
- Calling the module
- Module Inputs
- Module Output
- Module best practices
- Benefits of Modules
1. Create your first module
To keep things simple we will be creating two modules named module-1, module-2.
Each module will have its own main.tf
files.
1.1 Install apache httpd server in each module
To keep things more granular we will install two apache httpd servers in each module.
Let us start with the module-1 -
1. Specify the terraform required version
1terraform {
2 required_version = ">=0.12"
3}
2. Create an ec2 aws_instance along with the user_data block in which we will be writing bash commands for installing the apache2 httpd server.
(Note - Here in this instance I have used the key_name aws_key
. Follow this post on how to create aws keys for your ec2 instance )
1resource "aws_instance" "ec2_example" {
2
3 ami = "ami-0767046d1677be5a0"
4 instance_type = "t2.micro"
5 key_name= "aws_key"
6 vpc_security_group_ids = [aws_security_group.main.id]
7}
and here is the user_data block for installing the apach2 httpd server.
We will be using the same user_data block for installing the apache2 httpd server on module-2 also.
1user_data = <<-EOF
2 #!/bin/sh
3 sudo apt-get update
4 sudo apt install -y apache2
5 sudo systemctl status apache2
6 sudo systemctl start apache2
7 sudo chown -R $USER:$USER /var/www/html
8 sudo echo "<html><body><h1>Hello this is module-1 at instance id `curl http://169.254.169.254/latest/meta-data/instance-id` </h1></body></html>" > /var/www/html/index.html
9 EOF
3. Define the aws_security_group along with ingress rules for port 80 and 22.
We need to open the port 80 on the ec2 instance to access the apache httpd server home page and port 22 will need to ssh into the ec2 instance.
Here is the resource block -
1resource "aws_security_group" "main" {
2 name = "EC2-webserver-SG-2"
3 description = "Webserver for EC2 Instances"
4
5 ingress {
6 from_port = 80
7 protocol = "TCP"
8 to_port = 80
9 cidr_blocks = ["0.0.0.0/0"]
10 }
11
12 ingress {
13 from_port = 22
14 protocol = "TCP"
15 to_port = 22
16 cidr_blocks = ["115.97.103.44/32"]
17 }
18
19 egress {
20 from_port = 0
21 protocol = "-1"
22 to_port = 0
23 cidr_blocks = ["0.0.0.0/0"]
24 }
25}
4. Complete terraform script for module-1 and module -2
module-1
1terraform {
2 required_version = ">=0.12"
3}
4
5resource "aws_instance" "ec2_example" {
6
7 ami = "ami-0767046d1677be5a0"
8 instance_type = "t2.micro"
9 key_name= "aws_key"
10 vpc_security_group_ids = [aws_security_group.main.id]
11
12 user_data = <<-EOF
13 #!/bin/sh
14 sudo apt-get update
15 sudo apt install -y apache2
16 sudo systemctl status apache2
17 sudo systemctl start apache2
18 sudo chown -R $USER:$USER /var/www/html
19 sudo echo "<html><body><h1>Hello this is module-1 at instance id `curl http://169.254.169.254/latest/meta-data/instance-id` </h1></body></html>" > /var/www/html/index.html
20 EOF
21}
22
23resource "aws_security_group" "main" {
24 name = "EC2-webserver-SG-1"
25 description = "Webserver for EC2 Instances"
26
27 ingress {
28 from_port = 80
29 protocol = "TCP"
30 to_port = 80
31 cidr_blocks = ["0.0.0.0/0"]
32 }
33
34 ingress {
35 from_port = 22
36 protocol = "TCP"
37 to_port = 22
38 cidr_blocks = ["115.97.103.44/32"]
39 }
40
41 egress {
42 from_port = 0
43 protocol = "-1"
44 to_port = 0
45 cidr_blocks = ["0.0.0.0/0"]
46 }
47}
48
49
50resource "aws_key_pair" "deployer" {
51 key_name = "aws_key"
52 public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbvRN/gvQBhFe+dE8p3Q865T/xTKgjqTjj56p1IIKbq8SDyOybE8ia0rMPcBLAKds+wjePIYpTtRxT9UsUbZJTgF+SGSG2dC6+ohCQpi6F3xM7ryL9fy3BNCT5aPrwbR862jcOIfv7R1xVfH8OS0WZa8DpVy5kTeutsuH5FMAmEgba4KhYLTzIdhM7UKJvNoUMRBaxAqIAThqH9Vt/iR1WpXgazoPw6dyPssa7ye6tUPRipmPTZukfpxcPlsqytXWlXm7R89xAY9OXkdPPVsrQA0XFQnY8aFb9XaZP8cm7EOVRdxMsA1DyWMVZOTjhBwCHfEIGoePAS3jFMqQjGWQd rahul@rahul-HP-ZBook-15-G2"
53}
module-2
1terraform {
2 required_version = ">=0.12"
3}
4
5resource "aws_instance" "ec2_example" {
6
7 ami = "ami-0767046d1677be5a0"
8 instance_type = "t2.micro"
9 key_name= "aws_key"
10 vpc_security_group_ids = [aws_security_group.main.id]
11
12 user_data = <<-EOF
13 #!/bin/sh
14 sudo apt-get update
15 sudo apt install -y apache2
16 sudo systemctl status apache2
17 sudo systemctl start apache2
18 sudo chown -R $USER:$USER /var/www/html
19 sudo echo "<html><body><h1>Hello this is module-2 at instance id `curl http://169.254.169.254/latest/meta-data/instance-id` </h1></body></html>" > /var/www/html/index.html
20 EOF
21}
22
23resource "aws_security_group" "main" {
24 name = "EC2-webserver-SG-1"
25 description = "Webserver for EC2 Instances"
26
27 ingress {
28 from_port = 80
29 protocol = "TCP"
30 to_port = 80
31 cidr_blocks = ["0.0.0.0/0"]
32 }
33
34 ingress {
35 from_port = 22
36 protocol = "TCP"
37 to_port = 22
38 cidr_blocks = ["115.97.103.44/32"]
39 }
40
41 egress {
42 from_port = 0
43 protocol = "-1"
44 to_port = 0
45 cidr_blocks = ["0.0.0.0/0"]
46 }
47}
48
49
50resource "aws_key_pair" "deployer" {
51 key_name = "aws_key"
52 public_key = "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbvRN/gvQBhFe+dE8p3Q865T/xTKgjqTjj56p1IIKbq8SDyOybE8ia0rMPcBLAKds+wjePIYpTtRxT9UsUbZJTgF+SGSG2dC6+ohCQpi6F3xM7ryL9fy3BNCT5aPrwbR862jcOIfv7R1xVfH8OS0WZa8DpVy5kTeutsuH5FMAmEgba4KhYLTzIdhM7UKJvNoUMRBaxAqIAThqH9Vt/iR1WpXgazoPw6dyPssa7ye6tUPRipmPTZukfpxcPlsqytXWlXm7R89xAY9OXkdPPVsrQA0XFQnY8aFb9XaZP8cm7EOVRdxMsA1DyWMVZOTjhBwCHfEIGoePAS3jFMqQjGWQd rahul@rahul-HP-ZBook-15-G2"
53}
2. Module structure
In the step 1 we have seen the main.tf
terraform files of module-1 and module-2.
Now let's talk about the complete structure of your terraform project where you will have your parent main.tf
file which will be calling the module-1
and module-2
3. Calling the module
Alright in the previous steps we have seen how our modules main.tf
files along with the structure of the modules along with the parent main.tf
files.
Once you define the modules the next important step would be to call the modules from the parent main.tf
file and terraform makes it easy to call the modules based on the relative paths-
Here is my parent main.tf
calling the module-1 and module-2
1provider "aws" {
2 region = var.web_region
3 access_key = var.access_key
4 secret_key = var.secret_key
5}
6
7module "jhooq-webserver-1" {
8 source = ".//module-1"
9}
10
11module "jhooq-webserver-2" {
12 source = ".//module-2"
13}
4. Module Inputs
Just like any other programming language terraform also supports re-usability with the help of terraform module. You might be familiar with the concept of functions in other programming languages. In general functions always have some input parameter which needs to passed during the function call, similarly terraform module can also accept input parameters.
Example -
Let's take the same example where we have defined the module-1 but instead of hard coding ec2 instance type let's create an input variable which can be passed later -
1. Create a input variable inside module
Here is code snippet of module-1 where I have replaced hardcoded instance type to a variable var.web_instance_type
-
Path - terraform-modules/module-1/main.tf
1resource "aws_instance" "ec2_module_1" {
2
3 ami = var.ami_id
4 instance_type = var.web_instance_type
5 key_name= "aws_key"
6 vpc_security_group_ids = [aws_security_group.main.id]
7
8 user_data = <<-EOF
9 #!/bin/sh
10 sudo apt-get update
11 sudo apt install -y apache2
12 sudo systemctl status apache2
13 sudo systemctl start apache2
14 sudo chown -R $USER:$USER /var/www/html
15 sudo echo "<html><body><h1>Hello this is module-1 at instance id `curl http://169.254.169.254/latest/meta-data/instance-id` </h1></body></html>" > /var/www/html/index.html
16 EOF
17}
2. Pass the module input variable value while calling terraform module
In the following code snippet we are calling module-1 but also we are passing input variable .i.e. web_instance_type="t2.large", so based on your need and environment you can update instance type without changing the code your terraform module.
Path - terraform-modules/main.tf
1provider "aws" {
2 region = var.web_region
3 access_key = var.access_key
4 secret_key = var.secret_key
5}
6
7module "jhooq-webserver-1" {
8 source = ".//module-1"
9
10 web_instance_type = "t2.large"
11}
12
13module "jhooq-webserver-2" {
14 source = ".//module-2"
15}
16
5. Module Output
Terraform module also have the capability to produce the return output just like function which do return some value back after calling them.
Example -
1. Create an output variable public_ip_ec2
Let's again take the example of same terraform module-1, define an output block inside terraform module-1.
The following code snippet is for adding an output variable inside module-1 (Refer to GitHub for code repo)
Path - terraform-modules/module-1/main.tf
1resource "aws_instance" "ec2_module_1" {
2
3 ami = var.ami_id
4 instance_type = var.web_instance_type
5 key_name= "aws_key"
6 vpc_security_group_ids = [aws_security_group.main.id]
7
8 user_data = <<-EOF
9 #!/bin/sh
10 sudo apt-get update
11 sudo apt install -y apache2
12 sudo systemctl status apache2
13 sudo systemctl start apache2
14 sudo chown -R $USER:$USER /var/www/html
15 sudo echo "<html><body><h1>Hello this is module-1 at instance id `curl http://169.254.169.254/latest/meta-data/instance-id` </h1></body></html>" > /var/www/html/index.html
16 EOF
17}
18
19output "public_ip_ec2" {
20 value = aws_instance.app_server.public_ip
21 description = "Public IP address of EC2 instance"
22}
2. Use module output variable to show the output
After declaring module output variable inside terraform module we can refer to the output variable inside main. tf. But to access the output variable you need to use following syntax
1 module.<MODULE_NAME>.<OUTPUT_VARIABLE_NAME>
Here is the main.tf accessing the output variable public_ip_ec2
Path - terraform-modules/main.tf
1 provider "aws" {
2 region = var.web_region
3 access_key = var.access_key
4 secret_key = var.secret_key
5}
6
7module "jhooq-webserver-1" {
8 source = ".//module-1"
9
10 web_instance_type = "t2.large"
11}
12
13output "public_ip_ec2" {
14 value = module.module-1.public_ip_ec2
15 description = "Public IP of EC2"
16}
17
18module "jhooq-webserver-2" {
19 source = ".//module-2"
20}
21
Read More on Terraform module -
How to use module output variables as inputs for another module?
5. Module best practices
- You must always write the provider name inside your terraform file. For example, if you are using the AWS then the following line should be added as provider -
1provider "aws" {
2}
For google you can write the provider in the following manner -
1 provider "google" {
2}
-
While you are trying to write you're terraforming module always keep in mind that you have to go for simplicity and not to increase the complexity of your terraform project. Always keep the modules bare minimum if possible so that it will help other developers to understand and troubleshoot the issues if needed.
-
Terraform has a very good concept of local modules which you use to encapsulate your infrastructure logic. Use of modules is always recommended from the start of the project so that you have more control over the terraform code and it does not get spread sporadically.
-
Refer to the terraform public registry for finding more stable modules so that you do not have to re-invent the wheel.
-
Always keep and habit of publishing the modules so that it can be used by other teams members also
6. Benefits of Modules
Well, you will always get benefited from modules if you implement it properly. But here are certain advantages you will get with the terraform modules -
-
Organize configuration - With modules it is always easy to navigate and it helps any developer to understand the terraform project with ease. Certain with the module you can break down very complex infrastructure modules into very simplistic terraform modules.
-
Encapsulation - You can benefit from the encapsulation also and with terraform module you can hide the internal implementation of your infrastructure set up so that you can prevent unwanted changes happening to your modules by other developers.
-
Re-usability - If you have broken down your infrastructure into smaller and generic modules then it would help you to re-use the modules into another infrastructure setup.
-
Consistency - Following the best practices(encapsulation, organize, simple terraform module) will help you to achieve consistent behavior across all the different environments which will help you to reduce the debugging time and reduce the infrastructure maintenance cost.
If this is the first time you are trying to learn to terraform then I would highly recommend reading my Getting started guide on Terraform -
Read More - Terragrunt -
Posts in this Series
- Securing Sensitive Data in Terraform
- Boost Your AWS Security with Terraform : A Step-by-Step Guide
- How to Load Input Data from a File in Terraform?
- Can Terraform be used to provision on-premises infrastructure?
- Fixing the Terraform Error creating IAM Role. MalformedPolicyDocument Has prohibited field Resource
- In terraform how to handle null value with default value?
- Terraform use module output variables as inputs for another module?
- How to Reference a Resource Created by a Terraform Module?
- Understanding Terraform Escape Sequences
- How to fix private-dns-enabled cannot be set because there is already a conflicting DNS domain?
- Use Terraform to manage AWS IAM Policies, Roles and Users
- How to split Your Terraform main.tf File into Multiple Files
- How to use Terraform variable within variable
- Mastering the Terraform Lookup Function for Dynamic Keys
- Copy files to EC2 and S3 bucket using Terraform
- Troubleshooting Error creating EC2 Subnet InvalidSubnet Range The CIDR is Invalid
- Troubleshooting InvalidParameter Security group and subnet belong to different networks
- Managing strings in Terraform: A comprehensive guide
- How to use terraform depends_on meta argument?
- What is user_data in Terraform?
- Why you should not store terraform state file(.tfstate) inside Git Repository?
- How to import existing resource using terraform import comand?
- Terraform - A detailed guide on setting up ALB(Application Load Balancer) and SSL?
- Testing Infrastructure as Code with Terraform?
- How to remove a resource from Terraform state?
- What is Terraform null Resource?
- In terraform how to skip creation of resource if the resource already exist?
- How to setup Virtual machine on Google Cloud Platform
- How to use Terraform locals?
- Terraform Guide - Docker Containers & AWS ECR(elastic container registry)?
- How to generate SSH key in Terraform using tls_private_key?
- How to fix-Terraform Error acquiring the state lock ConditionalCheckFiledException?
- Terraform Template - A complete guide?
- How to use Terragrunt?
- Terraform and AWS Multi account Setup?
- Terraform and AWS credentials handling?
- How to fix-error configuring S3 Backend no valid credential sources for S3 Backend found?
- Terraform state locking using DynamoDB (aws_dynamodb_table)?
- Managing Terraform states?
- Securing AWS secrets using HashiCorp Vault with Terraform?
- How to use Workspaces in Terraform?
- How to run specific terraform resource, module, target?
- How Terraform modules works?
- Secure AWS EC2s & GCP VMs with Terraform SSH Keys!
- What is terraform provisioner?
- Is terraform destroy needed before terraform apply?
- How to fix terraform error Your query returned no results. Please change your search criteria and try again?
- How to use Terraform Data sources?
- How to use Terraform resource meta arguments?
- How to use Terraform Dynamic blocks?
- Terraform - How to nuke AWS resources and save additional AWS infrastructure cost?
- Understanding terraform count, for_each and for loop?
- How to use Terraform output values?
- How to fix error configuring Terraform AWS Provider error validating provider credentials error calling sts GetCallerIdentity SignatureDoesNotMatch?
- How to fix Invalid function argument on line in provider credentials file google Invalid value for path parameter no file exists
- How to fix error value for undeclared variable a variable named was assigned on the command line?
- What is variable.tf and terraform.tfvars?
- How to use Terraform Variables - Locals,Input,Output
- Terraform create EC2 Instance on AWS
- How to fix Error creating service account googleapi Error 403 Identity and Access Management (IAM) API has not been used in project before or it is disabled
- Install terraform on Ubuntu 20.04, CentOS 8, MacOS, Windows 10, Fedora 33, Red hat 8 and Solaris 11