How to use Terragrunt?
Terragrunt is an additional wrapper that is built on top of the Terraform. Terraform is a great Infrastructure as Code tool for managing your cloud infrastructure. But as the project size grows and you have multiple environments (Development, Testing, Staging, Production, etc..) to manage then you will realize Terraform has a lot of gaps for managing a complex and large project.
Challenges with Terraform - If you are managing multiple environments (Development, Testing, Staging, and Production etc..) infrastructure with Terraform then here are the challenges you might face with Terraform -
- Redundancy of code - Multiple copies of the same code for each environment.
- Manual update of code - If there are the same variables that are being used for all the environments then you have to remember and manually update each variable.
Terragrunt by gruntwork was built to improve the shortcomings around the Terraform for effectively managing the infrastructure code so that developers can use the same code without any kind of duplication by keeping the terraform code dry. Terragrunt not only helps you with managing your terraform workspaces effectively but it can also help you with multiple terraform modules, managing Terraform remote state, etc.
This guide will you get started with Terragrunt and also you will find example codes for your reference. It will be a very beginner's level guide for those who want to learn Terragrunt and in this guide, we will be taking one basic example of Setting up EC2 instance using Terragrunt for DEV and TEST
Table of Content
- How to install Terragrunt?
- Write your first Terragrunt implementation using the ec2-instance module.
- How to run - Terragrunt plan, Terragrunt apply, and Terragrunt destroy
- Verify the EC2 instance on AWS manually via AWS console as well as using - Terragrunt output
- Remove the AWS resources using - Terragrunt destroy
- Locals and Values files handling
- Handling Inputs and Variable Precedence
- Summary
1. How to install Terragrunt?
Before installing Terragrunt you must complete one pre-requisite by installing Terraform because Terragrunt is just a wrapper around terraform, so to use Terragrunt you must install Terraform.
There are a couple of ways of installing Terragrunt -
1.1 Manually download the Terragrunt Binary
- Download - Goto GitHub release pages of gruntwork-io/terragrunt and download the binary based on the operating
system
of your choice. (At the time of writing this blog post v0.37.1 was the latest from gruntwork)
Terragrunt release page for downloaing the binary - Rename - Rename the downloaded binary file to terragrunt.
- Make it Executable - Change the binary file permission to make it executable.
1chmod u+x terragrunt
- Move the binary file to
/usr/local/bin
(Linux and Mac)
1mv terragrunt /usr/local/bin/terragrunt
Here is the screenshot of my /usr/local/bin
directory after installation -
1.2 Install Terragrunt using package manager
If you do not like the manual installation of the Terragrunt then you can go with the package manager based on your operating system choice.
- Windows- For windows you can rely on chocolatey package manager -
1choco install terragrunt
- macOS- For mac you can install it with the help of Homebrew -
1brew install terragrunt
- Linux- For Linux either you could use Homebrew just like mac or you could go with manual installation. For Arch Linux user you could use the following installation command -
1pacman -S terragrunt
- FreeBSD - You can install it using Pkg -
1pkg install terragrunt
1.3 Verify Terragrunt Installation
After installing Terragrunt you can verify the installation by running the following command
1terragrunt version
(*Note - At the time of writing this blog post the latest version of Terragrunt is v1.2.0)
2. Write your first Terragrunt implementation using the ec2-instance module
In the previous step we have installed the Terragrunt along with Terraform. Now next problem we have is How to use Terragrunt for provisioning the cloud infrastructure?
But before we answer the question there are few points which you should know about terragrunt -
- Terragrunt never recommends duplication of code.
- Terragrunt heavily relies on modules, so that we can keep our code dry without polluting with the duplicate code
What are the necessary configuration you need to write your first Terragrunt implementation?
- Terraform AWS modules (Note - If you are working with Google Cloud then refer to Google Module)
- You will need
terragrunt.hcl
for writing your Terragrunt configuration - Use commands -
terragrunt plan, terragrunt apply, terragrunt output, terragrunt destroy
2.1 Package structure for my Terragrunt project
To keep the things simple I am gonna use the very a basic example in which we are going to setup an EC2 instance on DEV and TEST environments using terragrunt
- Project Structure - Here is the screenshot of my project structure -
- EC2 Instance Module - To use Terragrun we need to know the modules which we are going to use in our project. But since this is a getting started a project we are doing, I decided to go ahead with EC2 Module.
Here are a few starting lines of your terrgrunt.hcl
where you need to mention the reference of EC2 Module -
1#path - /terragrunt-ec2/dev/terragrunt.hcl
2
3terraform {
4 source = "tfr:///terraform-aws-modules/ec2-instance/aws?version=4.0.0"
5}
Notes-
- As you can see we are referring to the remote EC2 Module with -
Source URI - "tfr:///terraform-aws-modules/ec2-instance/aws?version=4.0.0"
- provider details- The next configuration we need to add inside
terragrunt.hcl
is the provider configuration. Provider configuration generally consists of three important elements -
- profile
- region
- credentials: In my current setup, I am using
shared_credential_file
to pass the credentials but you can useaccess_key
andsecret_key
instead
Here is the example configuration for the provider which we will be using inside the terragrunt.hcl
-
1#path - /terragrunt-ec2/dev/terragrunt.hcl
2
3generate "provider" {
4 path = "provider.tf"
5 if_exists = "overwrite_terragrunt"
6 contents = <<EOF
7 provider "aws" {
8 profile = "default"
9 region = "eu-central-1"
10 shared_credentials_file = "/Users/rwagh/credentials"
11 # access_key = "<insert_your_access_key>"
12 # secret_key = "<insert_your_secret_key>"
13 }
14EOF
15}
Notes -
-
The provider can be optional and it can be avoided if the provider.tf configuration exists already inside DEV or TEST environment.
-
The above provider configuration will overwrite any existing local provider configuration
-
Inputs- Next we need to define the inputs configuration because for each Terragrunt implementation we need to provide some mandatory inputs. So for our case of EC2 we will need the following mandatory parameters -
- ami
- tags
1#path - /terragrunt-ec2/dev/terragrunt.hcl
2
3inputs = {
4 ami = "ami-0767046d1677be5a0"
5 tags = {
6 Name = "Terragrunt Tutorial: EC2"
7 }
8}
2.2 Here is my final terragrunt.hcl
In the previous steps, we have gone through all the necessary configuration parameters which are needed for preparing
the terragrunt.hcl
. Here is my complete terragrunt.hcl
-
1#path - /terragrunt-ec2/dev/terragrunt.hcl
2
3terraform {
4 source = "tfr:///terraform-aws-modules/ec2-instance/aws?version=4.0.0"
5}
6
7
8generate "provider" {
9 path = "provider.tf"
10 if_exists = "overwrite_terragrunt"
11 contents = <<EOF
12 provider "aws" {
13 profile = "default"
14 region = "eu-central-1"
15 shared_credentials_file = "/Users/rwagh/credentials"
16 # access_key = "<insert_your_access_key>"
17 # secret_key = "<insert_your_secret_key>"
18 }
19EOF
20}
21
22
23inputs = {
24 ami = "ami-0767046d1677be5a0"
25 tags = {
26 Name = "Terragrunt Tutorial: EC2"
27 }
28}
3. How to run - terragrunt plan, terragrunt apply, terragrunt output and terragrunt destroy
3.1 terragrunt init
Let's apply the Terragrunt configuration to create EC2 instances. The first command you need to run is trragrunt init
.
Before running the terragrunt init
command you must switch to the directory where the terragrunt.hcl
is present.
1terragrunt init
And it should produce output -
3.2 terragrunt plan
The next command you can run is terragrunt plan
which is also similar to the terraform plan command. The terragrunt
plan command is going to validate your Terragrunt configuration and will return you back with the information on
how many resources it is going to add, update or destroy.
Since this is the first time we are going to create a EC2 instance so it should show that 1 resource it is going to add.
1 terragrunt plan
Here is the output of the Terragrunt plan command -
3.3 terragrunt apply
If all of your Terragrunt configurations is correct and there is no error associated with it then you can simply
execute terragrunt apply
command which will create an EC2 instance
1terragrunt apply
Here is the output of the Terragrunt apply command -
If you see a similar output onto your console it means you have successfully executed your Terragrunt and your EC2 instance has been created. In the next step, we are going to verify the EC2 instance by logging onto the AWS console.
4. Verify the EC2 instance on AWS
4.1 Manually verify by logging onto the AWS console
Let's log in onto the AWS console with the correct region and then go to the EC2 dashboard.
As you can see in the above screenshot we have one EC2 instance up and running. You can click on the Instances link to get more details on the running instance.
Here are a few things to verify -
- Tags - In the
terragrunt.hcl
we have specified the values of tags as Terragrunt Tutorial: EC2 and the same tag name we can see on the AWS EC2 dashboard. - Instance state - If you have successfully started the instance then the state should be up and running.
4.2 Verify using Terragrunt output command
Terragrunt also provides you with one more useful command terragrunt output
to verify the details of the resources
which you have provisioned using the Terragrunt.
As in our use case, we have created an EC2 instance, so we could also verify the details of running the EC2 instance
with the help of command terragrunt output
1terragrunt output
Here are the details of the running instance -
5. Remove the AWS resources using - terragrunt destroy
Now the last command for Terragrunt is - terragrunt destroy
. In all the previous steps we have set up and started
EC2 instance. After doing the successful verification of the instance let's try to delete/destroy the EC2
instance with the terragrunt destroy
command.
Goto terminal and simply run the following command -
1terragrunt destroy
Once you have executed the terragrunt destroy
command successfully then you should see the following output onto your
terminal -
Now you have successfully deleted your EC2 instance with terragrunt destroy
command.
6. Locals and Values files handling
Now we have seen how you can get yourself started with Terragrunt but there is one more very essential feature of Terragrunt knows a values files. Terragrunt allows you to externalize the values files so that you can define common input variables as well as environment-specific variables.
For Example - We have taken two environments for this tutorial .i.e. - DEV, and TEST. To make our Terragrunt setup
more generic we can create one common common-environment.yaml
which we can place at the root of the project. So
that all the common input variables can be defined in it.
Along with that we can also create environment specific values file .i.e. - dev-environment.yaml, test-environment. yaml
for DEV and TEST
Here is the screenshot of my project structure -
After creating the values files, we have to define some common variables for DEV, TEST as well as define some environment-specific variables for each environment.
- Define some variables inside common-environment.yaml- As you know this is just a tutorial so we are going to define a common variable for EC2 instance type
1#path - /terragrunt-ec2/common-environment.yaml
2
3locals:
4 instance_type : "t2.micro"
- How to include common-environment.yaml?- In the previous step we have defined a common variable
instance_type
but to use that variable inside terragrunt.hcl you have to use yamldecode function inside the input block.
Example -
1 #path - /terragrunt-ec2/common-environment.yaml
2
3locals {
4 env_vars = yamldecode(
5 file("${find_in_parent_folders("common-environment.yaml")}"),
6 )
7}
After including the common-environment.yaml we have to use the variable instance_type
inside the inputs
section of terragrunt-hcl
. You can use the same locals rules which we use while referring to locals inside
terraform.
Example -
1#path - /terragrunt-ec2/common-environment.yaml
2
3inputs = {
4 ami = "ami-00c90dbdc12232b58"
5 instance_type = local.env_vars.locals.instance_type
6 tags = {
7 Name = "Terragrunt Tutorial: EC2"
8 }
9}
7. Handling Inputs and Variable Precedence
Let's take the same example to deep dive into the input parameters which we have defined inside the terragrunt. hcl
. The input block is there for you to set values for the module's input parameter.
There are a few rules on how you should create input block -
- the block should start with
inputs
Example -
1inputs = {
2 ami = "ami-00c90dbdc12232b58"
3 instance_type = "t2.micro"
4}
- You need to create an input block inside
terragrunt.hcl
- As soon as you execute
terragrunt apply
command all the values you have defined inside the input block will be pass in as environment variables for your module. - TF_VAR_XXX - If you have already defined
variables.tf
inside your terraform project or modules then inputs the block will not override the values from the inputs block.
7.1 Overriding precedence of variables in terragrunt
The variables overriding precedence are the same as Terraform. But if the same variables have been defined multiple numbers of times then it will use the last assigned value.
But if you are interested in knowing the order of overriding precedence then you should follow these orders -
- Terragrunt will first try to load the variables from Environment Variables
- It will look for the same variables in terraform.tfvars
- It will also look inside terraform.tfvars.json
- Anything defined inside .auto.tfvars/.auto.tfvars.json will be processed in order of their filenames
- Anything passed as an option -var/-var-file from the command line will take the precedence
8. Summary
If you have followed all the steps of this blog then I can say for sure you have a good understanding of the basics of Terragrunt and you can make your implementation of Terragrunt. Let's recap what we have learned so far -
- We have installed Terragrunt along with the terraform and also verified the installation of terragrunt
- Setup a small EC2 instance using Terragrunt's EC2 instance module
- Along with the EC2 instance we have also seen how to layout your project structure for different environment (Development, Test, Stage, Prod)
- We also set up the environment-specific
terragrunt.hcl
for DEV as well as TEST environment - Considering this post as a beginners tutorial for Terragrunt you will have a better understanding of how to -
- Include module
- Define locals inside your Terragrunt project
- write provider block
- Write input block for module's input
- In the blog post we have also seen how you can externalize the
common-environment.yaml
so that you can define common local variables as well as define environment specific variables.
I have some more thoughts on Terragrunt so in the upcoming months, I will write some more posts on Terragrunt and will be updating the list here. Here are my upcoming post topics on Terragrunt -
- Setting up S3 Bucket and remote state file using Terragrunt
- Creating your own module on Github for Terragrunt
- Terragrunt goto guide with more example on - dependencies, merge
My Terraform and Terragrunt Repository- You can find the same code which I have used in my blog on GitHub repository - https://github.com/rahulwagh/Terraform-Topics
Read More - Terragrunt -
Posts in this Series
- Securing Sensitive Data in Terraform
- Boost Your AWS Security with Terraform : A Step-by-Step Guide
- How to Load Input Data from a File in Terraform?
- Can Terraform be used to provision on-premises infrastructure?
- Fixing the Terraform Error creating IAM Role. MalformedPolicyDocument Has prohibited field Resource
- In terraform how to handle null value with default value?
- Terraform use module output variables as inputs for another module?
- How to Reference a Resource Created by a Terraform Module?
- Understanding Terraform Escape Sequences
- How to fix private-dns-enabled cannot be set because there is already a conflicting DNS domain?
- Use Terraform to manage AWS IAM Policies, Roles and Users
- How to split Your Terraform main.tf File into Multiple Files
- How to use Terraform variable within variable
- Mastering the Terraform Lookup Function for Dynamic Keys
- Copy files to EC2 and S3 bucket using Terraform
- Troubleshooting Error creating EC2 Subnet InvalidSubnet Range The CIDR is Invalid
- Troubleshooting InvalidParameter Security group and subnet belong to different networks
- Managing strings in Terraform: A comprehensive guide
- How to use terraform depends_on meta argument?
- What is user_data in Terraform?
- Why you should not store terraform state file(.tfstate) inside Git Repository?
- How to import existing resource using terraform import comand?
- Terraform - A detailed guide on setting up ALB(Application Load Balancer) and SSL?
- Testing Infrastructure as Code with Terraform?
- How to remove a resource from Terraform state?
- What is Terraform null Resource?
- In terraform how to skip creation of resource if the resource already exist?
- How to setup Virtual machine on Google Cloud Platform
- How to use Terraform locals?
- Terraform Guide - Docker Containers & AWS ECR(elastic container registry)?
- How to generate SSH key in Terraform using tls_private_key?
- How to fix-Terraform Error acquiring the state lock ConditionalCheckFiledException?
- Terraform Template - A complete guide?
- How to use Terragrunt?
- Terraform and AWS Multi account Setup?
- Terraform and AWS credentials handling?
- How to fix-error configuring S3 Backend no valid credential sources for S3 Backend found?
- Terraform state locking using DynamoDB (aws_dynamodb_table)?
- Managing Terraform states?
- Securing AWS secrets using HashiCorp Vault with Terraform?
- How to use Workspaces in Terraform?
- How to run specific terraform resource, module, target?
- How Terraform modules works?
- Secure AWS EC2s & GCP VMs with Terraform SSH Keys!
- What is terraform provisioner?
- Is terraform destroy needed before terraform apply?
- How to fix terraform error Your query returned no results. Please change your search criteria and try again?
- How to use Terraform Data sources?
- How to use Terraform resource meta arguments?
- How to use Terraform Dynamic blocks?
- Terraform - How to nuke AWS resources and save additional AWS infrastructure cost?
- Understanding terraform count, for_each and for loop?
- How to use Terraform output values?
- How to fix error configuring Terraform AWS Provider error validating provider credentials error calling sts GetCallerIdentity SignatureDoesNotMatch?
- How to fix Invalid function argument on line in provider credentials file google Invalid value for path parameter no file exists
- How to fix error value for undeclared variable a variable named was assigned on the command line?
- What is variable.tf and terraform.tfvars?
- How to use Terraform Variables - Locals,Input,Output
- Terraform create EC2 Instance on AWS
- How to fix Error creating service account googleapi Error 403 Identity and Access Management (IAM) API has not been used in project before or it is disabled
- Install terraform on Ubuntu 20.04, CentOS 8, MacOS, Windows 10, Fedora 33, Red hat 8 and Solaris 11