How to Load Input Data from a File in Terraform?

Terraform, HashiCorp's infrastructure as code (IaC) tool, is a favorite among developers and system administrators for its simplicity and power in managing and provisioning data center infrastructure.

However, as with any technology, there are complexities that arise as we go deeper. One such complexity is loading input data from a file. Let's address this topic in detail.

Table of Content

1. What is Input Data in Terraform?
2. Why Load Input Data from a File?
3. Loading Input Data: Terraform's file() Function
4. Parsing Structured Data: Terraform's jsondecode() and yamldecode() Functions
5. Load Input Data from a File using templatefile Function
6. Load Input Data from a File using local_file Data Source
7. Load Input Data from a File using template_file Data Source (Deprecated)
8. Wrapping Up: Leveraging Terraform's File-Based Inputs

1.What is Input Data in Terraform?

Before we delve into the details, let's understand what input data is in Terraform. Inputs in Terraform are like function arguments that allow you to customize aspects of the resource being created. They can include things like IP addresses, usernames, and any other parameters needed to configure your resources.

2. Why Load Input Data from a File?

Loading input data from a file can make your code more dynamic and flexible. For instance, you might have different configurations for different environments like development, staging, and production.

Having your input data in separate files allows you to quickly and easily change configurations without modifying your Terraform code.

3. Loading Input Data: Terraform's file() Function

One common way to load data from a file is by using Terraform's built-in file() function. This function reads the file at the given path and returns the file's content as a string.

1# main.tf 
2
3 resource "aws_instance" "example" {
4  user_data = file("${path.module}/user_data.sh")
5  # other configuration...
6}

In this example, the file() function is used to load a user data script that is run when the EC2 instance is launched.

Remember, the file() function reads the entire file as a string. If you're trying to load structured data such as JSON or YAML, you will need to use additional functions to parse the data.

4. Parsing Structured Data: Terraform's jsondecode() and yamldecode() Functions

Terraform provides the jsondecode() and yamldecode() functions to parse JSON and YAML files, respectively.

Here's an example of loading and parsing a JSON file:

1# main.tf 
2
3variable "config" {
4  type = map(any)
5  default = jsondecode(file("${path.module}/config.json"))
6}

In this example, the file() function is used to load a JSON configuration file, and jsondecode() is used to parse the loaded JSON data.

Important Considerations

1. File Path: Always ensure that the file path provided to the file() function is correct. The path can be either relative to the directory where Terraform is run, or absolute.

2. File Format: The file() function reads the file as a string. If the file contains structured data (like JSON or YAML), remember to use the appropriate function to parse the data.

3. Sensitive Data: If the file contains sensitive data (like passwords or API keys), be cautious about where and how you use this data. Avoid outputting sensitive data to the console or storing it in state files if possible.

5. Load Input Data from a File using templatefile Function

In a previous section, we discussed loading input data from a file in Terraform using the file() function.

Now, let's discuss some more advanced ways to load data from a file in Terraform using the templatefile function.

The templatefile function is an upgrade over the file function and can be used to render a template from a file. This function reads the file at the given path and treats its content as a template to be rendered.

This is a typical use-case:

1# main.tf
2# Read the template file user_data.sh.tpl
3
4resource "aws_instance" "example" {
5  user_data = templatefile("${path.module}/user_data.sh.tpl", { instance_name = "example-instance" })
6  # other configuration...
7} 

Here is the template file -

 1
 2#!/bin/bash
 3echo "Hello, this is ${instance_name}!"
 4
 5# Add any setup commands you need to run on your instance.
 6# For example, to update and upgrade an Ubuntu server, you might include:
 7apt-get update -y
 8apt-get upgrade -y
 9
10# You can use the instance_name variable anywhere you need it in this script.
11echo "${instance_name} setup complete!"

In this example, templatefile reads the file user_data.sh.tpl and substitutes the variables in the template with the values provided in the second argument (in this case, instance_name is substituted with "example-instance").

6. Load Input Data from a File using local_file Data Source

The local_file data source is used to read a local file's content into Terraform and use the data within the Terraform configuration.

Here is an example:

1data "local_file" "example" {
2  filename = "${path.module}/example.txt"
3}
4
5output "file_content" {
6  value = data.local_file.example.content
7} 

In this example, the local_file data source reads the file example.txt and the content is then accessible via data.local_file.example.content.

7. Load Input Data from a File using template_file Data Source (Deprecated)

The template_file data source is used to render a file template. Note that as of Terraform 0.12, this data source has been deprecated and replaced with the templatefile function. Here is how it was typically used:

 1data "template_file" "example" {
 2  template = "${file("${path.module}/example.tpl")}"
 3
 4  vars = {
 5    instance_name = "example-instance"
 6    environment   = "development"
 7  }
 8}
 9
10resource "aws_instance" "example" {
11  user_data = data.template_file.example.rendered
12  # other configuration...
13}

Here is the template file -

 1{
 2  "instance_name": "${instance_name}",
 3  "description": "This is an example instance named ${instance_name}.",
 4  "tags": [
 5    {
 6      "key": "Name",
 7      "value": "${instance_name}"
 8    },
 9    {
10      "key": "Environment",
11      "value": "${environment}"
12    }
13  ]
14}

In this example, template_file reads the file example.tpl and substitutes the variables in the template with the values provided in vars (in this case, instance_name is substituted with "example-instance").

8. Wrapping Up: Leveraging Terraform's File-Based Inputs

We've uncovered the power and flexibility of Terraform's file-based input capabilities. We started with the straightforward file() function, a tool that allows us to load an entire file as a string, useful for instances where we need to incorporate a static script or a configuration file in our Terraform setup.

When our needs became more complex, demanding dynamic inputs and variable substitutions, we discovered the templatefile function. This utility took our file inputs to a new level, enabling us to render file templates with dynamic variable substitution. This is ideal for situations where we want to incorporate changing variables into scripts or configuration files that need to be attached to our resources.

For scenarios where we simply wanted to read data from a local file into our Terraform configurations, we employed the local_file data source. This data source facilitates seamless integration of local file data into our Terraform workflow, empowering us to easily reference the content within our infrastructure setup.

Lastly, we delved into the history books and revisited the deprecated template_file data source. While it has been replaced by the templatefile function in Terraform 0.12 and later, understanding its usage provided us valuable insight into Terraform's evolution. However, for modern Terraform setups, it's advisable to use the templatefile function for the added efficiency and support.

Across all these methods, it's evident how Terraform has catered to a multitude of scenarios, ensuring that developers and system administrators have the tools they need to manage and provision infrastructure with ease and finesse.

As you venture forth in your Terraform journey, remember the lessons learned here. Understand your needs and choose the right method to manage your file-based input data. Whether it's a simple file(), a dynamic templatefile, or a local_file data source, Terraform's versatility is ready to back you up, simplifying your infrastructure management tasks and making your code more robust, dynamic, and maintainable.