Terraform Conditional Data Source: Create Flexible Infrastructure

I will start off this post with a simple statement and then try to show you why this statement is absolutely true.

Here it goes:

Terraform Conditional Data Source is the key to building flexible infrastructure.

And yes, I’m talking about conditional data source. Of course, Terraform Data Source is in itself an extremely useful tool. But you can supercharge the capabilities of a simple data source once you are able to add conditions to it.

Of course, it might be better if you first go through my detailed post on data sources in Terraform and then return to this post. You can also open them side by side.

In this post, you will learn everything important about Terraform conditional data source.

1 – What is a Terraform Conditional Data Source?

A Terraform conditional data source helps us read data conditionally from a resource within a Terraform configuration.

Right away, it opens up a number of possibilities:

• Check Resource Existence – You might be spinning up a few EC2 instances and want to assign them a security group that already exists. But you don’t want to take chances if the security group does not exist. You can use a conditional data source to make sure Terraform first checks if the security group already exists and creates a new one only if it doesn’t exist.
• Environment Specific Resources – Consider a common scenario where you have different environments such as development, staging and production. Imagine you want to use a different S3 bucket for each environment without modifying the Terraform’s configuration file. You can use Terraform conditional data source to check which the run-time environment and use the appropriate resource.
• Validating Input Variables – Imagine you want to make sure that the input variables to your Terraform configuration meet certain requirement. For example, a variable for IP address should contain a valid IP. You can use Terraform conditional data source to check the validity of the input variables and customize your configuration accordingly.

2 – Using Conditional Data Sources in Terraform

Let’s now see a usage example of Terraform conditional data source.

One of the most common uses is using environment specific resources. For example, based on the environment, you might want to use a different S3 bucket.

Check the below example:

provider "aws" {
    region = "us-west-2"
    profile = "terraform-user"
}

resource "aws_s3_bucket" "dev_bucket" {
  bucket = "saurabh-dev-bucket-2023-01-19"
}

resource "aws_s3_bucket" "prod_bucket" {
  bucket = "saurabh-prod-bucket-2023-01-19"
}

data "aws_s3_bucket" "example" {
  bucket = var.environment == "production" ? aws_s3_bucket.prod_bucket.bucket : aws_s3_bucket.dev_bucket.bucket
}

variable "environment" {
  type = string
  default = "development"
}

Here, you have two aws_s3_bucket resources. One is for development and the second for production.

Then, you have a data source denoted by the data block.

This is a Terraform conditional data source. It uses a ternary operator to evaluate the value of the environment variable. If the value of environment is “production”, the data source references the prod_bucket resource. If not, it references the dev_bucket resource.

In the current example, the value of environment is set to “development”.

So, how can you use this data source?

Here’s an example:

resource "aws_s3_object" "example" {
  bucket = data.aws_s3_bucket.example.bucket
  key    = "demo.txt"
  source = "demo.txt"
}

output "s3_bucket_name" {
  value = data.aws_s3_bucket.example.bucket
}

Assuming that you want to store a particular file into the bucket, you create a aws_s3_object resource named example.

The resource takes the bucket name as one of the input arguments. This is where you use the data source aws_s3_bucket.example. The data source will reference the bucket based on the selected environment. When Terraform processes this configuration, it’s going to create the resource and store the demo.txt file within the appropriate bucket.

You can create the infrastructure by executing terraform init followed by terraform apply.

Below is a screenshot of the S3 buckets that are created when you run the above configuration.

Also, you can find the demo.txt in the development bucket.

3 – Advantages of Conditional Data Sources

Here are some important advantages of Terraform conditional data sources

• You get more flexibility since you can reference different resources based on different conditions. In my opinion, this is the biggest draw while using conditional data sources.
• Your code becomes a lot simpler when you use a conditional data source. Even in the S3 example in the previous section, you can see what’s going on at one glance.
• Conditional data sources also help you drastically improve the reusability of your Terraform configuration. For example, handling multiple resources for different environments would have forced you to maintain different Terraform files or other solutions. With conditional data source, it is a simple matter of using a ternary operator.

4 – Pitfalls of Terraform Conditional Data Source

Generally speaking, conditional data sources are great. However, you would do well to keep a few pitfalls in mind while working with them:

• Don’t make the conditions too complex. Terraform is an infrastructure as code tool. It’s not a high-level programming language. Keep the conditions simple and easy to understand.
• Make sure you have the dependencies between resources properly mapped out. Terraform does not guarantee any specific ordering of operations. If there is a complex relationship between a couple of resources, you can use depends_on to make the dependency explicit.
• Terraform data sources (conditional or not) need to fetch data before you can use them. If the data fetching process is slow, it might negatively impact the performance of your Terraform configuration.

Conclusion

With this, we have successfully understood the usage of Terraform conditional data source.

Conditional data sources have several useful applications. While it is a great tool to design flexible infrastructure, you need to also evaluate the potential pitfalls while using it in your configuration code.

The code discussed in this post is available on Github for reference. Don’t forget to execute terraform destroy to destroy the infrastructure after you are done with your trials.

In case you want to delve deeper into Terraform, here are a few suggested posts that you might find interesting.

• How to create an AWS EC2 instance using Terraform?
• How to deploy Nginx on Docker using Terraform?
• Deploy Static Website on S3 with Terraform

If you liked this post or found it useful, consider sharing it with friends and colleagues. Do write your queries in the comments section below.