Search for a command to run...
A DevOps/Cloud Architect experienced with building and automating DevOps processes and tools for engineering teams encompassing cloud-native applications. Extensive background experience working with AWS and Microsoft Azure cloud services building hands-on robust and scalable solutions and services.
No comments yet. Be the first to comment.
The Evolution of DevOps - The Golden Age Let's wind the clock back a couple of decades to the late 90s and early 2000s, a time when most organisations had a single gatekeeper: the system or IT administrator. If engineering teams needed to run their a...
This guide presents a comprehensive set of rules for software engineering inspired by the movie Zombieland. The rules are not simply a list of arbitrary guidelines, but rather a carefully crafted set of principles that are designed to promote efficie...
Organisations today are increasingly adopting cloud services from different providers to meet their business needs. However, managing multi-cloud infrastructure can be challenging, especially when it comes to infrastructure management, which is where...
What is AWS CloudFront? AWS CloudFront is a content delivery network (CDN). It securely delivers data, videos, applications, and APIs to customers globally with low latency and high transfer speeds. It offers a simple and cost-effective way to distri...
DevOps is more than just a software development methodology – it's a culture. The DevOps culture emphasizes 3 main pillars: collaboration, communication, and shared responsibility between development and operations teams. In this post, we will discus...
This post discusses the benefits of using Terraform for cloud automation. It explains the limitations of manual infrastructure management, the inception of IaC and how Terraform can help mitigate those issues. The document covers topics such as the Terraform configuration language, backends, workspaces, and Terraform Cloud while using AWS as a cloud provider example.
In the Iron Age, managing IT infrastructure was done manually. People would physically set up servers and configure them based on required settings by the OS and applications before deploying the application. This manual process often resulted in several problems, such as cost, scalability, availability, monitoring, and performance visibility.
To maintain all of this infrastructure, you would need to hire IT professionals and engineers. However, this can create a variety of issues such as inconsistencies, silos among teams when raising tickets to IT teams to provision new resources, miscommunications, and delays in reviewing and processing infrastructure requests.
Furthermore, when engineering teams need new infrastructure quickly, they typically use cloud consoles, such as AWS, Azure, Google, or Proxmox, to provision those resources. This approach is not an issue, and some engineers prefer spinning up resources this way to gain a better understanding of the prerequisites of a Cloud service or resource. While documentation can be helpful in this case, many engineers prefer hands-on experience and getting down to the nitty-gritty.
However, there is an issue to consider. What if you need to make changes that are not available through the user interface console? What if you need to make changes that depend on other changes being made first? This could have a widespread impact on the resources you were trying to set up. It can be difficult to keep track of all the resources and you might overlook updating or naming some of them with best practices in mind.
With modern technology, these problems can indeed be mitigated through automated processes, leading to improved efficiency and reduced costs. That's where Infrastructure as Code comes in.
Terraform is a powerful tool that allows you to manage your infrastructure as code. With Terraform, you can define, provision, and manage resources such as virtual machines, cloud instances, and containers across multiple cloud providers using a declarative configuration language known as HashiCorp Configuration Language, or HCL for short.
Imagine an orchestra of musicians. In this case, Terraform acts as the conductor who ensures that the right number of instruments (providers, modules) are playing and the sound (configuration) is correct. If there is an issue, the conductor replaces the broken instrument with a working one.
CloudFormation is one option for provisioning your infrastructure resources, but if you prefer to deploy to a different cloud provider of your choice, it can be a challenge. Terraform, on the other hand, excels at managing infrastructure for multiple cloud platforms. In addition, compared to CloudFormation and other cloud provider configuration languages, Terraform's configuration files are simpler to understand due to their flat format. This makes it easier to manage dependencies and references, which is where Terraform truly shines with its confident approach.
resource "aws_instance" "helloworld" {
ami = "ami-12d3df"
instance_type = "t3.micro"
tags = {
Name = "EC2-Instance-HelloWorld"
}
}
The Terraform configuration language, designed for human readability, allows you to write infrastructure code with ease. As shown in the snippet above, provisioning an EC2 instance on AWS is a straightforward task.
Each Terraform configuration has a backend associated with it that determines how operations are carried out and where data, such as the Terraform state, is stored. The state is a necessary requirement for Terraform to function since it is used to compare existing resources with newly added ones specified in the configuration files.
Backends are typically used to store Terraform state snapshots. A Terraform configuration can specify a backend, integrate with Terraform Cloud, or store state locally by default.
By default, Terraform stores its state in a local file directory named terraform.tfstate. However, if you need to, you can choose to use alternative backend repositories, such as the following:
Amazon S3
Kubernetes
Remote
Postgres
AzureRM (via Storage accounts)
Consul
and many others
The above can be configured by using a backend block in the configuration file (typically the main.tf file). This is how an S3 backend would look terin your configuration file:
terraform {
backend "s3" {
bucket = "sample"
key = "path/to/my/samplekey"
region = "ap-southeast-1"
}
}
To store the backend states on S3, simply supply your AWS IAM credentials from you local machine or your CI/CD tools.
In my opinion, it is best to store state files remotely in a cloud storage because this option allows for state locking. This means that two people working on the same project cannot cause conflicting issues when making changes to the resources.
There is a lot that you can achieve with Terraform states and their respective backends, but you should now have a general understanding of how states work in Terraform and why they are important to have.
In Terraform, workspaces are separate instances of state data that can be used from the same working directory. Workspaces can be used to manage multiple non-overlapping groups of resources with the same configuration (for example, staging, production, uat, etc.).
Almost all directories that have a main configuration file have a default workspace, named default. If you want to configure new workspaces, simply run the following command: terraform workspace new <workspace_name>.
Basically, it creates a new child working directory in your project directory that will contain your state files as such:
terraform.tfstate.d/
- staging/
- terraform.tfstate
- prod
- terraform.tfstate
- uat
- terraform.tfstate
- ...
With that, you can maintain different state files for all of your environments. All you have to do is switch workspaces accordingly when working for that specific environment, using terraform workspace select <workspace_name>.
Utilize workspaces when your organization operates in multiple environments where your services are deployed.
In typical scenarios, engineering teams commonly integrate Terraform with their preferred CI/CD toolset (e.g. Github Actions, Azure DevOps, Gitlabs, etc.) to ensure flexibility and self-governance over their state files and configuration, and to maintain coherence within their tech ecosystem. However, if teams are not familiar with CI/CD tools, they can rely on Terraform Cloud.
Terraform Cloud is a managed service offering by HashiCorp that eliminates the need for excessive tooling. You can provision your infrastructure in a remote environment hosted by HashiCorp that is optimized for Terraform workflows. This means that you can simply link your Cloud provider of choice with Terraform Cloud and let the platform handle all of the heavy lifting for you. All you need to do from your end is build and commit your Terraform (HCL) files to your repository and link your Git repository to your Terraform Cloud organization. Ideally, this follows the GitOps process. Write your infrastructure code, commit the changes, and Terraform Cloud executes your changes in a remote environment.
Here are some of the pros and cons of using CI/CD tools and Terraform Cloud. They are broken down into 4 criteria:
Control: With CI/CD tools, you have complete control over your infrastructure automation workflows, and you can customize your pipeline as required. With Terraform Cloud, you have less control over your infrastructure automation workflows, but you don't have to manage your own infrastructure.
Integration: CI/CD tools provide a wide range of integrations with other tools and services. Terraform Cloud integrates with a limited set of cloud providers, but it is optimized for Terraform workflows.
Maintenance: With CI/CD tools, you need to manage and maintain your own infrastructure for running the toolset if they are mostly on-premise/self-managed. With Terraform Cloud, you don't have to manage and maintain your own infrastructure.
Cost: CI/CD tools can be expensive, especially if you require a lot of customization and integrations. Terraform Cloud has a free tier, which provides basic functionality, and paid tiers that provide additional features.
So there you have it. Terraform is undeniably a powerful tool for managing infrastructure as code. Its declarative language, ability to manage resources across multiple cloud providers, and powerful set of tools make it an unrivalled choice for managing infrastructure and third-party systems. Whether you are managing a small set of resources or a large, complex infrastructure, Terraform can help you to manage it unfailingly and consistently. That’s why go-to tool for managing infrastructure and a great solution in DevOps.
By adopting Terraform, you can guarantee that your infrastructure is constantly in a known state and that any modifications are made in a controlled manner. Regardless of whether you're overseeing a small or large infrastructure, Terraform is a useful tool for increasing efficiency and effectiveness.