Using Make for SysAdmin Tasks
When most people hear about
make they think of compiling source code. While that is often what it’s used for, there are other good uses as well. Here’s an example of using it with Terraform/Ansible to build your infrastructure. Make has some advantages over just using a shell script. One is the native concept of dependencies. You can easily define which targets are dependencies of other targets. Make also has some idempotence baked in (as long as everything isn’t a phony target). Another advantage is that parallelism is built directly into make. You can run targets in parallel with other targets as long as they don’t have dependencies.
So first off here’s the contents of the Makefile:
.PHONY: clean download plan apply inventory roles destroy all: download plan apply inventory roles download: test -s terraform || wget https://releases.hashicorp.com/terraform/0.12.3/terraform_0.12.3_linux_amd64.zip -O terraform.zip test -s terraform || unzip terraform.zip plan: download cd $(env)/services && \ ../../terraform init -backend-config="bucket=bucket-name" cd $(env)/services && \ ../../terraform plan -out plan apply: plan cd $(env)/services && \ ../../terraform apply -input=false plan destroy: download cd $(env)/services && \ ../../terraform init -backend-config="bucket=bucket-name" && \ ../../terraform destroy -auto-approve inventory: apply cd $(env)/services && \ cp inventory ../../ansible/ roles: git clean -fdx ansible/roles/ ansible-galaxy install -r ansible/roles/requirements.yml clean: git clean -fd git clean -fx git reset --hard
It’s important to note that you could rename the download target as
terraform and make it a file target instead of phony. This way you wouldn’t need the
test -s terraform statement. Make will look to make sure that a file called the name of the target is created before the target is run. If that file exists, make won’t run the target. This is where it’s built in idempotence comes in to play.
If you look after each target you will see a name of another target. These are the dependencies for that target. So
plan depends on
apply depends on
plan, and so on. However
download don’t have any dependencies.
all target is first, if you just run
make it would be the same as running
make all which includes all of the targets listed next to all. We can run them in parallel with
make -j. If you don’t give make a number after the
-j it will run as many in parallel as possible. If you give it a number it will only run that many jobs at once.
For this Makefile, if we run
make -j env=dev make will download Terraform and unzip it while also downloading the Ansible roles we have defined both in parallel. Then it runs
terraform init with the specific bucket we put in the Makefile. It then runs
terraform plan and writes that to a file named plan. After that target is finished, it runs
terraform apply using the plan file we created. Once that target is finished, it copies the inventory file that Terraform created to a directory for Ansible to use afterward.
This workflow is really nice for both local development and through a pipeline. Now you don’t have to edit your CI/CD pipeline stages directly you can just edit the Makefile and only have one or two stages in the pipeline. And if you’re running locally you can run each target independently and it will only run the dependencies that specific target needs.
This specific Makefile also lets you destroy the infrastructure by running
make destroy env=dev (or whatever the directory name for your Terraform environment is). And if you want to wipe out your local changes, you can run
make clean and it will reset your local git repo to wherever HEAD is pointed.