Using Make for Sysadmin Tasks

2019-08-11

When most people hear about makemake they think of compiling source code. While that is often what it’s used for, there are other good uses as well. Here’s an example of using it with Terraform/Ansible to build your infrastructure. Make has some advantages over just using a shell script. One is the native concept of dependencies. You can easily define which targets are dependencies of other targets. Make also has some idempotence baked in (as long as everything isn’t a phony target). Another advantage is that parallelism is built directly into make. You can run targets in parallel with other targets as long as they don’t have dependencies.

So first off here’s the contents of the Makefile:

.PHONY: clean download plan apply inventory roles destroy
all: download plan apply inventory roles
download:
        test -s terraform || wget https://releases.hashicorp.com/terraform/0.12.3/terraform_0.12.3_linux_amd64.zip -O terraform.zip
        test -s terraform || unzip terraform.zip
plan: download
        cd $(env)/services && \
        ../../terraform init -backend-config="bucket=bucket-name"
        cd $(env)/services && \
        ../../terraform plan -out plan
apply: plan
        cd $(env)/services && \
        ../../terraform apply -input=false plan
destroy: download
        cd $(env)/services && \
        ../../terraform init -backend-config="bucket=bucket-name" && \
        ../../terraform destroy -auto-approve
inventory: apply
        cd $(env)/services && \
        cp inventory ../../ansible/
roles:
        git clean -fdx ansible/roles/
        ansible-galaxy install -r ansible/roles/requirements.yml
clean:
        git clean -fd
        git clean -fx
        git reset --hard

.PHONY: clean download plan apply inventory roles destroy
all: download plan apply inventory roles
download:
        test -s terraform || wget https://releases.hashicorp.com/terraform/0.12.3/terraform_0.12.3_linux_amd64.zip -O terraform.zip
        test -s terraform || unzip terraform.zip
plan: download
        cd $(env)/services && \
        ../../terraform init -backend-config="bucket=bucket-name"
        cd $(env)/services && \
        ../../terraform plan -out plan
apply: plan
        cd $(env)/services && \
        ../../terraform apply -input=false plan
destroy: download
        cd $(env)/services && \
        ../../terraform init -backend-config="bucket=bucket-name" && \
        ../../terraform destroy -auto-approve
inventory: apply
        cd $(env)/services && \
        cp inventory ../../ansible/
roles:
        git clean -fdx ansible/roles/
        ansible-galaxy install -r ansible/roles/requirements.yml
clean:
        git clean -fd
        git clean -fx
        git reset --hard

It’s important to note that you could rename the download target as terraformterraform and make it a file target instead of phony. This way you wouldn’t need the test -s terraformtest -s terraform statement. Make will look to make sure that a file called the name of the target is created before the target is run. If that file exists, make won’t run the target. This is where it’s built in idempotence comes in to play.

If you look after each target you will see a name of another target. These are the dependencies for that target. So planplan depends on downloaddownload, applyapply depends on planplan, and so on. However rolesroles, cleanclean, and downloaddownload don’t have any dependencies.

Since the allall target is first, if you just run makemake it would be the same as running make allmake all which includes all of the targets listed next to all. We can run them in parallel with make -jmake -j. If you don’t give make a number after the -j-j it will run as many in parallel as possible. If you give it a number it will only run that many jobs at once.

For this Makefile, if we run make -j env=devmake -j env=dev make will download Terraform and unzip it while also downloading the Ansible roles we have defined both in parallel. Then it runs terraform initterraform init with the specific bucket we put in the Makefile. It then runs terraform planterraform plan and writes that to a file named plan. After that target is finished, it runs terraform applyterraform apply using the plan file we created. Once that target is finished, it copies the inventory file that Terraform created to a directory for Ansible to use afterward.

This workflow is really nice for both local development and through a pipeline. Now you don’t have to edit your CI/CD pipeline stages directly you can just edit the Makefile and only have one or two stages in the pipeline. And if you’re running locally you can run each target independently and it will only run the dependencies that specific target needs.

This specific Makefile also lets you destroy the infrastructure by running make destroy env=devmake destroy env=dev (or whatever the directory name for your Terraform environment is). And if you want to wipe out your local changes, you can run make cleanmake clean and it will reset your local git repo to wherever HEAD is pointed.