Getting started with Ansible and Terraform

By unixsysadminAnsible
Ansible Terraform Integration

Terraform and Ansible are both open source tools which can help automate the deployment and ongoing management of a server estate. One approach is to use Terraform to build infrastructure – these may be virtual servers, networks, firewalls, etc – and then use Ansible to manage the configuration of the infrastructure afterwards.

This walkthough shows a very basic configuration where two virtual machines are provisioned by Terraform, the provisioning state from Terraform is subsequently used as an Ansible inventory source, and finally Ansible configures the services that run on those servers.

It’s also worth reading Ansible Blog: Providing Terraform with that Ansible Magic for additional background.

RHEL 9, KVM and the Red Hat Developer Subscription

To perform this setup, I’m using KVM on my RHEL 9 laptop. This should work with the Red Hat developer subscription and out of the box open source repositories.

Setup KVM on the laptop – nothing special is required – I set it to start at boot and use the default network. I do however, add a storage pool for my virtual machines in a directory called /VMStorage.

In this example, I download the RHEL 9.1 QCOW image from Red Hat (follow the link on Red Hat Developer downloads or Software & Download Center on the Red Hat Customer Portal) and placed it in the following location:

/VMStorage/rhel-baseos-9.1-x86_64-kvm.qcow2

Terraform and Ansible control host

I use a Virtual Machine from where I run my Terraform and Ansible code – this mirrors a common setup where you may have a dedicated ‘admin’, ‘jumphost’ or ‘control host’ from where you manage servers. It’s from this admin VM that we run the commands

Install ansible-core from the standard RHEL 9 repository and terraform from the Hashicorp repository

sudo dnf install -y ansible-core
sudo dnf install -y dnf-plugins-core
sudo dnf config-manager --add-repo https://rpm.releases.hashicorp.com/RHEL/hashicorp.repo
sudo dnf install -y terraform

Terraform Basics

We’ll create a working directory from where we will run our Terraform and Ansible code

mkdir terraform_ansible_kvm && cd terraform_ansible_kvm

Alternatively, the code from this walk-through can be downloaded from https://github.com/unixsysadmin/terraform_ansible_kvm.

Terraform uses a plugin system to extend manageability of infrastructure components.

Terraform relies on plugins called providers to interact with cloud providers, SaaS providers, and other APIs.

Each provider adds a set of resource types and/or data sources that Terraform can manage.

Every resource type is implemented by a provider; without providers, Terraform can’t manage any kind of infrastructure.

Most providers configure a specific infrastructure platform (either cloud or self-hosted). Providers can also offer local utilities for tasks like generating random numbers for unique resource names.

https://developer.hashicorp.com/terraform/language/providers

Terraform providers

We need to tell Terraform to use the libvirt and ansible providers so we add the following to a file called main.tf

terraform {
  required_providers {
    libvirt = {
      source = "dmacvicar/libvirt"
    }
    ansible = {
      # version = "~> 1.0.0"
      source  = "ansible/ansible"
    }
  }
}

provider "libvirt" {
  # Configuration options
  # uri = "qemu:///system"
  # uri   = "qemu+ssh://root:password@kvm_host/system?sshauth=ssh-password"
}

Some notes on the above:

  • The version entries are not provided. This means that Terraform will fetch the latest version that is provided. We could optionally specify a version if we needed to.
  • In order to interact with the KVM server there are a couple of different options. You could SSH with a password, you could provide SSH keys, or you could be running terraform on the KVM host directly. Depending on your scenario, change the uri entry to a suitable value. Leaving a password in a configuration file is a bad idea so SSH keys is probably a better idea, or you can set the special LIBVIRT_DEFAULT_URI envirionment variable

export LIBVIRT_DEFAULT_URI="qemu+ssh://root:password@192.168.122.1/system?sshauth=ssh-password"

With that done, initialise terraform. This will download the Terraform providers you are going to use.

terraform init

We’ll now create three Terraform files. One for our generic cloud-init ISO image, one for our database server and one for our webserver.

SSH Keys for the servers

We’ll create a new SSH keypair for our servers, of type rsa:

ssh-keygen -t rsa

This creates a brand new private key in ~/.ssh/id_rsa and a public key in ~/.ssh/id_rsa.pub. This keypair will be referenced in the code below.

Cloud init

There are two parts to the cloud init configuration. First we need a terraform file (cloudinit.tf) that defines a cloud init ISO disk which we’ll attach to our KVM instance. We also need to include a cloudinit configuration file on the disk which defines what should be run when the server boots. The two files are shown below.

cloudinit.tf

# Define a cloudinit ISO disk

resource "libvirt_cloudinit_disk" "cloud-init-rhel9" {
  name = "commoninit.iso"
  pool = "guest_images_dir"
  user_data = data.template_file.user_data.rendered
}

data "template_file" "user_data" {
  template = file("${path.module}/cloud-init-rhel9.cfg")
}

cloud-init-rhel9.cfg

#cloud-config
users:
- name: ansible
  gecos: Ansible User
  groups: users,admin,wheel
  sudo: ALL=(ALL) NOPASSWD:ALL
  shell: /bin/bash
  lock_passwd: true
  ssh_authorized_keys:
    ## Use the key generated in the previous step
    - "ssh-rsa AAA...HSmn user1@yourdomain.com"

## ssh_pwauth: True
## ssh_authorized_keys:
##  - ssh-rsa AAA...SDvz user1@yourdomain.com
##  - ssh-rsa AAB...QTuo user2@yourdomain.com
chpasswd:
  list: |
    root:redhat
## cloud-user:redhat
  expire: False

In the above we’re creating an ‘ansible’ user with sudo privileges with SSH access allowed if the correct SSH keypair is provided.

Note that a number of lines are commented. These could allow SSH access for the cloud-user account with a password, allow access with keys, set a password on the root/cloud-user accounts and/or unlock them. In our example, we set the root password – this means you can access it on the console if needed.

Hint: Ensure that the file begins with the line #cloud-config – if it does not then cloud-init may not be able process the file. Similarly, check if you have items commented out and for correct yaml syntax. The format above should work.

Our first KVM server

Now we are ready to define our first server, which we will called ‘webserver’. To define this we create a webserver.tf terraform configuration file that does the following:

  • Creates a new qcow virtual disk based on our RHEL 9.1 image
  • Creates a VM server with our specified amount of storage and memory
  • Uses the cloudinit disk we created above. The cloud-init script will create an ansible user and allow access from our control host via the ~/.ssh/id_rsa key
  • Attaches the server to the correct network
  • Creates an Ansible resource linked to SSH credentials – this allows us to use the Terraform instance state as an inventory file using the Terraform plugin for Ansible.

webserver.tf

# Define a VM Volume

resource "libvirt_volume" "webserver-qcow2" {
  # What do name the virtual disk image
  name = "webserver.example.com.qcow2"
  # Where to store the virtual disk image
  pool = "guest_images_dir" # List storage pools using virsh pool-list
  # We build on a base image, defined by base_volume_name
  # found in base_volume_pool 
  base_volume_pool = "guest_images_dir"
  base_volume_name = "rhel-baseos-9.1-x86_64-kvm.qcow2"
  format = "qcow2"
}

# Define KVM domain to create
resource "libvirt_domain" "webserver" {
  name   = "webserver"
  memory = "2048"
  vcpu   = 2
  cpu {
    mode = "host-passthrough"
  }

  cloudinit = libvirt_cloudinit_disk.cloud-init-rhel9.id

  network_interface {
    network_name = "default" # List networks with virsh net-list
    wait_for_lease = true
  }

  disk {
    volume_id = "${libvirt_volume.webserver-qcow2.id}"
  }

  console {
    type = "pty"
    target_type = "serial"
    target_port = "0"
  }

  graphics {
    type = "vnc"
    listen_type = "address"
  }
}

# Define the host as an Ansible resource
resource "ansible_host" "webserver" {          #### ansible host details
  name = "webserver"
  groups = ["apache"]
  variables = {
    ansible_user                 = "ansible",
    ansible_ssh_private_key_file = "~/.ssh/id_rsa",
    ansible_python_interpreter   = "/usr/bin/python3"
    ansible_ssh_host   = "${libvirt_domain.webserver.network_interface[0].addresses.0}"
  }
}

Since we are going to use Ansible to manage the server once it is up and running, the cloud-init script will create an ansible user and allow access from our control host via the ~/.ssh/id_rsa key. We build the server to our specification, and finally add an ‘ansible_host’ resource. This allows us to use the Terraform instance state as an inventory file using the plugin.

Our second KVM server

We’ll create a second server called dbserver. It looks almost identical to the webserver execpt that the name is different and we specify a different amount of memory. We create a dbserver.tf terraform file to define the server:

dbserver.tf

# Define a VM Volume

resource "libvirt_volume" "dbserver-qcow2" {
  # What do name the virtual disk image
  name = "dbserver.example.com.qcow2"
  # Where to store the virtual disk image
  pool = "guest_images_dir" # List storage pools using virsh pool-list
  # We build on a base image, defined by base_volume_name
  # found in base_volume_pool 
  base_volume_pool = "guest_images_dir"
  base_volume_name = "rhel-baseos-9.1-x86_64-kvm.qcow2"
  format = "qcow2"
}

# Define KVM domain to create
resource "libvirt_domain" "dbserver" {
  name   = "dbserver"
  memory = "4096"
  vcpu   = 2
  cpu {
    mode = "host-passthrough"
  }

  cloudinit = libvirt_cloudinit_disk.cloud-init-rhel9.id

  network_interface {
    network_name = "default" # List networks with virsh net-list
    wait_for_lease = true
  }

  disk {
    volume_id = "${libvirt_volume.dbserver-qcow2.id}"
  }

  console {
    type = "pty"
    target_type = "serial"
    target_port = "0"
  }

  graphics {
    type = "vnc"
    listen_type = "address"
  }
}

# Define the host as an Ansible resource
resource "ansible_host" "dbserver" {          #### ansible host details
  name = "dbserver"
  groups = ["database"]
  variables = {
    ansible_user                 = "ansible",
    ansible_ssh_private_key_file = "~/.ssh/id_rsa",
    ansible_python_interpreter   = "/usr/bin/python3"
    ansible_ssh_host   = "${libvirt_domain.dbserver.network_interface[0].addresses.0}"
  }
}

Note that we are able to use the same cloudinit disk in both cases. The idea here is that Terraform brings our (virtual) servers up according to our base configuration and applies an Ansible account. Once provisioned, Ansible will then manage the host and configure it accordingly.

Ready to Terraform

At this point, you should have the following directory structure with the following 5 files:

.
├── cloud-init-rhel9.cfg
├── cloudinit.tf
├── dbserver.tf
├── main.tf
└── webserver.tf

Apply the Terraform configuration

With the configuration files in place, we are now ready to provision our two servers. Once nice thing about Terraform is you can review the changes before you apply them using the plan option. You can optionally save this plan with the --out option so you know exactly what infrastructure will change when you apply it. Be aware that if you chose to save the plan, any sensitive data may be included within it, so treat these files accordingly.

terraform plan --out=changes.tfplan
# or
terraform plan

You can now apply the changes as follows:

terraform apply changes.tfplan #if saved
# or
terraform apply

Two new virtual machines should be created. You should be able to login on the console as the root account. You should also be able to ssh as the ansible user using the ~/.id_rsa key

If you wish to see the state of the resources that have been deployed, run:

terraform show

Terraform Changes

One key thing about Terraform compared with Ansible, is that is maintains it’s own copy of what it thinks the state of the resources are. This provides some advantages – say you want to increase the memory of the database host. You can modify the dbserver configuration file and then run terraform plan to see what would happen. You should see that the dbserver will be replaced and the webserver will be left alone. For more background on terraform state and why it exists, see Terraform Documentation: Purpose of Terraform State

Ansible on the provisioned hosts

Like Terraform, Ansible functionality can also be extended, in this case by collections.

Collections are a distribution format for Ansible content that can include playbooks, roles, modules, and plugins. You can install and use collections through a distribution server, such as Ansible Galaxy, or a Pulp 3 Galaxy server.

https://docs.ansible.com/ansible/latest/collections_guide/index.html

We can use the open source Ansible Galaxy repository to install the cloud.terraform collection to enable Ansible and Terraform to work together. Installation can be performed as follows:

ansible-galaxy collection install cloud.terraform

Ansible can comsume the terraform state file using a plugin. We define an Ansible inventory.yml to consume the state:

inventory.yml

---
plugin: cloud.terraform.terraform_provider

We can straight away test whether Ansible can use the inventory plugin and communicate with our servers by performing:

ansible all -i inventory.yml -m ping

If all is well, we should see the following:

$ ansible all -i inventory.yml -m ping
webserver | SUCCESS => {
    "changed": false,
    "ping": "pong"
}
dbserver | SUCCESS => {
    "changed": false,
    "ping": "pong"
}

We’re almost done – now that Terraform and Ansible are working together, we want to use Ansible to manage the configuration of the servers. To do that, we’ll write a simple playbook configure_servers.yml

configure_servers.yml

---
- name: Configure servers according to their groups
  hosts: all
  tasks:
    - name: Set the hostname
      ansible.builtin.hostname:
        name: "{{ inventory_hostname }}"
        use: systemd
      become: true

    - name: Register server with Red Hat
      community.general.redhat_subscription:
        state: present
        username: joe_user
        password: somepass
      become: true

    - name: Install database software on DB servers
      ansible.builtin.dnf:
        name: mariadb-server
        state: present
      become: true
      when: inventory_hostname in groups["database"]

    - name: Enable database service on DB servers
      ansible.builtin.systemd:
        name: mariadb
        state: started
        enabled: true
      become: true
      when: inventory_hostname in groups["database"]

    - name: Install web server software on apache servers
      ansible.builtin.dnf:
        name: httpd
        state: present
      become: true
      when: inventory_hostname in groups["apache"]

    - name: Enable web service on apache servers
      ansible.builtin.systemd:
        name: httpd
        state: started
        enabled: true
      become: true
      when: inventory_hostname in groups["apache"]
...

The above first sets the hostname so we can distinguish them, we register them with the Red Hat portal (we could use an activation key, register them with Red Hat Satellite or manually configure yum/dnf repositories too if we needed) and finally we install, enable and start either apache or mariadb depending on whether the host is a database server or a web server.

As before, we need to first ensure that the community.general Ansible collection is installed:

ansible-galaxy collection install community.general

We can run the playbook as follows: 

ansible-playbook -i inventory.yml configure_servers.yml

Summary

We have used Terraform to deploy two servers with different specifications whilst applying a common Ansible user to each server. We’ve not had to worry ourselves with the IP addresses that have been allocated and don’t need to keep track of it. We’ve used the Terraform state file to consume details about the virtual machine in the form of an Ansible inventory file. Finally, we’ve used Ansible to configure the software on each server according to our requirements.

Room for improvement?

Looking at the webserver.tf and dbserver.tf files we see that they are very similar. How about we use Ansible to generate those files from templates and have Ansible call Terraform to instantiate that infrastructure? And how we can store the Terraform state? Stay tuned for the next post…

Leave a Reply

Your email address will not be published. Required fields are marked *