In the previous post – Creating custom RHEL bootc disk images – we created a QCOW image from which we’ll create our Image Mode servers. This image is perfect for use with KVM, but we could have equally chosen ISO, AMI or VMDK disk images. We will use our newly build image to deploy onto three different environments – Dev/Test, Stage and Production.

Deployment Options

The deployment method you use will depend on your environment. For example, do all of the servers require static IP addresses and hostnames or are they assigned via DHCP? For deployment onto bare metal servers, do you have existing ways of provisioning them (eg PXE boot or remotely mounting ISO images)? Either way, you are likely to have an existing deployment mechanism for your environment. The disk image that you’ve created in the previous step can likely be incorporated into this process.

Deploying onto KVM

For a test lab, we’ll create 3 servers from our QCOW image. These will represent our Dev/Test, Stage and Production servers and also map to our Red Hat Satellite Lifecycle environment. We’ll name the servers as follows:

In our environment, we’ll use DHCP reservations for each of three servers based on static MAC addresses. We copy over the QCOW image into our KVM host and place them in our image storage area (/VMStorage). We’ll use the following script to bring up each of the three servers.

#!/bin/bash
# List of hosts
cat <<EOF > /tmp/buildhosts
bootc-dev,52:54:40:35:0a:62
bootc-stage,52:54:40:35:0a:64
bootc-prod,52:54:40:35:0a:65
EOF

cat /tmp/buildhosts | while read line
do
SERVER=`echo $line|awk -F',' '{print $1}'`.london.example.com
MAC=`echo $line|awk -F',' '{print $2}'`
virt-install --name ${SERVER} --memory 2048 \
--vcpu 1 --disk /VMStorage/${SERVER}.qcow2 --import \
--os-variant rhel9-unknown \
--network network=default,mac=${MAC} \
--unattended \
--noautoconsole
done

Fast Provisioning

The above only took a few seconds to run, and we’ve now got three custom Image Mode for RHEL servers up and running. We can use a web browser and see that our servers are providing content. Here is our bootc-prod.london.example.com server:

Same Image, Different Environments

You will recall that when we followed Creating custom RHEL bootc disk images the servers are all set to look for updates from the same source. In our case, this was from the container image satellite.london.example.com/example_organization-dev-test-rhel9-with-products-my-product-web-app

However, we have just built servers in Stage and Production environments so we need to modify where they look for updates. This can be achieved with a command such as:

bootc switch satellite.london.example.com/example_organization-stage-rhel9-with-products-my-product-web-app

Since we need to register the servers with Satellite and Insights, we can perform the task in the next step.

Registering the Image Mode servers

You may recall that as part of the container build process we create a service to register the servers to Red Hat Satellite and Red Hat Insights on boot. Let’s check the status of this rhc-connect.service:

[root@bootc-dev ~]# systemctl status rhc-connect.service 
× rhc-connect.service - Run subscription-manager and insights at boot
     Loaded: loaded (/usr/lib/systemd/system/rhc-connect.service; enabled; preset: disabled)
     Active: failed (Result: resources)
        CPU: 0

May 30 14:52:52 bootc-dev systemd[1]: rhc-connect.service: Failed to load environment files: No such file or directory
May 30 14:52:52 bootc-dev systemd[1]: rhc-connect.service: Failed to run 'start-pre' task: No such file or directory
May 30 14:52:52 bootc-dev systemd[1]: rhc-connect.service: Failed with result 'resources'.
May 30 14:52:52 bootc-dev systemd[1]: Failed to start Run subscription-manager and insights at boot.

The service has failed as it doesn’t have environment files. This makes sense – at the moment we’re still in the ‘provisioning’ phase of our builds. For each of our servers, we want a unique hostname and registration to Red Hat Satellite and Insights with some unique attributes (such as content view access and Insights tags). As we created an ‘ansible’ user in the QCOW disk we can apply the following playbook to complete the setup of our RHEL Image Mode hosts.

---
- name: Configure the bootc hosts
  hosts: bootc-dev.london.example.com,bootc-stage.london.example.com,bootc-prod.london.example.com
  gather_facts: false
  tasks:
    - name: Set hostname
      ansible.builtin.hostname:
        name: "{{ inventory_hostname }}"
        use: systemd
      notify: reboot server
    - name: Update Red Hat Insights Tags to reflect the environment
      ansible.builtin.lineinfile:
        path: /etc/insights-client/tags.yaml
        line: "app_environment: {{ env }}"
        create: yes
      notify: reboot server
    - name: Populate /etc/rhsm/.rhc_connect_credentials
      ansible.builtin.copy:
        dest: /etc/rhsm/.rhc_connect_credentials
        owner: root
        group: root
        mode: '0600'
        content: |
         RHC_ACT_KEY={{ activation_key}}
         RHC_ORG_ID=Example_Organization
         RHC_SERVER_URL=https://satellite.london.example.com:443/rhsm
      notify: reboot server
    - name: Ensure bootc is using correct Lifecycle Environment Registry
      ansible.builtin.shell: 
        "bootc switch {{ satellite_registry }}/{{ satellite_org }}-\
        {{ satellite_lifecycle }}-{{ satellite_cv }}-{{ satellite_product_repo }}"
      register: bootc_output
      changed_when: "'Queued' in bootc_output.stdout"
      notify: reboot server

  handlers:
    - name: reboot server
      ansible.builtin.reboot:

And our simple inventory file looks like this:

[all]
bootc-dev.london.example.com env=dev satellite_lifecycle=dev-test activation_key=RHEL9-Dev-Test
bootc-stage.london.example.com env=stage satellite_lifecycle=stage activation_key=RHEL9-Stage
bootc-prod.london.example.com env=prod satellite_lifecycle=prod-dr activation_key=RHEL9-Prod-DR

[all:vars]
satellite_registry=satellite.london.example.com
satellite_org=example_organization
satellite_cv=rhel9-with-products
satellite_product_repo=my-product-web-app

The Ansible playbook does the following:

• Sets the hostname to be the FQDN on each server
• Updates the Red Hat Insights app_environment tag to reflect the environment in which the server resides (dev, stage or prod)
• Adds credentials for connecting to our Satellite server and ensures the servers register in the correct content view
• Uses bootc switch to ensure that each server gets it’s image updates from the correct Lifecycle registry.

When we run the playbook, we connect as our ansible user and supply the password to the ansible account (-K) as we run the tasks as root via the -b flag:

ansible-playbook -i bootc_hosts.ini -u ansible -b configure_bootc.yml -K -D

After successfully running the above play, the configuration will be apply, the hosts rebooted and they will register with Red Hat Satellite and Red Hat Insights.

[root@bootc-dev ~]# systemctl status rhc-connect.service 
○ rhc-connect.service - Run subscription-manager and insights at boot
     Loaded: loaded (/usr/lib/systemd/system/rhc-connect.service; enabled; preset: disabled)
     Active: inactive (dead) since Thu 2024-05-30 15:59:10 UTC; 31s ago
    Process: 1210 ExecStart=/usr/bin/insights-client --register (code=exited, status=0/SUCCESS)
    Process: 2030 ExecStop=/bin/rm -f /etc/rhsm/.rhc_connect_credentials (code=exited, status=0/SUCCESS)
   Main PID: 1210 (code=exited, status=0/SUCCESS)
        CPU: 8.162s

May 30 15:58:32 bootc-dev.london.example.com insights-client[1222]: Automatic scheduling for Insights has been enabled.
May 30 15:58:33 bootc-dev.london.example.com insights-client[1263]: Starting to collect Insights data for bootc-dev.london.example.com
May 30 15:59:00 bootc-dev.london.example.com podman[1609]: 2024-05-30 15:59:00.340723374 +0000 UTC m=+0.289618102 system refresh
May 30 15:59:08 bootc-dev.london.example.com insights-client[1263]: Writing RHSM facts to /etc/rhsm/facts/insights-client.facts ...
May 30 15:59:08 bootc-dev.london.example.com insights-client[1263]: Uploading Insights data.
May 30 15:59:09 bootc-dev.london.example.com insights-client[1263]: Successfully uploaded report from bootc-dev.london.example.com to account XXXXXXX.
May 30 15:59:09 bootc-dev.london.example.com insights-client[1263]: View the Red Hat Insights console at https://console.redhat.com/insights/
May 30 15:59:10 bootc-dev.london.example.com systemd[1]: rhc-connect.service: Deactivated successfully.
May 30 15:59:10 bootc-dev.london.example.com systemd[1]: Finished Run subscription-manager and insights at boot.
May 30 15:59:10 bootc-dev.london.example.com systemd[1]: rhc-connect.service: Consumed 8.162s CPU time.

We can see the full logs for the service by using journalctl. This shows the first boot failure, but second boot success.

[root@bootc-dev ~]# journalctl -u rhc-connect.service 
May 30 14:52:52 bootc-dev systemd[1]: rhc-connect.service: Failed to load environment files: No such file or directory
May 30 14:52:52 bootc-dev systemd[1]: rhc-connect.service: Failed to run 'start-pre' task: No such file or directory
May 30 14:52:52 bootc-dev systemd[1]: rhc-connect.service: Failed with result 'resources'.
May 30 14:52:52 bootc-dev systemd[1]: Failed to start Run subscription-manager and insights at boot.
-- Boot 1d8639753edf4bdda4d23283162e4007 --
May 30 15:58:18 bootc-dev.london.example.com systemd[1]: Starting Run subscription-manager and insights at boot...
May 30 15:58:20 bootc-dev.london.example.com subscription-manager[678]: Registered system with identity: 31b0bf6b-2eca-4e09-XXXX-XX
May 30 15:58:21 bootc-dev.london.example.com subscription-manager[678]: Added subscription for 'Content Access' contract 'None'
May 30 15:58:21 bootc-dev.london.example.com subscription-manager[678]: Added subscription for product ' Content Access'
May 30 15:58:21 bootc-dev.london.example.com subscription-manager[678]: The system has been registered with ID: 31b0bf6b-2eca-4e09-8da5-XX
May 30 15:58:21 bootc-dev.london.example.com subscription-manager[678]: The registered system name is: bootc-dev.london.example.com
May 30 15:58:32 bootc-dev.london.example.com insights-client[1222]: Successfully registered host bootc-dev.london.example.com
May 30 15:58:32 bootc-dev.london.example.com insights-client[1222]: Automatic scheduling for Insights has been enabled.
May 30 15:58:33 bootc-dev.london.example.com insights-client[1263]: Starting to collect Insights data for bootc-dev.london.example.com
May 30 15:59:00 bootc-dev.london.example.com podman[1609]: 2024-05-30 15:59:00.340723374 +0000 UTC m=+0.289618102 system refresh
May 30 15:59:08 bootc-dev.london.example.com insights-client[1263]: Writing RHSM facts to /etc/rhsm/facts/insights-client.facts ...
May 30 15:59:08 bootc-dev.london.example.com insights-client[1263]: Uploading Insights data.
May 30 15:59:09 bootc-dev.london.example.com insights-client[1263]: Successfully uploaded report from bootc-dev.london.example.com to account XXXXXXX.
May 30 15:59:09 bootc-dev.london.example.com insights-client[1263]: View the Red Hat Insights console at https://console.redhat.com/insights/
May 30 15:59:10 bootc-dev.london.example.com systemd[1]: rhc-connect.service: Deactivated successfully.
May 30 15:59:10 bootc-dev.london.example.com systemd[1]: Finished Run subscription-manager and insights at boot.
May 30 15:59:10 bootc-dev.london.example.com systemd[1]: rhc-connect.service: Consumed 8.162s CPU time.

Why register the Image Mode servers to Red Hat Satellite?

An Image Mode for RHEL server has no requirement to connect to a Red Hat Satellite server or the Red Hat Subscription Management service. Nor does it need to connect to Red Hat Insights or a container registry (until you need to apply updates). One of Satellite’s great features is performing as a Content Hub, so using it as a registry allows for servers to receive curated, approved content according to business Lifecycle requirements without relying on Internet connectivity.

Connecting servers to both Satellite and Insights allows us some monitoring capabilities, and if we are using these tools for package mode installations, why not extend these to image mode too? For cases where servers don’t have direct network access to the Internet routing Insights traffic via a Satellite server makes sense.

Summary

In this post we’ve shown how we can create servers from the disk image created in the previous post. We’ve shown how we can use automation to provide some initial configuration of the servers and have them connect to our existing Red Hat Satellite infrastructure. Following this initial setup, we expect that configuration changes will apply by updating the images. Again, this could be triggered automatically via bootc, automated via Ansible or performed manually.

Now that the build and configuration of our Image Mode for RHEL servers is complete – we can take advantage of Red Hat Satellite and Insights to monitor them:

• Managing Image Mode for RHEL with Satellite and Insights

Note

This post is not endorsed or affiliated with Red Hat – the information provided is based on experience, documentation and publicly available information. Feel free to leave feedback at the end of this page if anything needs correction.
For an up to date roadmap discussion on Image Mode for RHEL please contact your Red Hat Account rep.