Bringing the Cloud to your Laptop: Provisioning Red Hat CoreOS VMs with a Secondary Disk in minutes on Mac OSX with QEMU+Ignition

4 min readJul 29, 2020

I have received requests to do another article tailored to provisioning Red Hat CoreOS VMs due to the usage of the operating system in Openshift 4. The steps are similar to those required to boot a Fedora CoreOS VM that are outlined in this article. Since they are similar, this walkthrough is going to outline a new addition: provisioning the Red Hat CoreOS VM with an additional virtual secondary disk. The additional disk can be used for extra storage or to create a dedicated encrypted file system for example (those steps will not be covered in this article).

To accomplish this goal, the QEMU virtual machine monitor for Mac needs to be installed so VMs can be launched. Brew can be used to install these packages:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)" 
brew install qemu
brew install jq

Now that those dependencies are installed the Red Hat CoreOS cloud image needs to be downloaded. Note the default cloud image only has a couple Gigabytes of primary disk space so the commands below will resize the primary disk to 30 Gigabytes to give the reader more disk space for any experiments he or she wants to run in the Red Hat CoreOS VM.

mkdir -p /tmp/rhcosqemu/imagecd /tmp/rhcosqemu/image
RHCOS_IMAGE_FILE=$(curl https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/4.4/latest/sha256sum.txt | grep qemu | awk '{print $2}')
DOWNLOAD_URL="https://mirror.openshift.com/pub/openshift-v4/dependencies/rhcos/4.4/latest/$RHCOS_IMAGE_FILE"
curl "$DOWNLOAD_URL" --output rhcos-qemu.x86_64.qcow2.xz
gunzip rhcos-qemu.x86_64.qcow2.xz
qemu-img resize /tmp/rhcosqemu/image/rhcos-qemu.x86_64.qcow2 30G

A secondary virtual disk can now be created using the qemu-img command. This tutorial will create a 30 Gigabyte secondary disk.

qemu-img create -f qcow2 /tmp/rhcosqemu/image/secondarydisk.qcow2 30G

Next, the Ignition configuration file needs to be generated. This file will be sent into the VM which the Ignition process will read to determine how to configure the machine when it first boots. The config generated below will add a public ssh key into the authorized keys file of the core user.

mkdir -p /tmp/rhcosqemu/ignitionmetadata
cd /tmp/rhcosqemu/ignitionmetadata
ssh-keygen -b 2048 -t rsa -f id_rsa_rhcosboot -P ""
chmod 0600 /tmp/rhcosqemu/ignitionmetadata/id_rsa_rhcosboot
PUBLIC_KEY=$(cat /tmp/rhcosqemu/ignitionmetadata/id_rsa_rhcosboot.pub)
cat <<EOF >/tmp/rhcosqemu/ignitionmetadata/ignitionconfig.ign
{
    "ignition": {
        "version": "2.2.0"
    },
    "passwd": {
        "users": [
            {
                "name": "core",
                "sshAuthorizedKeys": [
                    "${PUBLIC_KEY}"
                ]
            }
        ]
    }
}
EOF

Everything is now in place to boot the Red Hat CoreOS VM. The following command is going to pass the ignition metadata to the machine as a firmware configuration (fw_cfg) device and allow the reader to connect to the Red Hat CoreOS VM over ssh. The secondary disk will also be passed to the machine in the -hdbcommand line flag. The command below takes in the image, metadata, and specs of the VM (memory, CPU, network) and boots the machine up in the QEMU environment.

qemu-system-x86_64 -m 2048 -smp 4 -hda /tmp/rhcosqemu/image/rhcos-qemu.x86_64.qcow2 -hdb /tmp/rhcosqemu/image/secondarydisk.qcow2 -fw_cfg name=opt/com.coreos/config,file=/tmp/rhcosqemu/ignitionmetadata/ignitionconfig.ign -device e1000,netdev=net0 -netdev user,id=net0,hostfwd=tcp::5556-:22 -nographic

The VM is now beginning the boot process. Monitor the boot logs until messages similar to the following appear (exact values will change over time but the message format will be mostly the same).

Red Hat Enterprise Linux CoreOS 44.81.202004260825-0 (Ootpa) 4.4
SSH host key: SHA256:RRW8F0uqILyeTsxDJio8npMb1AiJ4f7nT1tU7SUrqSY (ECDSA)
SSH host key: SHA256:/JRSS/HOuXMQRDepBdC08qv/ulVkDAUutSt8yrITJTs (ED25519)
SSH host key: SHA256:buonHY71OBusBbaKX+jqncYds0cPVYREh7gSuodpBqk (RSA)
ens3: 10.0.2.15 fec0::fda9:60b6:9cea:fa05
localhost login:

Once those are logged, open a new terminal to ssh into the VM using the following command:

ssh  -o UserKnownHostsFile=/dev/null -p 5556 -i /tmp/rhcosqemu/ignitionmetadata/id_rsa_rhcosboot core@localhost
The authenticity of host '[localhost]:5556 ([127.0.0.1]:5556)' can't be established.
ECDSA key fingerprint is SHA256:RRW8F0uqILyeTsxDJio8npMb1AiJ4f7nT1tU7SUrqSY.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[localhost]:5556' (ECDSA) to the list of known hosts.
Red Hat Enterprise Linux CoreOS 44.81.202004260825-0
  Part of OpenShift 4.4, RHCOS is a Kubernetes native operating system
  managed by the Machine Config Operator (`clusteroperator/machine-config`).WARNING: Direct SSH access to machines is not recommended; instead,
make configuration changes via `machineconfig` objects:
  https://docs.openshift.com/container-platform/4.4/architecture/architecture-rhcos.html---
[core@localhost ~]$ [core@localhost ~]$ lsblkNAME                         MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                            8:0    0   30G  0 disk
├─sda1                         8:1    0  384M  0 part /boot
├─sda2                         8:2    0  127M  0 part /boot/efi
├─sda3                         8:3    0    1M  0 part
└─sda4                         8:4    0 29.5G  0 part
└─coreos-luks-root-nocrypt 253:0    0 29.5G  0 dm   /sysroot
sdb                            8:16   0   30G  0 disk
sr0                           11:0    1 1024M  0 rom[core@localhost ~]$

Congratulations! You now have a Red Hat CoreOS machine that you can run experiments in. The secondary disk appears as the sdb block device. If you are interested in learning more about Ignition or QEMU there is online documentation listed in the “Want to Learn More” section of this article.

Fully Automated Approach

Terminal window 1 run:

Terminal window 2 run after the VM has booted up:

ssh  -o UserKnownHostsFile=/dev/null -p 5556 -i /tmp/rhcosqemu/ignitionmetadata/id_rsa_rhcosboot core@localhost
The authenticity of host '[localhost]:5556 ([127.0.0.1]:5556)' can't be established.
ECDSA key fingerprint is SHA256:RRW8F0uqILyeTsxDJio8npMb1AiJ4f7nT1tU7SUrqSY.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[localhost]:5556' (ECDSA) to the list of known hosts.
Red Hat Enterprise Linux CoreOS 44.81.202004260825-0
  Part of OpenShift 4.4, RHCOS is a Kubernetes native operating system
  managed by the Machine Config Operator (`clusteroperator/machine-config`).WARNING: Direct SSH access to machines is not recommended; instead,
make configuration changes via `machineconfig` objects:
  https://docs.openshift.com/container-platform/4.4/architecture/architecture-rhcos.html---
[core@localhost ~]$ [core@localhost ~]$

Want to Learn More

QEMU Documentation: https://wiki.qemu.org/Documentation
Ignition Documentation: https://docs.fedoraproject.org/en-US/fedora-coreos/fcct-config/

Bringing the Cloud to your Laptop: Provisioning Red Hat CoreOS VMs with a Secondary Disk in minutes on Mac OSX with QEMU+Ignition

Fully Automated Approach

Want to Learn More

Written by Tyler Lisowski

No responses yet