Check out the new F5 Distributed Cloud Technical Knowledge Hub

Objective

F5® Distributed Cloud Services support site deployment for a Kernel-based Virtual Machine (KVM) with libvirt. This document explains how to perform Distributed Cloud Services node installation on a server using a KVM with libvirt and to perform site registration on F5® Distributed Cloud Console (Console).

The following is a sample topology for the hypervisor and guest machine setup. The sample shows three guest virtual machines installed with F5 Distributed Cloud Services Site software running on the hypervisor host.

Only one virtual machine is required. However, if you require a high-availability (HA) deployment, you must deploy three VMs.

Important: Each virtual machine (VM) must have the same cluster name and a unique hostname.

There are three ways to deploy a Secure Mesh Site on KVM. Review the prerequisites and then see the following sections for information:

Prerequisites

A Distributed Cloud Services Account. If you do not have an account, see Create an Account.
A server supporting hardware virtualization. The KVM only works if the server CPU has hardware virtualization support (Intel VT or AMD-V).
KVM installed and running.
At least one interface with Internet access. Distributed Cloud Services Node Zero-Touch Provisioning requires Internet connectivity to the Distributed Cloud Console.
A KVM image file. Click here to download.
Resources required per node: Minimum 4 vCPUs and 14 GB RAM.
45 GB is the minimum amount required for storage. However, to deploy an F5 Distributed Cloud App Stack Site, F5 recommends a minimum of 100 GB of storage.
To ensure intra-cluster communication checks, you must enable Internet Control Message Protocol (ICMP) between the Customer Edge (CE) nodes on the Site Local Outside (SLO) interfaces.
By proceeding with the installation, download and/or access and use, as applicable, of the Distributed Cloud Services software, and/or Distributed Cloud Services platform, you acknowledge that you have read, understand, and agree to be bound by this agreement.

Note: The following procedures are based on virt-install packages. You can also use virt-manager using the same parameters.

Create Secure Mesh Site

Before you deploy a single-node or multi-node site (for HA) on KVM, you must create a site in the F5 Distributed Cloud Console. For information, see Create Secure Mesh Site.

When you create your site, make sure to use the following settings:

Name: The cluster name. For HA deployments, each node must have the same cluster name.
Hostname: A unique hostname for each site. To configure an HA deployment, each site must have a different hostname.
Generic Server Certified Hardware: For single interface deployments, set to kvm-voltmesh. For two-interface (OUTSIDE, INSIDE) deployments, set to kvm-regular-nic-voltmesh.

Important: When you create a Secure Mesh Site in the Distributed Cloud Console, you must also create a site token. You use that same site token later when you deploy the Secure Mesh Site on KVM.

Deploy a Secure Mesh Site with an ISO Image

To deploy a Secure Mesh Site from the ISO image you downloaded from F5, you must complete the following high-level steps:

Spawn a virtual machine (VM) to host the Secure Mesh Site.
Install the Red Hat Linux 9 (RHEL9) operating system (OS) on the VM.
Provision Distributed Cloud configuration.
Register the site and complete Distributed Cloud configuration.

Spawn a VM

To spawn a VM from an ISO image that you downloaded from F5, enter the following commands.

          % qemu-img create /var/lib/libvirt/images/kvm-sms.qcow2 45G

% virt-install \
    --name kvm-sms \
    --vcpus 4 \
    --memory 14336 \
    --network network=default,model=virtio \
    --disk path=/var/lib/libvirt/images/kvm-sms.qcow2,bus=virtio,cache=none,size=45 \
    --cdrom /var/lib/libvirt/images/RHEL-9.2024.11-Installer.iso \
    --accelerate \
    --os-variant rhl9 \
    --virt-type kvm \
    --noautoconsole \
    --graphics vnc

Depending on how you want to deploy your site, you can also consider the following actions:

To deploy a site with two interfaces (OUTSIDE, INSIDE), add another network section.
To build an HA cluster, spawn two additional VMs.

Install the RHEL9 Operating System

The following steps explain how to install the RHEL9 OS on your VM with the ISO image that you downloaded from F5.

Step 1: Install RHEL9.

Connect to a VNC console and complete the following steps to select the language and disk. To connect, type: virsh vncdisplay kvm-sms.

For KVM deployments, you must select VDA as the disk type. You specify disk size when you spawn the VM.

Select the language you want to use.

Select the disk where you want to deploy the RHEL9.

Review your configuration settings.

Complete the installation.

Step 2: Restart the VM.

After the VM reboots, you must restart the VM.

To restart the VM, type: virsh start kvm-sms.

Important: If you are configuring an HA deployment, you must restart all of the nodes in the HA cluster.

Note: After you reboot the VM, it takes a few minutes to prepare the site shell. Wait a few minutes before you start provisioning Distributed Cloud configuration.

Provision Distributed Cloud Configuration

Step 1: Connect to the VM.

Do one of the following to connect to the VM:

To connect to the VM through a console window:

Open a console window and type virsh console kvm-sms:
When prompted, enter the following information:
- Username: admin
- Default password: Volterra123

To connect to the VM using SSH:

From a command prompt, type virsh domifaddr kvm-sms.
Type ssh admin@192.168.122.9.

Step 2: Change the VM admin password.

The first time you connect to the VM, you are prompted to change the KVM Secure Mesh Site admin password.

Step 3: Configure KVM Secure Mesh Site network.

To configure the KVM Secure Mesh Site network, enter the required configuration information.

After you provide the required network configuration information, the system automatically reboots.

Step 4: Configure KVM Secure Mesh Site deployment parameters.

Configure KVM Secure Mesh Site deployment parameters.

Important: You must use the same cluster name, hostname and certified hardware parameter values and the same site token that you used when you added the Secure Mesh Site in the Distributed Cloud Console.

Step 5: Register the site and complete Distributed Cloud configuration.

After you spawn the VM and deploy a Secure Mesh Site, see Register the Site to finish Distributed Cloud configuration.

Deploy a Secure Mesh Site with a QCOW2 Image

To deploy a Secure Mesh Site from a QCOW2 image, you must complete the following high-level steps:

Spawn a VM to host the Secure Mesh Site.
Register the site and complete Distributed Cloud configuration.

Templates for building files can be found under these links. This repository is publicly available.

Before you begin, you must edit the following variables in user-data.tpl to match your entries in your Secure Mesh Site:

cluster-name
host-name
latitude and longitude
xc-environment-api-endpoint: You must set this to ves.volterra.io
certified-hw: For single interface deployments, replace certified-hw with kvm-voltmesh. For two-interface (OUTSIDE, INSIDE) deployments, replace certified-hw with kvm-regular-nic-voltmesh.

Step 1: Spawn a VM to host the Secure Mesh Site.

To spawn a VM from a QCOW2 image, enter the following commands.

          % genisoimage -output cloud-init.iso -volid cidata -joliet -rock user-data meta-data

% cp rhel-9.2024.6-20240216073447.qcow2 /var/lib/libvirt/images/kvm-sms.qcow2

% virt-install \
    --name kvm-sms \
    --vcpus 4 \
    --memory 14336 \
    --network network=default,model=virtio \
    --disk /var/lib/libvirt/images/kvm-sms.qcow2,device=disk,bus=virtio,format=qcow2 \
    --disk cloud-init.iso,device=cdrom \
    --import \
    --os-variant rhl9 \
    --virt-type kvm \
    --accelerate \
    --noautoconsole \
    --graphics none

Step 2: Register the site and complete Distributed Cloud configuration.

After you spawn the VM and deploy a Secure Mesh Site, see Register the Site to finish Distributed Cloud configuration.

Deploy a Secure Mesh Site Using Terraform Automation (QCOW2)

This is the recommended way to deploy a KVM Secure Mesh Site. View the README file for deployment information: https://github.com/f5devcentral/f5-xc-kvm-ce.

The repository includes all automation needed to deploy a KVM Secure Site Mesh with one terraform apply command.

Register the Site

After the Distributed Cloud Services Node is installed, it must be registered as a site in the Distributed Cloud Console.

Note: The USB allowlist is enabled by default. If you change a USB device, such as a keyboard after registration, the device no longer functions.

Single-Node Site Registration

Step 1: Navigate to the site registration page.

Log in to the Distributed Cloud Console.
Click Multi-Cloud Network Connect.
Click Manage > Site Management > Registrations.

Step 2: Complete site registration.

Under Pending Registrations, find the name of your node and click the blue checkmark.
In the form that appears, fill in all required fields with the asterisk symbol (*).
If you did not previously, enter a latitude value and a longitude value.
Enter other configuration information, if needed.
Click Save and Exit.

Step 3: Check site status and health.

It might take a few minutes for the site registration information to update.

Click Sites and then click on your site name.
Click the Dashboard tab and verify the following information:
- The Update Status field has a Successful value for the Operating System section.
- The Update Status field has a Successful value for the F5 Software Status section.
- The Tunnel status and Control Plane fields under the Connectivity section have Up values.

Multi-Node Site (HA) Registration

Step 1: Navigate to the site registration page.

Log into Console.
Click Multi-Cloud Network Connect.
Click Manage > Site Management > Registrations.

Step 2: Accept the registration requests.

Under Pending Registrations, find the name of each node and then click the blue checkmark to accept the registration requests from each of your nodes.
Enter the same values for the following parameters for all the registration requests:
- In the Cluster name field, enter a name for the cluster. Make sure that all master nodes have the same name.
- In the Cluster size field, enter 3. Ensure that all master nodes have the same cluster size.
Enter all mandatory fields marked with the asterisk (*) character.