Deploy Secure Mesh Site v2 on KVM (ClickOps)
Objective
This guide provides instructions on how to create an F5® Distributed Cloud Customer Edge (CE) on KVM using the Secure Mesh Site (/docs/how-to/site-management/create-secure-mesh-site-v2) via F5 Distributed Cloud Console. For more information on a CE Site, refer to F5 Distributed Cloud - Customer Edge.
Important: This guide does not provide instructions on how to deploy an F5® App Stack Site.
General Prerequisites
The following general prerequisites apply:
-
A Distributed Cloud Services Account. If you do not have an account, see Create an Account.
-
Resources required per node: Minimum 4 vCPUs, 14 GB RAM, and 80 GB disk storage. For a full listing of the resources required, see the Customer Edge Site Sizing Reference guide. All the nodes in a given CE Site should have the same resources regarding the compute, memory, and disk storage. When deploying in cloud environments, these nodes should use the same instance flavor.
-
Allow traffic from and to the Distributed Cloud public IP addresses to your network and allowlist related domain names. See Firewall and Proxy Server Allowlist Reference guide for the list of IP addresses and domain names.
-
F5 assumes that an existing subnet exists with Internet connectivity to attach to the node.
-
The new Secure Mesh Site workflow enables you to have up to eight interfaces. However, these interfaces should be in different subnets. Therefore, make sure you have the required subnets available before creating the CE Site nodes.
-
Internet Control Message Protocol (ICMP) needs to be opened between the CE nodes on the Site Local Outside (SLO) interfaces. This is needed to ensure intra-cluster communication checks.
Important: After you deploy the CE Site, the IP address for the SLO interface cannot be changed. Also, the MAC address cannot be changed.
Configuration Overview
To create a CE Site with KVM, here are the high-level steps:
- Create site: Create a Secure Mesh Site using F5 Distributed Cloud Console.
- Download and upload image: Download the node image from the configured Secured Mesh Site using the Distributed Cloud Console, and then upload this image to KVM host.
- Deploy site: Utilize the CE node image and create required CE nodes on KVM.
- Add Interfaces: Create additional interfaces for each node.
Procedure
In this guide, the procedure demonstrates the steps to deploy single-node and multi-node secure mesh sites, with single and dual interfaces.
Create Site Object
-
Create a secure mesh site object in Distributed Cloud Console. Refer to the Create Secure Mesh Site guide.
-
Set the
Provider Name
option toKVM
.
Figure: Provider Type
-
Choose to enable or keep disabled the
High Availability
option. Refer to the Create Secure Mesh Site guide. -
Leave the other options with default values. These options have intelligent default values and do not need further configuration. Refer to the Create Secure Mesh Site guide for more information on these options.
-
Click
Save and Exit
. -
For the site object, under
Actions
, click...
>Copy Image Name
to receive a download link to use for CLI with thecurl
orwget
commands. To download the image file locally, clickDownload Image
.
Figure: Copy or Download KVM Image
Connect CE Node Hosted on KVM to F5 Distributed Cloud SaaS
The CE node(s) require(s) connectivity to F5 Distributed Cloud using the public Internet. To facilitate this, the first interface associated with the CE node must be connected to an IPv4 subnet with connectivity to the public Internet. Traffic originating from this interface on a CE node must be allowed to access F5 Distributed Cloud services. Refer to the Firewall and Proxy Server Allowlist Reference for more information.
Prepare Node VM User and Metadata
Generate Node Token
A one-time node token is required to register a CE Site node to the Distributed Cloud Console. A new token must be generated for every new node in a CE Site. A token is valid for 24 hours. Make sure that the CE node is deployed soon after the token is generated.
-
In Distributed Cloud Console, select the
Multi-Cloud Network Connect
service. -
Navigate to
Manage
>Site Management
>Secure Mesh Sites v2
. -
For your site, click
...
>Generate Node Token
.
Figure: Node Token
-
Click
Copy
. -
Save the value locally. This token will be used later. The token value is hidden for security purposes.
Figure: Copy Node Token
-
Click
Close
. -
Generate one token per node you intend to deploy.
Create User Data File
- Create a user data file using the node token. By default, the node name is auto-generated.
#cloud-config
write_files:
- path: /etc/vpm/user_data
content: |
token: <token>
owner: root
permissions: '0644'
Additional Parameters
Configure static IP for SLO.
Add lines to user data file.
slo_ip: <IP>/<prefix>
slo_gateway: <GW IP>
slo_dns: <DNS IP>
Full example:
#cloud-config
write_files:
- path: /etc/vpm/user_data
content: |
token: <token>
slo_ip: <IP>/<prefix>
slo_gateway: <GW IP>
slo_dns: <DNS IP>
owner: root
permissions: '0644'
Configure dedicated proxy server.
Add line to user data file.
proxy: http://<IP>:<port>
Full example:
#cloud-config
write_files:
- path: /etc/vpm/user_data
content: |
token: <token>
proxy: http://<IP>:<port>
owner: root
permissions: '0644'
Configure dedicated DNS.
Add line to user data file.
dns: <DNS IP>
Full example:
#cloud-config
write_files:
- path: /etc/vpm/user_data
content: |
token: <token>
dns: <DNS IP
owner: root
permissions: '0644'
Download CE Node Image
-
For the site object, under
Actions
, click...
>Copy Image Name
to receive a download link to use for CLI with thecurl
orwget
commands. -
To download the image file locally, click
Download Image
.
Important: The CE node image needs to be downloaded and can be used to deploy multiple nodes.
Create KVM Virtual Machine
Launch the node VM using the user data and qcow2 image files. Ensure the first network provided has Internet connectivity. One interface and two interface examples are provided below.
For a cluster (three-node), three virtual machines need to be created with separate user-data files, where each node must have a new node token generated.
Important: The name of the VM should not have
.
in it. For example, the hostname can benode-0
ornode0
, but it cannot benode.f5.com
since it is not supported.
One Interface Configuration
cp f5xc-ce-9.2024.22-20240722122430.qcow2 /var/lib/libvirt/images/kvm-sms-single.qcow2
virt-install --name kvm-sms-single \
--ram 16384 \
--vcpus=4 \
--network network=default,model=virtio \
--disk path=/var/lib/libvirt/images/kvm-sms-single.qcow2,bus=virtio,format=qcow2 \
--cloud-init user-data=user-data.txt \
--accelerate \
--os-variant rhl9 \
--virt-type kvm \
--noautoconsole \
--import \
--graphics vnc
Two Interface Configuration
cp f5xc-ce-9.2024.22-20240722122430.qcow2 /var/lib/libvirt/images/kvm-sms-single.qcow2
virt-install --name kvm-sms-single \
--ram 16384 \
--vcpus=4 \
--network network=default,model=virtio \
--network bridge=inside-br,model=virtio \
--disk path=/var/lib/libvirt/images/kvm-sms-single.qcow2,bus=virtio,format=qcow2 \
--cloud-init user-data=user-data.txt \
--accelerate \
--os-variant rhl9 \
--virt-type kvm \
--noautoconsole \
--import \
--graphics vnc
- To increase the qcow2 image disk size from a default of 43 GB, you can use the following command:
qemu-img resize /var/lib/libvirt/images/kvm-sms-single.qcow2 100G
- To extend the file system during boot, add the following to the user data file:
runcmd:
- [ sh, -c, test -e /usr/bin/fsextend && /usr/bin/fsextend || true ]
Note: After the VM boots up, there is no need to log in through Console to provide any additional commands. The node is identified and associated with the CE Site using the token. To view the deployment progress, navigate to your site and confirm
System Health
is 100%. It may take some time for the status to update fromWaiting for Registration
toProvisioning
.
Figure: Site System Health
Add Network Interface
After CE Site registers successfully, you might want to add additional network interfaces to cater to different customer requirements.
Important: Adding a new network interface will cause the data plane services to restart. Therefore, it is strongly recommended to perform this operation during maintenance windows. As data plane services restart, traffic drops are expected, as well as tunnels going down.
When adding interfaces, it is important to make sure that the interfaces are added to each node in the cluster. Nodes with non-homogenous interfaces within a CE Site might cause issues. Therefore, each node in a given CE Site should have the same number of interfaces placed in the same VRFs.
-
Power down the VM prior to adding any new interfaces or modifying any existing interfaces.
-
In the
Multi-Cloud Network Connect
service, clickManage
>Site Management
>Secure Mesh Sites
. -
For the KVM site, click
...
>Manage Configuration
. Then, in the wizard, clickEdit Configuration
.
Figure: Site Edit
Figure: Site Edit
- In the
Provider
section, click the pencil button to edit the desired node.
Figure: Node Edit
- Click the pencil button to edit the desired interface.
Figure: Node Edit Interface
-
From the
Interface Type
menu, select the interface type from the following options:Ethernet Interface
: This is the default option.VLAN Interface
: Choose parent interface and VLAN ID.Bond Interface
: Choose bond name, members, and bond mode.
Figure: Interface Type
-
From the
IPv4 Interface Address Method
menu, select the IP address configuration from the following options:DHCP Client
Static IP
DHCP Server
: For a multi-node site, this option must be configured separately one each node.Disabled
Figure: IP Address Configuration
Figure: IP Address Configuration Example
Important: The IP address for the SLO interface cannot be modified. This change can damage cluster configuration.
- Assign interface configurations for the
Select VRF
option to VRF. The default and most common option isSite Local Inside (Local VRF)
, but can also be assigned toSegment (Global VRF)
.
Figure: VRF Configuration
- To view the SLI interface, navigate to the
Infrastructure
tab. TheInterfaces
section provides the SLI information.
Figure: Interfaces View
Troubleshooting
For troubleshooting issues, see the Troubleshooting Guide for Secure Mesh Site v2 Deployment guide. It provides step-by-step instructions to debug and resolve the issues that may arise due to registration and provisioning errors.
Concepts
On this page:
- Objective
- General Prerequisites
- Configuration Overview
- Procedure
- Create Site Object
- Connect CE Node Hosted on KVM to F5 Distributed Cloud SaaS
- Prepare Node VM User and Metadata
- Generate Node Token
- Create User Data File
- Additional Parameters
- Download CE Node Image
- Create KVM Virtual Machine
- One Interface Configuration
- Two Interface Configuration
- Add Network Interface
- Troubleshooting
- Concepts