Deploy Secure Mesh Site v2 in Nutanix (ClickOps)
Objective
This guide provides instructions on how to create a customer edge (CE) site using F5® Distributed Cloud Console to deploy a Linux Kernel-based Virtual Machine (KVM) on Nutanix cloud platform. For more information on a CE Site, refer to F5 Distributed Cloud - Customer Edge.
As part of the new site deployment workflow, you can deploy the CE site as a Secure Mesh Site using Nutanix.
Important: This guide does not provide instructions on how to deploy an F5® App Stack Site. This functionality is in Early Access (EA) and can be used for PoC/PoV deployments. This will be made Generally Available (GA) over the next couple of releases. Please reach out to your account representative for more information.
General Prerequisites
The following general prerequisites apply:
-
A Distributed Cloud Services Account. If you do not have an account, see Create an Account.
-
Resources required per node: Minimum 4 vCPUs, 14 GB RAM, and 80 GB disk storage. For a full listing of the resources required, see the Customer Edge Site Sizing Reference guide. All the nodes in a given CE Site should have the same resources regarding the compute, memory, and disk storage. When deploying in cloud environments, these nodes should use the same instance flavor.
-
Allow traffic from and to the Distributed Cloud public IP addresses to your network and allowlist related domain names. See Firewall and Proxy Server Allowlist Reference guide for the list of IP addresses and domain names.
-
F5 assumes that an existing IPv4 subnet exists with Internet connectivity to attach to the node.
-
The new Secure Mesh Site workflow enables you to have up to eight interfaces. However, these interfaces should be in different subnets. Therefore, make sure you have the required subnets available before creating the CE Site nodes.
-
Internet Control Message Protocol (ICMP) needs to be opened between the CE nodes on the Site Local Outside (SLO) interfaces. This is needed to ensure intra-cluster communication checks.
Important: After you deploy the CE Site, the IP address for the SLO interface cannot be changed. Also, the MAC address cannot be changed.
Configuration Overview
To create a Secure Mesh Site, here are the high-level steps:
- Site object creation: Configure the site within F5 Distributed Cloud Console.
- Image management: Gather all the information required to find the ISO installation image.
- Node management: Use the qcow2 image from the previous step to boot and install on all nodes.
- Interface management: Add additional interfaces on the nodes, if necessary.
Procedure
In this guide, the procedure demonstrates the steps to deploy a single-node site with dual interfaces (ingress/egress). However, this guide will also explain the necessary deviations from this specific model where necessary, making it flexible to adjust to different node and interface requirements.
Create Site Object
-
Create a secure mesh site object in Distributed Cloud Console. Refer to the Create Secure Mesh Site guide.
-
Set the
Provider Name
option toKVM
. TheOrchestration Mode
is set automatically toNot Managed by F5XC
(in other words, manual mode).
Figure: Provider Type
-
Confirm the
High Availability
(HA) setting. Refer to the Create Secure Mesh Site guide. For this procedure,High Availability
is disabled since a single node is being deployed. -
Leave the other options with default values. These options have intelligent default values and do not need further configuration. Refer to the Create Secure Mesh Site guide for more information on these options.
-
Click
Save and Exit
. -
For the site object, under
Actions
, click...
>Copy Image Name
to receive a download link to use for CLI with thecurl
orwget
commands. To download the image file locally, clickDownload Image
.
Figure: Copy or Download KVM Image
Note: The qcow2 filename will follow this naming convention:
f5xc-ce-<version>-<timestamp>.qcow2
. For example,f5xc-ce-9.2024.22-20240806132626.qcow2
. This downloaded qcow2 image will be used to bootstrap KVM nodes on Nutanix.
Connect CE Node Hosted on KVM to F5 Distributed Cloud SaaS
The CE node(s) require(s) connectivity to F5 Distributed Cloud using the public Internet. To facilitate this, the first interface associated with the CE node must be connected to an IPv4 subnet with connectivity to the public Internet. Traffic originating from this interface on a CE node must be allowed to access F5 Distributed Cloud services. Refer to the Firewall and Proxy Server Allowlist Reference for more information.
Generate Node Token
A one-time node token is required to register a CE Site node to the Distributed Cloud Console. A new token must be generated for every new node in a CE Site. A token is valid for 24 hours. Make sure that the CE node is deployed soon after the token is generated.
The token is included in the cloud-init information under the Content
variable. Also included are two variables commented out: slo_ip
and slo_gateway
. These variables can be commented out if you are using your own DNS service and not the default DNS service provided by F5.
-
In Distributed Cloud Console, select the
Multi-Cloud Network Connect
service. -
Navigate to
Manage
>Site Management
>Secure Mesh Sites v2
. -
For your site, click
...
>Generate Node Token
.
Figure: Node Token
-
Click
Copy cloud-init
. -
Save the value locally. This token will be used later. The token value is hidden for security purposes.
-
Click
Close
. -
Generate one token per node you intend to deploy.
Create CE Image in Nutanix
Upload the downloaded qcow2 image into Nutanix PRISM to create the CE Site image.
Important: The name of the VM should not have
.
in it. For example, the hostname can benode-0
ornode0
, but it cannot benode.f5.com
since it is not supported. If configuring a multi-node site, each node hostname must be unique.
-
Log into Nutanix PRISM cloud platform.
-
Select Settings from the upper left menu, then click ‘Image Configuration’ on the left pane.
-
Click
Upload Image
and either copy and paste the qcow2 image URL or upload the previously downloaded image from your computer.
Figure: Nutanix PRISM Image Configuration
- Name the image. For example, use the base name of the URL without the
.qcow2
suffix.
Figure: Nutanix PRISM Image Name
Bootstrap the Nutanix Node
- In the Nutanix PRISM web UI, select
VM
from the upper left bar, and then clickCreate VM
on the upper right side to create a new VM.
Figure: Nutanix PRISM VM Creation
-
Enter a name and optional description.
-
For compute resources, set at least 4 vCPUs and 16 GiB of memory (RAM). Keep default
Boot Configuration
(legacy BIOS).
Figure: Nutanix PRISM VM Resources
-
Scroll down to edit the
Disks
section and removeCD-ROM
and add new disk viaClone from Image Service
. -
Select the qcow2 image file by name from the
Image Services
created in the previous section.
Figure: Nutanix PRISM VM Disk
- Add at least one network interface that is connected to an existing IPv4 subnet with Internet access.
Figure: Nutanix PRISM VM Network Interface
-
Scroll further down and click
Custom Script
. -
In the
Type or Paste Script
field, paste the cloud-init information (which includes the site token) copied from the Generate Node Token section. -
Click
Save
.
Figure: Nutanix PRISM VM Cloud Config
- Power on the VM.
Figure: Nutanix PRISM VM Power On
Verify CE Site Registration
-
In Distributed Cloud Console, navigate to
Multi-Cloud Network Connect
>Overview
>Infrastructure
>Sites
. -
Select the site. The
Dashboard
tab should clearly show that the CE Site has registered successfully with theSystem Health
of 100% as well asData Plane
/Control Plane
both being up.
Figure: Confirm Site Health
- Click the
Infrastructure
tab to see theNodes
andInterfaces
with their IP addresses.
Note: For more information on the site registration process, see the Customer Edge Registration and Upgrade Reference guide.
Manage Network Interfaces
After the CE Site is successfully registered, you may want to add additional network interfaces to cater to different customer requirements.
Important: Adding a new network interface will cause the data plane services to restart. Therefore, it is strongly recommended to perform this operation during maintenance windows. As data plane services restart, traffic drops are expected, as well as tunnels going down.
When adding interfaces, it is important to make sure that the interfaces are added to each node in the cluster. Nodes with non-homogenous interfaces within a CE Site might cause issues. Therefore, each node in a given CE Site should have the same number of interfaces placed in the same VRFs.
Nutanix allows adding and removing of interfaces dynamically on running instances. Follow Nutanix documentation for more information.
Troubleshooting
For troubleshooting issues, see the Troubleshooting Guide for Secure Mesh Site v2 Deployment guide. It provides step-by-step instructions to debug and resolve the issues that may arise due to registration and provisioning errors.