Deploy Secure Mesh Site v2 in Nutanix (ClickOps)

Objective

This guide provides instructions on how to create a customer edge (CE) site using F5® Distributed Cloud Console to deploy a Linux Kernel-based Virtual Machine (KVM) on Nutanix cloud platform. For more information on a CE Site, refer to F5 Distributed Cloud - Customer Edge.

As part of the new site deployment workflow, you can deploy the CE site as a Secure Mesh Site using Nutanix.

Important: This guide does not provide instructions on how to deploy an F5® App Stack Site.

General Prerequisites

The following general prerequisites apply:

A Distributed Cloud Services Account. If you do not have an account, see Create an Account.
Resources required per node: Minimum 4 vCPUs, 14 GB RAM, and 80 GB disk storage. For a full listing of the resources required, see the Customer Edge Site Sizing Reference guide. All the nodes in a given CE Site should have the same resources regarding the compute, memory, and disk storage. When deploying in cloud environments, these nodes should use the same instance flavor.
Allow traffic from and to the Distributed Cloud public IP addresses to your network and allowlist related domain names. See Firewall and Proxy Server Allowlist Reference guide for the list of IP addresses and domain names.
F5 assumes that an existing IPv4 subnet exists with Internet connectivity to attach to the node.
The new Secure Mesh Site workflow enables you to have up to eight interfaces. However, these interfaces should be in different subnets. Therefore, make sure you have the required subnets available before creating the CE Site nodes.
Internet Control Message Protocol (ICMP) needs to be opened between the CE nodes on the Site Local Outside (SLO) interfaces. This is needed to ensure intra-cluster communication checks.

Important: After you deploy the CE Site, the IP address for the SLO interface cannot be changed. Also, the MAC address cannot be changed.

Configuration Overview

To create a Secure Mesh Site, here are the high-level steps:

Site object creation: Configure the site within F5 Distributed Cloud Console.
Image management: Gather all the information required to find the ISO installation image.
Node management: Use the qcow2 image from the previous step to boot and install on all nodes.
Interface management: Add additional interfaces on the nodes, if necessary.

Procedure

In this guide, the procedure demonstrates the steps to deploy a single-node site with dual interfaces (ingress/egress). However, this guide will also explain the necessary deviations from this specific model where necessary, making it flexible to adjust to different node and interface requirements.

Create Site Object

Create a secure mesh site object in Distributed Cloud Console. Refer to the Create Secure Mesh Site guide.
Set the Provider Name option to KVM. The Orchestration Mode is set automatically to Not Managed by F5XC (in other words, manual mode).

Figure: Provider Type

Confirm the High Availability (HA) setting. Refer to the Create Secure Mesh Site guide. For this procedure, High Availability is disabled since a single node is being deployed.
Leave the other options with default values. These options have intelligent default values and do not need further configuration. Refer to the Create Secure Mesh Site guide for more information on these options.
Click Save and Exit.
For the site object, under Actions, click ... > Copy Image Name to receive a download link to use for CLI with the curl or wget commands. To download the image file locally, click Download Image.

Figure: Copy or Download KVM Image

Note: The qcow2 filename will follow this naming convention: f5xc-ce-<version>-<timestamp>.qcow2. For example, f5xc-ce-9.2024.22-20240806132626.qcow2. This downloaded qcow2 image will be used to bootstrap KVM nodes on Nutanix.

Connect CE Node Hosted on KVM to F5 Distributed Cloud SaaS

The CE node(s) require(s) connectivity to F5 Distributed Cloud using the public Internet. To facilitate this, the first interface associated with the CE node must be connected to an IPv4 subnet with connectivity to the public Internet. Traffic originating from this interface on a CE node must be allowed to access F5 Distributed Cloud services. Refer to the Firewall and Proxy Server Allowlist Reference for more information.

Generate Node Token

A one-time node token is required to register a CE Site node to the Distributed Cloud Console. A new token must be generated for every new node in a CE Site. A token is valid for 24 hours. Make sure that the CE node is deployed soon after the token is generated.

In Distributed Cloud Console, select the Multi-Cloud Network Connect service.
Navigate to Manage > Site Management > Secure Mesh Sites v2.
For your site, click ... > Generate Node Token.

Figure: Node Token

Click Copy.
Save the value locally. This token will be used later. The token value is hidden for security purposes.

Figure: Copy Node Token

Click Close.
Generate one token per node you intend to deploy.

Create CE Image in Nutanix

Upload the downloaded qcow2 image into Nutanix PRISM to create the CE Site image.

Log into Nutanix PRISM cloud platform.
Select Settings from the upper left menu, then click ‘Image Configuration’ on the left pane.
Click Upload Image and either copy and paste the qcow2 image URL or upload the previously downloaded image from your computer.

Figure: Nutanix PRISM Image Configuration

Name the image. For example, use the base name of the URL without the .qcow2 suffix.

Figure: Nutanix PRISM Image Name

Bootstrap the Nutanix Node

In the Nutanix PRISM web UI, select VM from the upper left bar, and then click Create VM on the upper right side to create a new VM.

Figure: Nutanix PRISM VM Creation

Enter a name and optional description.
For compute resources, set at least 4 vCPUs and 16 GiB of memory (RAM). Keep default Boot Configuration (legacy BIOS).

Figure: Nutanix PRISM VM Resources

Scroll down to edit the Disks section and removeCD-ROM and add new disk via Clone from Image Service.
Select the qcow2 image file by name from the Image Services created in the previous section.

Figure: Nutanix PRISM VM Disk

Add at least one network interface that is connected to an existing IPv4 subnet with Internet access.

Figure: Nutanix PRISM VM Network Interface

Scroll further down and click Custom Script.
Copy-paste the following snippet into Type or Paste Script field, making sure to include the node token previously generated.

#cloud-config
write_files:
- path: /etc/vpm/user_data
  content: |
    token: <token>
  owner: root
  permissions: 0644

Copied!

Click Save.

Figure: Nutanix PRISM VM Cloud Config

Power on the VM.

Figure: Nutanix PRISM VM Power On

Verify CE Site Registration

In Distributed Cloud Console, navigate to Multi-Cloud Network Connect > Overview > Sites.
Select the site. The Dashboard tab should clearly show that the CE Site has registered successfully with the System Health of 100% as well as Data Plane/Control Plane both being up.

Figure: Confirm Site Health

Click the Infrastructure tab to see the Nodes and Interfaces with their IP addresses.

Manage Network Interfaces

After the CE Site is successfully registered, you may want to add additional network interfaces to cater to different customer requirements.

Important: Adding a new network interface will cause the data plane services to restart. Therefore, it is strongly recommended to perform this operation during maintenance windows. As data plane services restart, traffic drops are expected, as well as tunnels going down.

When adding interfaces, it is important to make sure that the interfaces are added to each node in the cluster. Nodes with non-homogenous interfaces within a CE Site might cause issues. Therefore, each node in a given CE Site should have the same number of interfaces placed in the same VRFs.

Nutanix allows adding and removing of interfaces dynamically on running instances. Follow Nutanix documentation for more information.

Troubleshooting

For troubleshooting issues, see the Troubleshooting Guide for Secure Mesh Site v2 Deployment guide. It provides step-by-step instructions to debug and resolve the issues that may arise due to registration and provisioning errors.