Deploy Secure Mesh Site v2 on Baremetal (ClickOps)

Published April 5, 2023 | Last modified February 24, 2025

Objective

This guide provides instructions on how to create a customer edge (CE) site using F5® Distributed Cloud Console to deploy on a baremetal server. For more information, see F5® Distributed Cloud - Customer Edge.

As part of the new site deployment workflow, you can deploy the CE site as a Secure Mesh Site using a baremetal server.

Important: This guide does not provide instructions on how to deploy an F5® App Stack Site. This functionality is in Early Access (EA) and can be used for PoC/PoV deployments. This will be made Generally Available (GA) over the next couple of releases. Please reach out to your account representative for more information.

General Prerequisites

The following general prerequisites apply:

A Distributed Cloud Services Account. If you do not have an account, see Getting Started with Console.
Resources required per node: Minimum 4 vCPUs, 14 GB RAM, and 80 GB disk storage. For a full listing of the resources required, see the Customer Edge Site Sizing Reference guide. All the nodes in a given CE Site should have the same resources regarding the compute, memory, and disk storage. When deploying in cloud environments, these nodes should use the same instance flavor.
Allow traffic from and to the Distributed Cloud public IP addresses to your network and allowlist related domain names. See Firewall and Proxy Server Allowlist Reference guide for the list of IP addresses and domain names.
F5 assumes that an existing subnet exists with Internet connectivity to attach to the node.
The new Secure Mesh Site workflow enables you to have up to eight interfaces. However, these interfaces should be in different subnets. Therefore, make sure you have the required subnets available before creating the CE Site nodes.
Internet Control Message Protocol (ICMP) needs to be opened between the CE nodes on the Site Local Outside (SLO) interfaces. This is needed to ensure intra-cluster communication checks.

Important: After you deploy the CE Site, the IP address for the SLO interface cannot be changed. Also, the MAC address cannot be changed.

Configuration Overview

To create a Secure Mesh Site, here are the high-level steps:

Site object creation: Configure the site within F5 Distributed Cloud Console.
Image management: Download node image.
Node management: Use the node image from previous step to boot and install required baremetal nodes.
Interface management: Add additional interfaces on the nodes, as necessary.

Procedure

The example below will deploy a single-node secure mesh site with dual interfaces. However, the procedure will also explain the optional deviations from this specific model, making it flexible to adjust to different node and interface requirements.

Create Site Object

Create a Secure Mesh Site.

Log into Console.
From the Console homepage, select the Multi-Cloud Network Connect service.
Click Manage > Site Management > Secure Mesh Sites.

Figure: Navigate to Secure Mesh Site

Click Add Secure Mesh Site.

Figure: Create New Secure Mesh Site

Enter a name for the new site.

Figure: Set Provider to Baremetal

Provider should be set to Baremetal. You will automatically see the Orchestration Mode set to Not Managed by F5XC.

Figure: Disable High Availability

For the High Availability option, refer to the Create Secure Mesh Site guide for information on the feature.
Apart from this, you can leave all settings with their default values.
Click Save and Exit to create the new site.
The ISO filename will follow this naming convention: f5xc-ce-<version>-securemeshv2-<timestamp>. For example, f5xc-ce-9.2024.22-securemeshv2-20240711-0205.iso. This downloaded ISO image will be used to bootstrap baremetal nodes.

Download Node Image

For the site object, under Actions, click ... > Copy Image Name to receive a download link to use for CLI with the curl or wget commands. To download the image file locally, click ... > Download Image.

Figure: Copy Image Name

The ISO filename will follow this naming convention: f5xc-ce-<version>-securemeshv2-<timestamp>. For example, f5xc-ce-9.2024.22-securemeshv2-20240711-0205.iso. This downloaded ISO image will be used to bootstrap baremetal nodes.

Connect CE Node hosted on Baremetal to F5 Distributed Cloud SaaS

The CE node(s) require connectivity to F5 Distributed Cloud using the public Internet. To facilitate this, the first interface associated with the CE node must be connected to an IPv4 subnet with connectivity to the public Internet. Traffic originating from this interface on a CE node must be allowed to access F5 Distributed Cloud services. Refer to the Firewall and Proxy Server Allowlist Reference for more information.

Generate Node Token

A one-time node token is required to register a CE Site node to the Distributed Cloud Console. A new token must be generated for every new node in a CE Site.

In Distributed Cloud Console, select the Multi-Cloud Network Connect service.
Navigate to Manage > Site Management > Secure Mesh Sites v2.
For your site, click ... > Generate Node Token.

Figure: Generate Node Token

Click Copy and save the value locally. This token will be used later in the Create Baremetal Node section. The token value is hidden for security purposes.

Figure: Copy Node Token

Create Baremetal Node

Important: The name of the VM should not have . in it. For example, the hostname can be node-0 or node0, but it cannot be node.f5.com since it is not supported. If configuring a multi-node site, each node hostname must be unique.

Step 1: Bootstrap the baremetal node.

Boot the node from the ISO image, and have console access available to monitor the installation. Accept the default boot option or simply wait for the timer to expire.

Figure: Boot the Node

The installer is fully automated, and you will be greeted with a login window once completed.

Figure: Node Login

Step 2: Provision the baremetal node.

Log in as user admin, with password Volterra123. You are asked to change this default password, followed by a choice of configuration options.
Use configure-network to configure the first interface with Internet access. Either manually set an IPv4 address or select DHCP.

Figure: Node Startup

The screenshot above shows NIC ens16f0np0 as primary outside NIC with DHCP and IPv6 auto-configuration enabled. Only set the HTTP_PROXY if you have your own proxy server deployed; otherwise, leave it empty to use the F5 Distributed Cloud proxy.
The next step requires copy-paste of the generated Node Token and this is best done over a network connection to the node. Use execcli ip a to display the network interfaces and assigned addresses:

Figure: Configure Node

Log into the node via SSH, run configure, and paste the node token generated earlier.

Figure: Node Final Configuration

Confirm configuration will start the registration process. No further interaction via CLI is required.

Step 3: Verify CE registration.

Navigate to Overview > Infrastructure > Sites and choose your new site from the Sites list at the bottom of the Sites dashboard.
The Site dashboard for your site should clearly show that CE has registered successfully with the System Health of 100 as well as Data Plane and Control Plane both being up.

Figure: Site Successfully Registered

Click the Infrastructure tab to see the Nodes and Interfaces with their IP addresses:

Figure: Node Infrastructure

Note: For more information on the site registration process, see the Customer Edge Registration and Upgrade Reference guide.

Add New Network Interface

Now that the CE has registered successfully, you might want to add additional interfaces to cater to different customer requirements.

Important: Adding a new network interface will cause the data plane services to restart. Therefore, it is strongly recommended to perform this operation during maintenance windows. As data plane services restart, traffic drops are expected, as well as tunnels going down. All nodes in a given CE site should have the same number of network interfaces attached. Each node in the CE site should have interfaces with the same VRFs assigned. For example: If a CE site has three nodes, each node having two interfaces - the first interface on each node will be auto-configured to be in the SLO VRF (to connect to F5 Distributed Cloud). If the second interface on node-1 is in the SLI VRF, then the second interface on node-2 and node-3 should also be in the SLI VRF.

When adding interfaces, it is important to make sure that the interfaces are added to each node in the cluster. Nodes with non-homogenous interfaces within a CE Site might cause issues. Therefore, each node in a given CE Site should have the same number of interfaces placed in the same VRFs.

Troubleshooting

For troubleshooting issues, see the Troubleshooting Guide for Secure Mesh Site v2 Deployment guide. It provides step-by-step instructions to debug and resolve the issues that may arise due to registration and provisioning errors.