Deploy Secure Mesh Site v2 on F5 BIG-IP rSeries Appliance (ClickOps)

Objective

This guide provides instructions on how to create an F5® Distributed Cloud Customer Edge (CE) on an F5 rSeries Appliance using the Secure Mesh Site via F5 Distributed Cloud Console. For more information, see F5® Distributed Cloud Site.

Important:

• This guide does not provide instructions on how to deploy an F5 rSeries Appliance. For more information, see F5 rSeries Documentation.
• This guide does not provide instructions on how to deploy an F5® App Stack Site.

General Prerequisites

The following general prerequisites apply:

• A Distributed Cloud Services Account. If you do not have an account, see Create an Account.

• The F5 rSeries appliances should be running F5OS v1.8.0 or above.

• The F5 rSeries model should be from this list: 5600 / 5800 / 5900/ 10600 / 10800 / 10900 / 12600 / 12800 / 12900.

• Resources required per node: Minimum 4 vCPUs and 14 GB RAM. For a full listing of the resources required, see the Customer Edge Site Sizing Reference guide.

• F5 assumes that an existing IPv4 subnet exists with Internet connectivity to attach to node(s) to Distributed Cloud SaaS services.

• Each CE node supports up to eight (8) interfaces. Each interface is required to be in a different subnet. Therefore, make sure you have the required subnets available before creating the CE Site nodes.

• Internet Control Message Protocol (ICMP) needs to be opened between the CE nodes on the Site Local Outside (SLO) interfaces. This is needed to ensure intra-cluster communication checks.

Important: After you deploy the CE Site, the IP address for the SLO interface cannot be changed. Also, the MAC address cannot be changed.

Configuration Overview

To create a CE Site with F5 rSeries Appliance, here are the high-level steps:

1. Create site: Create a Secure Mesh Site using F5 Distributed Cloud Console.
2. Download and upload image: Download the node image from the configured Secured Mesh Site using the Distributed Cloud Console, and then upload this image to the F5 rSeries Appliance.
3. Deploy site: Utilize the CE image and create a CE node as a Tenant using the Generic Tenant Deployment capabilities of F5 rSeries.

Procedure

In this guide, the procedure demonstrates the steps to deploy single-node and multi-node secure mesh sites with single and dual interfaces.

Create Site Object

• Create a secure mesh site object in Distributed Cloud Console. Refer to the Create Secure Mesh Site guide.

• Set the Provider Name option to F5 rSeries. The Orchestration Mode sets to Not Managed by F5XC (also known as manual mode).

Figure: Provider Type

• For the High Availability option, refer to the Create Secure Mesh Site guide for information on the feature.

Figure: High Availability

• Leave the other options with default values. These options have intelligent default values and do not need further configuration. Refer to the Create Secure Mesh Site guide for more information on these options.

• Click Save and Exit.

Download CE Image

• For the site object, under Actions, click ... > Copy Image Name to receive a download link to use for CLI with the curl or wget commands. To download the image file locally, click Download Image. Alternatively, click Copy Image Name to receive a download link to use for CLI with the curl or wget commands.

Important: The CE image needs to be downloaded and can be used to deploy multiple nodes.

Figure: Copy or Download rSeries Image

• Verify the integrity of the image file using MD5SUM. See screenshot:

Figure: Verify Integrity of Image

Upload the Image to F5 rSeries Appliance

Upload the downloaded image and upload it to F5 rSeries appliances, where the nodes will be created.

Important: The CE image is signed and trusted by F5OS.

Step 1: Upload image.

• In F5OS Console, navigate to TENANT MANAGEMENT > Tenant Images.

• Click Upload.

Figure: F5OS Console

• Select the image file to upload.

Step 2: Confirm status.

Confirm the file transfer status.

Figure: F5OS Console

Figure: F5OS Console

Figure: F5OS Console

Create Node Tenant Deployment in F5 rSeries Appliance

• Navigate to TENANT MANAGEMENT > Tenant Images > Tenant Deployments.

• Click Add to create a CE tenant deployment. The CE node will be deployed as a VM on F5 rSeries.

• Select Generic as a Type for CE deployment.

Figure: Deployment

Connect CE Node hosted on F5 rSeries to F5 Distributed Cloud SaaS

The node's first interface must be connected to an IPv4 subnet with connectivity to the public Internet.

For deployment, the first interface will be used as an F5 proxy, which is automatically configured by the deployment automation process. There is no need any for further action, except to allow your firewall to connect to proxy IP addresses.

The CE nodes require connectivity to F5 Distributed Cloud using the public Internet. To facilitate this, the first interface associated with the CE node must be connected to an IPv4 subnet with connectivity to the public Internet. The CE nodes will be deployed as a Generic virtual machine on F5 rSeries appliances. Traffic originating from this interface on a CE node must be allowed to access F5 Distributed Cloud services. Refer to the Distributed Cloud Services Firewall and Proxy Server Allowlist Reference for more information.

Prepare Node VM Metadata

Prepare metadata that is required for the CE node to successfully bootstrap and connect to F5 Distributed Cloud.

Use the following format to send the metadata to the CE node VM:

[key1:value1 key2:value2]

There are two mandatory fields (keys) that need to be entered:

primary-vlan: Enter the VLAN ID of the first interface attached to this VM. This interface/VLAN should have access to the public Internet in order to connect to F5 Distributed Cloud.

token: Checkout a node token from F5 Distributed Cloud Console. See [#generate-node-token] section for details.

[primary-vlan:<VLAN_ID> token:<TOKEN>]

See sample configuration:

[primary-vlan:251 token:eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzaXRlX25hbWUiOiJjZS1vbi1mNS1yc2VyaWVzIiwidGVuYW50X25hbWUiOiJwbGF5Z3JvdW5kLXd0cHB2YW9nIiwidG9rZW5fdXVpZCI6ImQwMTE5ZjBkLTJkMDItNGUzYy1iZTZlLWYyYTNmMDEyMDVkMSIsImh0dHBfcHJveHkiOiIxMy41MC4yOS4yMzA6NDQzIiwicmVnaXN0cmF0aW9uX3VybCI6InN0YWdpbmcudm9sdGVycmEudXMiLCJpc3MiOiJGNSBYQyBTaXRlIE1hbmFnZXIiLCJzdWIiOiJGNSBYQyBTaXRlIFRva2VuIiwiZXhwIjoxNzIyNTM4NjE0LCJpYXQiOjE3MjI0NTIyMTR9.xsFPM0OPqIyd4RhTucSjFnxM6XGsVRH5S2W7pnsWXYZ]

The following optional fields (keys) can be sent:

slo_dns: Enter the DNS servers to be configured. For example, [primary-vlan:<VLAN_ID> token: <TOKEN> slo_dns:1.1.1.1,2.2.2.2]. Note that this option works when static IP is used. By default, the CE node will use 8.8.8.8 as the DNS server.

Sample CE Tenant Deployments

This section provides a sample configuration to create a CE node as a Generic tenant deployment on F5 rSeries (F5OS) GUI and CLI.

One interface configuration using GUI

Important: The first network provided (primary VLAN) must have Internet connectivity. Provide an IPv4 address. Currently, IPv6 is not supported for connecting the CE node to F5 Distributed Cloud.

• Select the CE image that was uploaded.
• Select a VLAN.
• If you are using DHCP, set DHCP to enable. If using static configuration, set DHCP to disabled and then provide IP Address, Prefix Length and Gateway.
• Ensure MAC address is set to small size (8).
• Select the vCPUs and set Virtual Disk size (minimum 50 GB). Refer to the Customer Edge Site Sizing Reference guide.
• Enter the metadata.
• Change state to Deployed.
• Click Save.

Figure: Interface Configuration with GUI

cat tenant.create.api.data.txt
{
    "tenant": [
        {
            "name": <CE_NODE_NAME>,
            "config": {
                "type": "GENERIC",
                "image": <UPLOADED_CE_IMAGE>,
                "nodes": [
                    1
                ],
                "dhcp-enabled": true,
                "dag-ipv6-prefix-length": 128,
                "vlans": [
                    251
                ],
                "vcpu-cores-per-node": 4,
                "memory": 14848,
                "storage": {
                    "size": 50
                },
                "cryptos": "enabled",
    "mac-data": {
                    "f5-tenant-l2-inline:mac-block-size": "small"
                },
                "running-state": "deployed",
                "f5-tenant-metadata:metadata": [
                    "primary-vlan:251",
"token:eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzaXRlX25hbWUiOiJzcmktcnNlcmllcy10ZXN0LTE0IiwidGVuYW50X25hbWUiOiJwbGF5Z3JvdW5kLXd0cHB2YW9nIiwidG9rZW5fdXVpZCI6IjFkMDRmY2Y2LWMwYzItNDg1ZS1hZDk4LWEwNmY1MGE3ZmI3NiIsImh0dHBfcHJveHkiOiIzNC4yMjYuOTguMjEzOjgwODAiLCJyZWdpc3RyYXRpb25fdXJsIjoic3RhZ2luZy52b2x0ZXJyYS51cyIsImlzcyI6IkY1IFhDIFNpdGUgTWFuYWdlciIsInN1YiI6IkY1IFhDIFNpdGUgVG9rZW4iLCJleHAiOjE3MjIzODA5NTIsImlhdCI6MTcyMjI5NDU1Mn0.VZwO0gSbD1AiQA-wbHep_DrVC7-93EEYOEZ4-7ggz3c"
                ]
            }
        }
    ]
}

curl -k -X POST  -H "Content-Type: application/yang-data+json" --data-binary "@<PATH_TO_CE_NODE_TENANT_DEPLOYMENT_CONFIG_FILE>" -H "Accept: application/yang-data+json" -u <admin:password> 'https://<F5_RSERIES_IP:PORT>/restconf/data/f5-tenants:tenants'

Two-interface configuration using GUI

Important: The first network/VLAN provided must have Internet connectivity. Provide an IPv4 address. Currently, IPv6 is not supported for connecting the CE node to F5 Distributed Cloud. The second network/VLAN can be used for other networks (SLI - Local VRF or Segment - Global VRF). Interfaces added after the first interfaces will default to the SLI VRF and will need to be configured from the site configuration from the F5 Distributed Cloud Console, after addition.

• Select the CE image that was uploaded.
• Select two VLANs: one to be used to connect the CE node with F5 Distributed Cloud (requiring Internet access) and another one used for local network.
• If you are using DHCP, set DHCP to enable. If using static configuration, set DHCP to disabled and then provide IP Address, Prefix Length and Gateway.
• Ensure MAC address is set to small size (8).
• Select the vCPUs and set Virtual Disk size (minimum 50 GB). Refer to the Customer Edge Site Sizing Reference guide.
• Enter the metadata.
• Change state to Deployed.
• Click Save.

Figure: Two-Interface Configuration with GUI

cat tenant.create.api.data.txt
{
    "tenant": [
        {
            "name": <CE_NODE_NAME>,
            "config": {
                "type": "GENERIC",
                "image": <UPLOADED_CE_IMAGE>,
                "nodes": [
                    1
                ],
                "dhcp-enabled": true,
                "dag-ipv6-prefix-length": 128,
                "vlans": [
                    251, 252
                ],
                "vcpu-cores-per-node": 4,
                "memory": 14848,
                "storage": {
                    "size": 50
                },
                "cryptos": "enabled",
    "mac-data": {
                    "f5-tenant-l2-inline:mac-block-size": "small"
                },
                "running-state": "deployed",
                "f5-tenant-metadata:metadata": [
                    "primary-vlan:251",
"token:eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzaXRlX25hbWUiOiJzcmktcnNlcmllcy10ZXN0LTE0IiwidGVuYW50X25hbWUiOiJwbGF5Z3JvdW5kLXd0cHB2YW9nIiwidG9rZW5fdXVpZCI6IjFkMDRmY2Y2LWMwYzItNDg1ZS1hZDk4LWEwNmY1MGE3ZmI3NiIsImh0dHBfcHJveHkiOiIzNC4yMjYuOTguMjEzOjgwODAiLCJyZWdpc3RyYXRpb25fdXJsIjoic3RhZ2luZy52b2x0ZXJyYS51cyIsImlzcyI6IkY1IFhDIFNpdGUgTWFuYWdlciIsInN1YiI6IkY1IFhDIFNpdGUgVG9rZW4iLCJleHAiOjE3MjIzODA5NTIsImlhdCI6MTcyMjI5NDU1Mn0.VZwO0gSbD1AiQA-wbHep_DrVC7-93EEYOEZ4-7ggz3c"
                ]
            }
        }
    ]
}

curl -k -X POST  -H "Content-Type: application/yang-data+json" --data-binary "@/Users/suser/Downloads/tenant.create.api.data.txt"  -H "Accept: application/yang-data+json" -u admin:password 'https://192.168.4.13:8888/restconf/data/f5-tenants:tenants'

• To view the status, navigate to Tenant Deployments.

Figure: View Tenant Deployment Status

Note: After deploying the CE node tenant, the CE node will be identified and associated with the created Secure Mesh site. Progress of deployment can be observed using the F5 Distributed Cloud UI. The admin state will change from Waiting for Registration to Provisioning. After some time, the site will have admin status as Online.

Figure: View Deployment Status

Generate Node Token

A one-time node token is required to register a CE Site node to the Distributed Cloud Console. A new token must be generated for every new node in a CE Site. A token is valid for 24 hours. Make sure that the CE node is deployed soon after the token is generated.

• In Distributed Cloud Console, select the Multi-Cloud Network Connect service.

• Navigate to Manage > Site Management > Secure Mesh Sites.

• For your site, click ... > Generate Node Token.

Figure: Node Token

• Click Copy.

• Save the value locally. This token will be used later. The token value is hidden for security purposes.

Figure: Copy Node Token

• Click Close.

• Generate one token per node you intend to deploy.

Add Additional Network Interface to Site

After the CE Site registers successfully, you might want to add additional interfaces to cater to different customer requirements.

Important: Adding a new network interface will cause the data plane services to restart. Therefore, it is strongly recommended to perform this operation during maintenance windows. As data plane services restart, traffic drops are expected, as well as tunnels going down.

All nodes in a given CE site should have the same number of network interfaces attached.

Each node in the CE site should have interfaces with the same VRFs assigned. For example: If a CE site has three nodes, each node having two interfaces - the first interface on each node will be auto-configured to be in the SLO VRF (to connect to F5 Distributed Cloud). If the second interface on node-1 is in the SLI VRF, then the second interface on node-2 and node-3 should also be in the SLI VRF.

Step 1: Attach an interface/VLAN to the CE node tenant deployment from F5 rSeries.

• Power down the VM prior to adding any new interfaces or modifying any existing interfaces.

• From F5 rSeries console, navigate to TENANT MANAGEMENT > Tenant Deployments.

• Edit the CE node tenant deployment and select additional interface(s)/VLAN(s) to attach to the node.

Figure: Attach Interface

• Click Save & Close.

• Repeat these steps for all CE node deployments associated with the CE site.

Step 2: Configure the newly attached interface/VLAN on the CE node from F5 Distributed Cloud.

• In the Multi-Cloud Network Connect service, click Manage > Site Management > Secure Mesh Sites.

• For the site, click ... > Manage Configuration. Then, in the wizard, click Edit Configuration.

Figure: Node Edit

• In the Provider section, click the pencil button to edit the desired node.

Figure: Node Edit

• Click Add Item to a second interface.

• In the Metadata section, enter a name for the new interface. Optionally, add a short description to identify this new interface.

Figure: New Interface Metadata

• In the Device Configuration section, from the Interface Type menu, select VLAN Interface. Enter enp2so as the Parent Interface.

Figure: New Interface Device Configuration

• Enter VLAN number for the VLAN ID. The VLAN number should be the same number as assigned to the servers which the interface will communicate with.

Figure: New Interface Device Configuration

• Configure the IPv4 Interface Address Method option from the following:

  • DHCP Client
  • Static IP
  • DHCP Server: For a multi-node site, this option must be configured separately one each node.

Figure: IP Address

Figure: IP Address

Important: The IP address for the SLO interface cannot be modified. This change can damage cluster configuration.

• Assign interface to a VRF via the Select VRF option. The default and most common option is Site Local Inside (Local VRF), but can also be assigned to Segment (Global VRF).

Note: A Segment VRF being a global VRF allows stretching networks across CE sites, making it easy to connect networks across data center/branch/public cloud locations.

Figure: VRF Configuration

• Click Apply.

Figure: Apply Configuration

• Click Save and Exit.

• Repeat these steps for additional interfaces.

• To view the SLI interface, navigate to the Infrastructure tab. The Interfaces section provides the SLI information.

Figure: Interfaces View

• Power up the VM.

Troubleshooting

For troubleshooting issues, see the Troubleshooting Guide for Secure Mesh Site v2 Deployment guide. It provides step-by-step instructions to debug and resolve the issues that may arise due to registration and provisioning errors.

Concepts

• System Overview
• Core Concepts
• Networking
• F5 Distributed Cloud - Customer Edge