Deploy Secure Mesh Site v2 in OCI (ClickOps)

Objective

This guide provides instructions on how to create a customer edge (CE) Site using F5® Distributed Cloud Console to deploy to Oracle Cloud Infrastructure (OCI).

Important: This guide does not provide instructions on how to deploy an F5® App Stack Site.

Planning

Read the following documents before deploying a Secure Mesh Site in any provider environment:

• Understanding F5 Distributed Cloud - Customer Edge (CE)
• CE Datasheet
• CE Supported Platforms Guide
• Customer Edge Site Sizing Reference
• CE Performance Guide: Contact your account representative on CE performance-related information.
• Proxy for CE Registration and Upgrades Reference
• Secure Mesh Sites v2 Frequently Asked Questions
• Customer Edge Registration and Upgrade Reference
• F5 Customer Edge IP Address and Domain Reference for Firewall or Proxy Settings

General Prerequisites

The following general prerequisites apply:

• A Distributed Cloud Services Account. If you do not have an account, see Getting Started with Console.

• Resources required per node: Minimum 8 vCPUs, 32 GB RAM, and 80 GB disk storage. For a full listing of the resources required, see the Customer Edge Site Sizing Reference guide. All the nodes in a given CE Site should have the same resources regarding the compute, memory, and disk storage. When deploying in cloud environments, these nodes should use the same instance flavor.

• Allow traffic from and to the Distributed Cloud public IP addresses to your network and allowlist related domain names. See F5 Customer Edge IP Address and Domain Reference for Firewall or Proxy Settings guide for the list of IP addresses and domain names.

• F5 assumes that an existing subnet exists with Internet connectivity to attach to the node.

• The new Secure Mesh Site workflow enables you to have up to eight interfaces. However, these interfaces should be in different subnets. Therefore, make sure you have the required subnets available before creating the CE Site nodes.

• Internet Control Message Protocol (ICMP) needs to be opened between the CE nodes on the Site Local Outside (SLO) interfaces. This is needed to ensure intra-cluster communication checks.

Important: After you deploy the CE Site, the IP address for the SLO interface cannot be changed. Also, the MAC address cannot be changed.

IAM Required Roles

To perform the steps provided in this guide and launch a CE instance on OCI, you must be part of the account that has the following roles:

• compute-management-family
• virtual-network-family
• database-family
• object-family
• file-family
• instance-family
• app-catalog-listing

Allow group <GROUP_NAME> to manage compute-management-family in compartment <COMPARTMENT_NAME>
Allow group <GROUP_NAME> to manage virtual-network-family in compartment <COMPARTMENT_NAME>
Allow group <GROUP_NAME> to manage database-family in compartment <COMPARTMENT_NAME>
Allow group <GROUP_NAME> to manage object-family in compartment <COMPARTMENT_NAME>
Allow group <GROUP_NAME> to manage file-family in compartment <COMPARTMENT_NAME>
Allow group <GROUP_NAME> to manage instance-family in compartment <COMPARTMENT_NAME>
Allow group <GROUP_NAME> to read app-catalog-listing in tenancy

Configuration Overview

To create a Secure Mesh Site with OCI, here are the high-level steps:

• Site object configuration: Create and configure a Secure Mesh Site object using F5 Distributed Cloud Console.
• Node creation prerequisites: Create objects that are associated with the CE nodes, including network details, security lists, and more.
• Image management: Gather all the information required to find and load the custom OCI QCOW2 image.
• Node management: Use the OCI QCOW2 image to deploy CE nodes. Each CE node is a virtual machine (VM).
• Network interface management: Add additional interfaces on the nodes, if necessary.

Procedure

In this guide, the procedure demonstrates the steps to deploy a single-node Site with dual interfaces (ingress/egress). However, this guide also explains the necessary deviations from this specific model where necessary, making it flexible to adjust to different node and interface requirements.

Create Site Object

• Create a secure mesh Site object in Distributed Cloud Console. Refer to the Create Secure Mesh Site guide.

• Set the Provider Name option to OCI.

Figure: Provider Type

• For High Availability, choose an option. If it is Disabled, then the CE Site only supports one node. If it is Enabled, the CE Site requires three nodes. Additional nodes can only be added to CE sites when HA is Enabled.

Important: The High Availability mode cannot be changed after the CE Site is created.

• Leave the other options with their default values. These options have intelligent default values and do not need further configuration. Refer to the Create Secure Mesh Site guide for more information on these options.

• Click Add Secure Mesh Site.

Confirm Existing Virtual Cloud Network Details

In this procedure, a dual interface single-node site is being deployed. Therefore, you need two subnets: SLI (Site Local Inside) and SLO (Site Local Outside). Note that workload subnets are generally used but are not a requirement to deploy a CE Site.

• To view your virtual cloud network, navigate to Networking and select Virtual cloud networks.

Figure: View Existing Network

• Click on your virtual cloud network to see the available subnets.

Figure: View Existing Subnets

• View the following parameters:

  • Virtual Cloud Network name: oci-clickops-virtual-cloud-network
  • Site Local Outside (external) subnet: oci-clickops-external-subnet
  • Site Local Inside (internal) subnet: oci-clickops-internal-subnet

Create Security List

Create a security list that are attached to the F5 CE OCI instance. Note that this security list is used for the Site Local Outside (SLO) interface. Any other interfaces can use the default security list of the virtual cloud network, which should allow all traffic in both directions. By default, a security list only allows ICMP and SSH communication.

Step 1: Create security list.

• Navigate to your virtual cloud network and click Security Lists.

Figure: View Security Lists

• Click Create Security List.

Figure: Create Security List

• Enter an indicative name. This example uses f5-ce-security-list.

Step 4: Validate rules.

• Validate that the list was created successfully.

Figure: Validate Security List

• Navigate to the inbound/outbound rules to verify that they have been configured correctly.

Figure: Validate Ingress Rules

Figure: Validate Egress Rules

Step 5: Assign security list to subnet.

By adding the security list to a subnet, it includes those ingress/egress rules into its filtering policy.

• Navigate to Networking and select Virtual cloud networks.

• Click on the subnet unused for the Site Local Outside (SLO) interface.

Figure: List of Subnets

• Click Add Security List and select the custom security list created previously.

Figure: Select Custom Security List

• Confirm the security list is added to the desired subnet.

Figure: Confirm Security List Selected for Subnet

Download Node Image and Create OCI Compute Image

Download the qcow2 image from Distributed Cloud Console. This image file is uploaded in your OCI account.

Step 1: Create bucket in OCI.

• In Distributed Cloud Console, select the Multi-Cloud Network Connect workspace.

• Navigate to Manage > Site Management > Secure Mesh Sites v2.

• For your Site, click ... > Download Image.

Figure: Download Node Image

• After the image downloads, in the OCI Console, navigate to Storage > Buckets.

• Create a bucket to upload the node image file:

  • Click Create Bucket and give the bucket a name. This example uses smsv2-oci-bucket.

Figure: Create Bucket

Step 2: Upload image file to bucket.

• Click on the newly created bucket, and then click Upload.

• Select the node image file and then click Upload.

Figure: Upload Image File to Bucket

• Verify the image was properly uploaded by checking the Objects list.

Figure: Verify Image File in Bucket

Step 3: Create custom image file.

Create a custom image based on the qcow2 image.

• In the OCI Console, navigate to Compute > Custom Images.

• Click Import image.

Figure: Upload Custom Image File

• Use the following parameters:

  • For the Name, enter smsv2-oci-image.

  • For the Operating system, select RHEL.

  • Select Import from an Object Storage Bucket and specify the previously created bucket (smsv2-oci-bucket).

  • Select the previously uploaded qcow2 image.

  • For Image type, select QCOW2.

  • For Launch mode, select Paravirtualized mode.

Figure: Custom Image Parameters

• After you finish, click Import image. Wait for the image creation to be finalized. This process may take a few minutes.

Figure: Custom Image In-Progress

Figure: Custom Image Complete

Generate Node Token

A one-time node token is required to register a CE Site node to the Distributed Cloud Console. A new token must be generated for every new node in a CE Site. A token is valid for 24 hours. Make sure that the CE node is deployed soon after the token is generated.

The token is included in the cloud-init information under the Content variable. Also included are two variables commented out: slo_ip and slo_gateway. These variables can be commented out if you are using your own DNS service and not the default DNS service provided by F5.

• In Distributed Cloud Console, select the Multi-Cloud Network Connect workspace.

• Navigate to Manage > Site Management > Secure Mesh Sites v2.

• For your Site, click ... > Generate Node Token.

Figure: Node Token

• Click Copy cloud-init.

• Save the value locally. This token is used later. The token value is hidden for security purposes.

Figure: Copy Node Token

• Click Close.

• Generate one token per node you intend to deploy.

Create OCI VM Instance

Create the instance leveraging the previously created parameters.

Important: The name of the VM should not have "." in it. For example, the hostname can be node-0 or node0, but it cannot be node.f5.com since it is not supported. Your node VM name must adhere to DNS-1035 label requirements. This means the name must consist of lower case alphanumeric characters or “-“, start with an alphabetic character, and end with an alphanumeric character.

If configuring a multi-node Site, each node hostname must be unique.

Step 1: Create instance.

• To create the instance, click Create instance.

Figure: Create Instance

• Specify a name. This example uses smsv2-oci-instance.

• Choose the instance type. This example uses VM.Standard3.Flex with Intel and 2 OCPUs. Note that this is the minimum instance type required to run the F5 CE software.

Figure: Instance Type

Step 2: Configure instance network.

• Select the virtual cloud network and subnet (oci-clickops-external-subnet). The subnet chosen is the subnet for Network Interface 1 (Site Local Outside/external). You can decide on whether the IP address is allocated automatically or manually.

• Select the configuration for the private and public IP addresses.

Figure: Instance Network

Step 3: Configure instance advanced options.

• Select the Advanced options tab and change the launch options to Paravirtualized networking.

Figure: Instance Advanced Options

• Add your public SSH key to use after the site is provisioned. The username is cloud-user.

Figure: Instance SSH

• At the bottom of the page, Click Show advanced options to copy and paste a cloud init script.

• In the Cloud-init script text box, paste the cloud-init information (which includes the Site token) copied from the Generate Node Token section.

Figure: Cloud Init Script

• Click Create.

Create Internet Gateway for Default Route

When you create a VM with OCI, a default route table is also created, but it does not have connectivity to the public Internet. As such, you must create an Internet Gateway and add a route with 0.0.0.0/0 to the route table, with the Internet Gateway acting as the next hop.

The default route table should be attached to the SLO subnet only.

For more information about the Internet Gateway, see the official OCI guide.

Figure: Subnet Information

Figure: Create Internet Gateway

Figure: Attach Internet Gateway to Default Route

Verify CE Site Registration

• In Distributed Cloud Console, navigate to Multi-Cloud Network Connect > Overview > Infrastructure > Sites.

• Select the site. The Dashboard tab should clearly show that the CE Site has registered successfully with the System Health of 100% as well as Data Plane/Control Plane both being up.

Figure: Confirm Site Health

Note: For more information on the Site registration process, see the Customer Edge Registration and Upgrade Reference guide.

Manage Network Interfaces (Optional)

After your CE Site registers successfully, you might want to add additional network interfaces to meet your requirements. Ensure that you connect another network interface to the node VM.

Important: Adding a new network interface causes the data plane services to restart. Therefore, F5 strongly recommends that you perform this operation during maintenance windows. As data plane services restart, traffic drops are expected, as well as tunnels going down.

All nodes in a given CE Site should have the same number of network interfaces attached.

Each node in the CE Site should have interfaces with the same VRFs assigned. For example: If a CE Site has three nodes, each node having two interfaces - the first interface on each node is auto-configured to be in the SLO VRF (to connect to F5 Distributed Cloud). If the second interface on node-1 is in the SLI VRF, then the second interface on node-2 and node-3 should also be in the SLI VRF.

When adding interfaces, it is important to make sure that the interfaces are added to each node in the cluster. Nodes with non-homogenous interfaces within a CE Site might cause issues. Therefore, each node in a given CE Site should have the same number of interfaces placed in the same VRFs.

After you configure the SLO interface with a static IP address, DHCP will still be displayed in the Console. However, your static IP configuration is well taken into account. Also, remember that you cannot modify SLO parameters once the node is registered and deployed.

Add New Interface

For OCI sites, to add a new network interface to the CE Site, you need to use the same Virtual Cloud Network. This example uses subnet oci-clickops-internal-subnet.

Figure: Internal Subnet

• While the VM is up and running, navigate to Compute > Instances and click on the CE instance name.

• Click Attached VNICs and then click Create VNIC.

Figure: Create VNIC

• Optionally, specify a custom name for the new VNIC. Otherwise, a name is auto-generated by the system.

• Assign it to your Virtual Cloud Network and select the relevant subnet.

• Leave the other options with their default values.

Figure: Create VNIC

• Click on the new VNIC to review its configuration.

Figure: Review VNIC Configuration

Modify Interface Attributes

Once the VNIC is attached, you need to update the SLI configuration in Distributed Cloud Console. This process is valid if you need to modify any other interface attributes.

• While the VM is up and running, in Console, navigate to the Dashboard tab of your Secure Mesh Site.

Figure: View Dashboard

• Click Manage Configuration and then click Edit Configuration.

Figure: Edit Site

• Under the Nodes subsection, click the pencil icon under Actions to edit.

Figure: Edit Interface

• Choose one of the interfaces to edit by clicking the pencil icon. This example uses ens5.

Figure: Edit Interface

• Update the interface settings and specify the IP address associated with the new VNIC (for example, 10.0.1.203/24).

Figure: Edit Interface IP Address

• Save the changes.

• Confirm your Site to validate that the changes were applied successfully.

Figure: Confirm New VNIC

Day 2 Operations

• To monitor your Site, see the Monitor Site guide.
• To manage your Site software and OS updates, see the Manage Site guide.
• For troubleshooting issues, see the Troubleshooting Guide for Secure Mesh Site v2 Deployment guide. It provides step-by-step instructions to debug and resolve the issues that may arise due to registration and provisioning errors.
• For the latest on Distributed Cloud Services releases, see Changelogs.

Related Guides

To create a load balancer on the CE Site, see the HTTP Load Balancer or the TCP Load Balancer guides.

Concepts

• System Overview
• Core Concepts
• Networking
• F5 Distributed Cloud - Customer Edge