Deploy Secure Mesh Site v2 in GCP (ClickOps)

Objective

This guide provides instructions on how to create a customer edge (CE) site using F5® Distributed Cloud Console and Google Cloud Platform (GCP) Console and deploy to a GCP virtual private cloud (VPC). For more information on a CE Site, refer to F5 Distributed Cloud - Customer Edge.

As part of the new site deployment workflow, you can deploy the CE site as a Secure Mesh Site to a GCP VPC.

Important: This guide does not provide instructions on how to deploy an F5® App Stack Site.

General Prerequisites

The following general prerequisites apply:

• A Distributed Cloud Services Account. If you do not have an account, see Create an Account.

• An account with GCP with permissions to create objects in Compute Engine, VPC Networks, Network Services, and Cloud Storage services. See Required Access Policies for permissions needed to deploy site.

• A GCP Storage Bucket where the CE image file can be uploaded.

• Resources required per node: Minimum 4 vCPUs, 14 GB RAM, and 80 GB disk storage. For a full listing of the resources required, see the Customer Edge Site Sizing Reference guide. All the nodes in a given CE Site should have the same resources regarding the compute, memory, and disk storage. When deploying in cloud environments, these nodes should use the same instance flavor.

• Allow traffic from and to the Distributed Cloud public IP addresses to your network and allowlist related domain names. See Firewall and Proxy Server Allowlist Reference guide for the list of IP addresses and domain names.

• The new Secure Mesh Site workflow enables you to have up to eight (8) interfaces. However, these interfaces should be in different VPCs. Therefore, make sure you have the required VPC with subnets with non-overlapping CIDRs available before creating the CE Site nodes.

• Internet Control Message Protocol (ICMP) needs to be opened between the CE nodes on the Site Local Outside (SLO) interfaces. This is needed to ensure intra-cluster communication checks.

Important: After you deploy the CE Site, the IP address for the SLO interface cannot be changed. Also, the MAC address cannot be changed.

Configuration Overview

To create a Secure Mesh Site, here are the high-level steps:

1. Site object creation: Configure the site within F5 Distributed Cloud Console.
2. Node creation prerequisites: Create objects that will be associated to the nodes (VM instances) including image download and token generation from Distributed Cloud Console, and VM image, VPCs, subnets, firewall rules, external IP addresses, and more on GCP.
3. Node management: Create the VM instances for the CE Site nodes.
4. Interface management: Add additional interfaces on the nodes, if necessary.

Procedure

In this guide, the procedure demonstrates the steps to deploy a single-node secure mesh site with dual interfaces. However, this guide will also explain the necessary deviations from this specific model where necessary, making it flexible to adjust to different node and interface requirements.

Create Site Object

• Create a secure mesh site object in Distributed Cloud Console. Refer to the Create Secure Mesh Site guide.

• Set the Provider Name option to GCP.

Figure: Provider Type

Create VPC Networks and Subnets

Create two VPC networks, with one subnet each for the SLO and SLI interfaces. Note that this procedure creates a two-interface CE Site.

Create the SLO VPC and Subnet

• In GCP Console, navigate to VPC networks and click CREATE VPC NETWORK.

Figure: SLO Subnet

• In the Name field, specify the new network name. This example uses vpc-smsv2-slo.

Figure: SLO Subnet Name

• For Subnet creation mode, select Custom.

• Click New subnet.

Figure: SLO Subnet Custom

• Enter a subnet name, select the region, and ensure IPv4 (single-stack) is selected.

• Enter an IPv4 range.

Figure: SLO Subnet Custom

• Skip the firewall rule configurations, as this will be configured in the next section.

• Keep the rest of the default options.

• Click CREATE.

Create the SLI VPC and Subnet

Repeat the previous steps to create the SLI VPC and subnet. However, use the following parameters:

• VPC name: vpc-smsv2-sli
• Subnet name: sub-smsv2-sli
• IPv4 range: 10.10.2.0/24

Add Additional VPCs and Subnets

If you need more than two interfaces for your CE Site, you must create new VPCs and subnets for each network interface. This is optional. The CE Site does not require additional interfaces for regular operations.

To create new interfaces, repeat the steps above and use a non-overlapping IP address range. As an example, you can use the following parameters:

• VPC name: vpc-smsv2-sli2
• Subnet name: sub-smsv2-sli2
• IPv4 range: 10.10.3.0/24

Configure Firewall Rules

The CE data path automatically manages the ports and protocols allowed on the interfaces. Therefore, you need to create allow-all rules to use with the CE Site nodes.

• In GCP Console, navigate to VPC networks and click on the new SLO network created previously.

Figure: SLO Subnet Selection

• In the Firewall tab, click ADD FIREWALL RULE and create any-to-any allow-all traffic ingress and egress rules as shown in image. Make sure you add a tag (for example, smsv2) to match the Targets where the rule will be applied. The network interfaces of the CE Site node will be configured with the same tag to apply the rules to it.

Figure: SLO Firewall Rules

• Repeat the steps above for the SLI VPC network and add allow-all rules.

Figure: SLI Firewall Rules

Import CE Site Image

In Distributed Cloud Console, you can either download the node image file or copy its image name. The image file can be used to create multiple nodes in the same GCP region.

• Navigate to Manage > Site Management > Secure Mesh Sites v2.

• For your site, click ... > Download Image.

Figure: Download Node Image

• In GCP Console, navigate to Storage Bucket and then click UPLOAD FILES to upload the node image file.

Figure: Upload Node Image

• Navigate to Compute Engine > Images. Click CREATE IMAGE.

Figure: Create Node Image in GCP Console

• Enter a name for the image and select the Source as Cloud Storage File.

• Click Browse to select the image file from Storage Bucket.

Figure: Create Node Image in GCP Console

• Leave the rest of the options in their default configurations.

• Click CREATE.

Generate Node Token

A one-time node token is required to register a CE Site node to the Distributed Cloud Console. A new token must be generated for every new node in a CE Site. A token is valid for 24 hours. Make sure that the CE node is deployed soon after the token is generated.

• In Distributed Cloud Console, select the Multi-Cloud Network Connect service.

• Navigate to Manage > Site Management > Secure Mesh Sites v2.

• For your site, click ... > Generate Node Token.

Figure: Node Token

• Click Copy.

• Save the value locally. This token will be used later. The token value is hidden for security purposes.

Figure: Copy Node Token

• Click Close.

Create User Data File

• Create a user data file using the node token.

#cloud-config
write_files:
- path: /etc/vpm/user_data
  content: |
    token: <token>
  owner: root
  permissions: '0644'

Create GCP Virtual Machine

Create the virtual machine (VM) using the previously created parameters.

Important: The name of the VM should not have . in it. For example, the hostname can be node-0 or node0, but it cannot be node.f5.com since it is not supported.

• In GCP Console, navigate to Compute Engine > Images.

• Click on the name of the image previously created.

• Click CREATE INSTANCE.

Figure: Create Instance

• Enter a name for the VM.

• Select the region and where you want to deploy the node.

Figure: Create Instance

• Select the VM machine type. Recommended instance types are:

  • n1-standard-4 (4 vCPU, 15 GB RAM)
  • n1-standard-8 (8 vCPU, 30 GB RAM)
  • n1-standard-16 (16 vCPU, 60 GB RAM)

Figure: Select Instance Type

• Under Boot disk, Click CHANGE and make the following changes:

  • Set the Boot disk type to Standard persistent disk.
  • Set the Size (GB) to 80 GB. The minimum required is 45 GB.

Figure: Select Instance Size

• Click on Advanced options to expand the configuration.

• Under Networking, add network tags to match the tags for the firewall rules.

• Do not add a hostname. Keep this field empty.

• Enable IP forwarding.

Figure: Select Instance Tags

• Under Network interfaces expand the default interface and configure it as the SLO interface of the node with the following:

  • Select the SLO VPC network.
  • Select the SLO subnet.
  • Select Ephemeral (Automatic) for the Primary internal IPv4 address.

Figure: Configure Instance SLO Example

• For External IPv4 address, click on the drop-down menu. Click Reserve Static External IP address, provide a name, and then click Reserve.

Figure: Configure Instance SLO

• Click ADD A NETWORK INTERFACE to configure the SLI for the node as below:

  • Select the SLI VPC network.
  • Select the SLI subnet.
  • Select Ephemeral (Automatic) for the Primary internal IPv4 address.
  • For External IPv4 address, select None.

Figure: Configure Instance SLI Example

Note: If additional interfaces are required, refer to the [#add-new-network-interface] section below before proceeding. You cannot add additional interfaces after the instance is created.

• Under the Security section, expand Manage Access.

• Under Add manually generated SSH keys, click ADD ITEM and paste your SSH public key in the text box. This will be used to SSH into the node for debugging, if required.

Figure: Configure SSH

• Under the Management section, click Metadata. Click ADD ITEM and use the following information to add two metadata key-value pairs:

Figure: Configure Metadata

• Click CREATE to create the VM.

Verify CE Site Registration

• In Distributed Cloud Console, navigate to Multi-Cloud Network Connect > Overview > Sites.

• Select the site. The Dashboard tab should clearly show that the CE Site has registered successfully with the System Health of 100% as well as Data Plane/Control Plane both being up.

Figure: Confirm Site Health

Add New Network Interface

Important: Adding a new network interface will cause the data plane services to restart. Therefore, it is strongly recommended to perform this operation during maintenance windows. As data plane services restart, traffic drops are expected, as well as tunnels going down.

When adding interfaces, it is important to make sure that the interfaces are added to each node in the cluster. Nodes with non-homogenous interfaces within a CE Site might cause issues. Therefore, each node in a given CE Site should have the same number of interfaces placed in the same VRFs.

To add extra interfaces to an instance on GCP, ensure the following conditions are met:

• Additional interfaces can be applied to a VM only while creating it.
• GCP does not support adding new interfaces after the VM is created.
• Each interface must connect to a different VPC network.

Important: The IP address ranges of the two interfaces on the same instance must not overlap.

• Power down the VM prior to adding any new interfaces or modifying any existing interfaces.

• Click ADD A NETWORK INTERFACE to configure an additional SLI for the node. Use the following parameters:

  • Select the additional VPC network.
  • Select the additional subnet.
  • Select Ephemeral for the Primary internal IPv4 address.
  • For External IPv4 address, select None.

Figure: Configure Additional SLI

Modify Interface Attributes

• To modify any interface attributes, click Manage Configuration. Then, in the wizard, click Edit Configuration.

• Under the Nodes subsection, click the pencil icon under Actions to edit.

• Choose one of the interfaces to edit.

• Change the settings as required. In this example, the interface is being placed in the prod-segment. Therefore, the setting from the original Site Local Inside (Local VRF) to Segment was changed. Then the required segment is selected.

Figure: Edit Interface

Figure: Edit Interface

• Click Save and Exit.

• Power back up the VM.

Troubleshooting

For troubleshooting issues, see the Troubleshooting Guide for Secure Mesh Site v2 Deployment guide. It provides step-by-step instructions to debug and resolve the issues that may arise due to registration and provisioning errors.

Concepts

• System Overview
• Core Concepts
• Networking
• F5 Distributed Cloud - Customer Edge

References

• GCP VPC Permissions