Deploy Secure Mesh Site v2 in AWS (ClickOps)

Objective

This guide provides instructions on how to create a customer edge (CE) site using F5® Distributed Cloud Console and Amazon Web Services (AWS) Console and deploy to an AWS virtual private cloud (VPC). For more information on a CE Site, refer to F5 Distributed Cloud - Customer Edge.

As part of the new site deployment workflow, you can deploy the CE site as a Secure Mesh Site to an AWS VPC.

Important: This guide does not provide instructions on how to deploy an F5® App Stack Site.

General Prerequisites

The following general prerequisites apply:

• A Distributed Cloud Services Account. If you do not have an account, see Create an Account.

• An account with AWS. See Required Access Policies for permissions needed to deploy site.

• Resources required per node: Minimum 4 vCPUs, 14 GB RAM, and 80 GB disk storage. For a full listing of the resources required, see the Customer Edge Site Sizing Reference guide. All the nodes in a given CE Site should have the same resources regarding the compute, memory, and disk storage. When deploying in cloud environments, these nodes should use the same instance flavor.

• Allow traffic from and to the Distributed Cloud public IP addresses to your network and allowlist related domain names. See Firewall and Proxy Server Allowlist Reference guide for the list of IP addresses and domain names.

• F5 assumes that the VPC exists with a minimum of a single subnet where the CE Site node will be deployed. For three-node clusters, it is recommended to have three different subnets in three different Availability Zones (AZs).

• The new Secure Mesh Site workflow enables you to have up to eight interfaces. However, these interfaces should be in different subnets. Therefore, make sure you have the required subnets available before creating the CE Site nodes.

• Internet Control Message Protocol (ICMP) needs to be opened between the CE nodes on the Site Local Outside (SLO) interfaces. This is needed to ensure intra-cluster communication checks.

Important: After you deploy the CE Site, the IP address for the SLO interface cannot be changed. Also, the MAC address cannot be changed.

Configuration Overview

To create a Secure Mesh Site, here are the high-level steps:

1. Site object creation: Configure the site within F5 Distributed Cloud Console.
2. Node creation prerequisites: Create objects that will be associated to the nodes (AWS EC2 instances) including, security groups, key pairs, and more.
3. Image management: Gather all the information required to find the Amazon Machine Image (AMI).
4. Node management: Use the AMI found in the previous step to create the EC2 instances (in other words, the nodes that constitute the CE Site).
5. Interface management: Add additional interfaces on the nodes, if necessary.

Procedure

In this guide, the procedure demonstrates the steps to deploy a single-node secure mesh site with dual interfaces. However, this guide will also explain the necessary deviations from this specific model where necessary, making it flexible to adjust to different node and interface requirements.

Create Site Object

• Create a secure mesh site object in Distributed Cloud Console. Refer to the Create Secure Mesh Site guide.

• Set the Provider Name option to AWS.

Figure: Provider Type

Create Security Group

Create the security group that will be attached to the F5 CE Site EC2 instance. Note that this security group will be used for the Site Local Outside (SLO) interface. Any other interfaces can use the default security group of the VPC, which allows all traffic in both directions.

Step 1: Create security group.

• In AWS Console, navigate to the EC2 service.

• From the left panel, under Network & Security, click Security Groups.

Figure: Security Groups

• Click Create security group.

Figure: Security Groups

• Enter a name. In this example, f5-ce-security-group is the name for the security group.

Step 1.1: Create inbound rules.

• For the inbound rules, ensure the following:

  • Allowed SSH from the machine’s public address. This is where AWS will figure out the public IP address that a user is configuring from and allows it. You can also use custom and put your corporate public address space.

  • Allowed ICMP for troubleshooting.

  • Allowed TCP Port 65500 for the local UI on the CE Site.

  • For three-node clusters, ensure that the traffic is allowed between the nodes using either the IP address or, even better, by referencing the same security group as an allowed source.

Important: If you are creating load balancers to publish applications, you will need to add additional rules in your security group to accept the traffic that comes to your VIP.

Step 1.2: Create outbound rules.

• For outbound traffic, create an allow all policy.

• After you finish, click Create security group.

Figure: Security Groups Rules Created

Step 2: Verify rules created.

Confirm rules created successfully. Use the Inbound rules and Outbound rules tabs to list the rules.

Figure: Verify Security Groups Rules

Figure: Verify Security Groups Rules

Create SSH Key Pair

Create the key pair that will be used to SSH login to the EC2 instance for troubleshooting purposes, if needed.

Step 1: Create key pair.

• In AWS console, navigate to Network & Security > Key Pairs.

• Click Create key pair.

• Enter a name, and select the type of key pair.

Figure: Create SSH Key

• Click Create key pair.

Step 2: Verify key pair.

Verify the SSH key pair was created.

Figure: Verify SSH Key

Existing VPC Details

In this procedure, a dual interface single-node CE site is being deployed. Since the site has two interfaces, two subnets are required. One for SLI and the other for SLO. Both subnets are in the same AWS Availability Zone (AZ). In this example, us-west-1c is the AZ where the SLI and SLO subnets are located.

The workload subnet is generally used, but it is not required to deploy a node for the CE Site.

Figure: Existing VPC Details

Figure: Existing Subnet Details

Create Elastic IP Address

Create an Elastic IP (EIP) address to attach to the CE Site.

• In AWS console, navigate to Network & Security > Elastic IPs.

• Click Allocate Elastic IP address.

• Ensure Name is selected for Key and enter a name for the EIP address in the Value field. This example uses f5-ce-eip.

Figure: Configure EIP Address

Find AMI

• For your site, click ... under Actions.

• Click Copy Image URL. This action will copy the AMI name within AWS. It is important to note that the AMI name for the CE Site node images will be the same regardless of the chosen AWS region.

Figure: Copy Image URL

Note: The AMI name follows the naming convention of f5xc-ce-<version>. For example, f5xc-ce-9.2. The copied image name is used to find the AMI in AWS Console.

• In AWS Console, navigate to the EC2 service.

• From the left panel, click Images > AMIs.

• Select Public images.

• In the search box, paste the AMI name. You should have one matching entry. The AMI name to look for is f5xc-ce-<version> where version consists of numbers only.

Figure: Find AMI

Workaround: Unable to find the AMI in local region.

While the AMIs are published across all global regions, there might be cases where the AMI does not exist (for example: a new region just went online). If you already found the AMI in the previous section, you can skip this section. This section explains how you can still deploy CE sites in regions where the AMIs do not exist.

You can use the functionality within AWS that is documented here. Switch to us-east-1 in the AWS Console and locate the AMI by its name as described above. In this example, the AMI is being copied to São Paulo (South America) region. It is highly recommended keeping the name and description matching to the original name or at a minimum, provide meaningful names that will help you distinguish the AMI. It is important to realize that this copy is now a Private AMI. In other words, it is only accessible to your account whereas the original AMI is a public AMI for all other customers to consume.

Figure: Copy AMI to Target Region

Figure: Copy AMI to Target Region

Generate Node Token

A one-time node token is required to register a CE Site node to the Distributed Cloud Console. A new token must be generated for every new node in a CE Site. A token is valid for 24 hours. Make sure that the CE node is deployed soon after the token is generated.

• In Distributed Cloud Console, select the Multi-Cloud Network Connect service.

• Navigate to Manage > Site Management > Secure Mesh Sites v2.

• For your site, click ... > Generate Node Token.

Figure: Node Token

• Click Copy.

• Save the value locally. This token will be used later. The token value is hidden for security purposes.

Figure: Copy Node Token

• Click Close.

Create AWS EC2 Instance

Create the EC2 instance virtual machine (VM).

Important: The name of the VM should not have . in it. For example, the hostname can be node-0 or node0, but it cannot be node.f5.com since it is not supported.

• In AWS console, launch an EC2 instance.

• Click Add additional tags. Add the following two tags: ves-io-site-name (equals the site name) and kubernetes.io/cluster/ (the value after /cluster/ is the site name).

Figure: EC2 Instance Name

• For the instance type, select t3.xlarge. This is the minimum instance type required to run F5 CE Site software.

• Select the SSH key pair previously created.

Figure: Configure Instance

• Select the VPC, Subnet (SLO), and security group created previously. The subnet chosen in this is the subnet for Network Interface 1 (SLO). Labeling the subnets helps here so that you can easily place the interface in the SLO subnet.

• Ensure that Auto-assign public IP is disabled as the Elastic IP address is used instead.

Figure: Configure Instance

• Configure the storage requirement (80 GiB).

Figure: Configure Instance

• For the User data field, copy the information below into an editor of your choice and replace the token value with the token corresponding to your specific site.

#cloud-config
write_files:
- path: /etc/vpm/user_data
  content: |
    token: <token>
  owner: root
  permissions: '0644'

Figure: Configure Instance

• Complete creating the EC2 instance.

Associate the Elastic IP Address to SLO Interface

After the is created, you need to allocate the previously created Elastic IP address. However, take note of the SLO Interface ID by navigating to the networking tab under the F5 CE Site instance and getting the ENI ID of the SLO. Keep it handy as you will need to leverage it when assigning the Elastic IP address to the interface.

Figure: Associate EIP

Figure: Associate EIP

Verify CE Site Registration

• In Distributed Cloud Console, navigate to Multi-Cloud Network Connect > Overview > Sites.

• Select the site. The Dashboard tab should clearly show that the CE Site has registered successfully with the System Health of 100% as well as Data Plane/Control Plane both being up.

Figure: Confirm Site Health

Add New Network Interface

After the CE Site registers successfully, you might want to add additional interfaces to cater to different customer requirements.

When adding interfaces, it is important to make sure that the interfaces are added to each node in the cluster. Nodes with non-homogenous interfaces within a CE Site might cause issues. Therefore, each node in a given CE Site should have the same number of interfaces placed in the same VRFs.

Important: Adding a new network interface will cause the data plane services to restart. Therefore, it is strongly recommended to perform this operation during maintenance windows. As data plane services restart, traffic drops are expected, as well as tunnels going down.

For AWS sites, to add another network interface to the CE, you need to create a new network interface as part of the same VPC in a different subnet with auto-assigned addresses preferably and attach a security group to the interface. In the example below, the default VPC security group allows traffic in both directions, as this is an internal-only interface. Tagging the interface with the instance name is a good practice to easily understand the reason behind this interface.

Figure: Add New Interface

Figure: Add New Interface

Figure: Add New Interface

Step 1: Attach interface to instance.

• Power down the VM prior to adding any new interfaces or modifying any existing interfaces.

• After the interface is created, attach it to the particular instance. Select the particular interface you created and click Actions > Attach.

Figure: Attach to Instance

• Specify the correct details of the VPC and instance and then click Attach.

Figure: Attach to Instance

• In Console, navigate to your site and edit the node configuration.

Figure: Edit Interface

• For the desired interface, click the pencil button.

• Ensure you set the IPv4 Interface Address Method as DHCP Client.

• Click Apply, and then click Save and Exit.

Step 2: Verify changes in AWS Console.

Verify from the AWS Console EC2 Instance under the Networking tab that the instance now has two interfaces.

Figure: Verify Instance in AWS

Step 3: Verify changes in Console.

• Power back up the VM.

• You can also verify the same information from Distributed Cloud Console by navigating to the site Dashboard under Multi-Cloud Network Connect > Overview > Sites and choosing the site name. Navigate to the Infrastructure tab to see the Interfaces table.

Figure: Verify Instance in Console

Stop Source/Destination Checks

In AWS, the source/destination check is a feature that ensures that an EC2 instance is only responsible for traffic that it sends or receives. By default, this check is enabled for all EC2 instances, meaning that each instance is expected to handle only the network traffic that originates from or is destined to its own IP address.

In the case of an F5 CE Site, the instance is a Network Virtual Appliance (NVA) that outgoing and incoming traffic needs to transit through, and therefore we need to disable the source/destination check on the F5 CE EC2 instance.

• For you EC2 instance, navigate to Actions > Networking > Change source/destination check.

Figure: Networking

• Check the box for Stop.

Figure: Stop Checkbox

• Click Save.

Modify Interface Attributes

Important: The IP address for the SLO interface cannot be modified. In addition, you cannot modify any MAC addresses for any interfaces.

• Power down the VM prior to adding any new interfaces or modifying any existing interfaces.

• To modify any interface attributes, click Manage Configuration. Then, in the wizard, click Edit Configuration.

Figure: Edit Interface

Figure: Edit Interface

• Under the Nodes subsection, click the pencil icon under Actions to edit.

Figure: Edit Interface

• Choose one of the interfaces to edit. This example uses ens6.

Figure: Edit Interface

• Change the settings as required. In this example, the interface is being placed in the prod-segment. Therefore, the setting from the original Site Local Inside (Local VRF) to Segment was changed. Then the required segment is selected.

Figure: Edit Interface

Figure: Edit Interface

• Click Save and Exit.

• Power back up the VM.

Troubleshooting

For troubleshooting issues, see the Troubleshooting Guide for Secure Mesh Site v2 Deployment guide. It provides step-by-step instructions to debug and resolve the issues that may arise due to registration and provisioning errors.

Concepts

• System Overview
• Core Concepts
• Networking
• F5 Distributed Cloud - Customer Edge

References

• AWS VPC Permissions