Deploy Site in AWS (ClickOps)

Objective

Important: This is a legacy workflow for deploying Customer Edge (CE) Sites and is not recommended to use. A new workflow for deploying Customer Edge (CE) Sites has been introduced and is now Generally Available (GA). It is recommended to use the new Secure Mesh Site (v2) workflow for all Customer Edge deployments. You can find this workflow here.

This guide provides instructions on how to create a customer edge (CE) site using the Amazon Web Services (AWS) Console and deploy to an AWS virtual private cloud (VPC). For more information on sites, see F5® Distributed Cloud Site.

This guide will show you how to create a single-node mesh site with dual interfaces (ingress/egress gateway). However, this guide will also incorporate the differences that you can successfully deploy an AWS CE Site using both Mesh or App Stack and in any supported combination of nodes and interfaces.

Site Types and Scenarios

The scenarios below in the table have been tested successfully. The Network Address Translation Gateway (NAT GW) device in Table 1 references the AWS NAT Gateway. However, technically this should also work with third-party firewalls.

Note: In the Number of Nodes column, 1/3 indicates 1 or 3 nodes. IGW references Internet Gateway.

Considerations for Sites Behind NAT Gateway

In a regular deployment, each CE node has an Elastic IP address associated with the Site Local Outside (SLO) interface, and the SLO route table routes the outgoing traffic via the Internet Gateway using the Elastic IP address as the NAT IP address.

But for scenarios where there is a requirement to have the CE site deployed without a public IP address, you can place the CE site behind a NAT Gateway. This can be the AWS NAT gateway or any third-party instance, like a firewall used as a NAT gateway.

If you are deploying a site in this scenario, there are a few differences to note:

• There is no public IP association with the CE(s).

• You must ensure the CE can get to the Internet through its SLO interface.

• From the AWS side, ensure that the routing table from the SLO subnet has a default route 0.0.0.0/0 that points to the NAT Gateway.

• The NAT Gateway is a zonal object. In other words, it belongs to one availability zone. Therefore, in the case of a three-node CE cluster, you might want to deploy additional NAT Gateways for high availability.

Prerequisites

The following prerequisites apply:

• A Distributed Cloud Services Account. If you do not have an account, see Getting Started with Console.

• An account with AWS. See Required Access Policies for permissions needed to deploy site. To create a cloud credentials object, see Cloud Credentials.

• Resources required per node: Minimum 4 vCPUs, 14 GB RAM, and 80 GB disk storage. However, to deploy an F5® Distributed Cloud App Stack Site, 100 GB is the recommended minimum amount of storage. For a full listing of the resources required, see the Customer Edge Site Sizing Reference guide. All the nodes in a given CE Site should have the same resources regarding the compute, memory, and disk storage. When deploying in cloud environments, these nodes should use the same instance flavor.

• Allow traffic from and to the Distributed Cloud public IP addresses to your network and allowlist related domain names. See Firewall and Proxy Server Allowlist Reference guide for the list of IP addresses and domain names.

• F5 assumes that the VPC exists with a minimum of two subnets: one for the Site Local Outside (SLO) and one for the Site Local Inside (SLI). The CE generally references the SLO interface as eth0 and the SLI interface as eth1.

• Internet Control Message Protocol (ICMP) needs to be opened between the CE nodes on the Site Local Outside (SLO) interfaces. This is needed to ensure intra-cluster communication checks.

Important: After you deploy the CE Site, the IP address for the SLO interface cannot be changed. Also, the MAC address cannot be changed.

Procedure

This procedure provides instructions for creating a Secure Mesh Site with two interfaces (ingress/egress) on an AWS VPC using AWS Console. Most of the objects and configurations are the same for single-node and multi-node sites.

The steps differ only for the node instance and elastic IP address creation, where you must repeat the steps for each node in a multi-node site.

Create IAM Policy and Role

You must create an Identity and Access Management (IAM) role that will be attached to the CE site, with minimal permissions configured for that role using an IAM policy.

Step 1: Navigate to policy creation page.

• In AWS Console, navigate to the IAM service.

• Under Access management, click Policies and then click Create policy.

Figure: Create Policy

Step 2: Create and configure IAM policy.

• From the Service menu, select EC2.

• In the Filter Actions box, search for and select the DescribeInstances, DescribeTags, and autoscaling:DescribeAutoScalingInstances permissions.

Figure: Filter Actions

Figure: Filter Actions

• Alternatively, you can also add the tags using the JSON option.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances",
                "ec2:DescribeTags",
                "autoscaling:DescribeAutoScalingInstances"
            ],
            "Resource": "*"
        }
    ]
}

Copied!

• Click Next.

• Enter a Policy name and an optional short description.

• Click Create policy.

• Confirm the policy was created.

Figure: IAM Policy Confirmation

Step 3: Create and configure IAM role.

• Under Access management, click Roles and then click Create role.

• Confirm AWS service is selected.

• From the Service or use case menu, select EC2. Then click Next.

Figure: Create IAM Role

• In the search box, find and select the policy previously created.

• Click Next.

• Enter a Role name and short description. Ensure that the role name and description clearly relate to the policy.

Figure: IAM Role Configuration

• Click Create role.

Create SSH Key Pairs

You need to create key pairs for SSH login into the EC2 instance for troubleshooting purposes.

Locate AMI ID and Hardware Image

Specific information is required to deploy a CE site. For example, you will need to your site type, number of interfaces, and region. This information will help determine and locate the F5 Distributed Cloud Services certified hardware image and Amazon Machine Image (AMI) ID. Note that the AMI ID corresponds to a certified hardware image.

Only this information is required to derive the AMI ID:

• Cloud provider (AWS) in which you are deploying the Site.

• Whether you are deploying a Mesh Site or App Stack Site?

• For a Mesh Site, single-NIC or multi-NIC?

• The cloud provider region in which you plan to deploy the CE nodes.

As an example, if you are deploying a multi-NIC node in us-west-1:

• Region: us-west-1

• Site Type: Mesh

For the example above, the AMI ID of ami-0858c196c17ebf057 corresponds to the F5 Distributed Cloud Services hardware image of aws-byol-multi-nic-voltmesh.

• To locate and download the AMI file, navigate to F5 Distributed Cloud Services AMI.

• In the table, use the Region and AMI columns to search for your AMI file.

Figure: AMI Search

Figure: AMI Search

Create Site Token

Create a site token or use an existing token. If you are configuring a multi-node site, use the same token for all nodes. To create the token, see the Create Site Token guide.

Modify the EC2 User Data File

Download the raw .yml file to your machine and input the value of the site token into the Token field. See GitHub User Data File for more information. Afterwards, save the file on your machine as you will need it for the user data when creating the F5 CE virtual machine.

Figure: Raw YAML File

#cloud-config
#Only values that need to be inserted are the token and site name. Insert as is without parenthesis
write_files:
- path: /etc/hosts
  content: |
    # IPv4 and IPv6 localhost aliases 
    127.0.0.1 localhost
    127.0.0.1 vip
  permissions: 0644
  owner: root
- path: /etc/vpm/config.yaml
  permissions: 0644
  owner: root
  content: |
    Vpm:
      ClusterType: ce
      ClusterName: #### TO BE REPLACED BY THE F5XC SECURE MESH SITE NAME ####
      Token: #### TO BE REPLACED BY F5XC API TOKEN ####
      MauricePrivateEndpoint: https://register-tls.ves.volterra.io
      MauriceEndpoint: https://register.ves.volterra.io
      CertifiedHardwareEndpoint: https://vesio.blob.core.windows.net/releases/certified-hardware/aws.yml
    Kubernetes:
      EtcdUseTLS: True
      Server: vip
      CloudProvider: disabled

Create CE Security Group

The CE site’s instance security is internally managed by the CE data path. Therefore, you must configure a security group with allow all rules for both inbound and outbound traffic to be used with the site deployment.

Figure: Create Security Group

Deploy Multi-Node Site

Follow these steps to create a three-node Secure Mesh Site.

Create Elastic IP Addresses

Create an Elastic IP address to attach to the site.

Step 2: Configure Elastic IP address(es).

• Click Add new tag.

• Enter a key name and a name (for example, f5-ce-eip-1) for the value.

• Click Allocate.

Figure: Allocate IP Addresses

• Repeat the above steps to create two more Elastic IP addresses (for example, f5-ce-eip-2 and f5-ce-eip-3) for node two and node three of your site. The above elastic IP address will be used for node one of the Secure Mesh Site.

Existing VPC Details

In this example, a dual interface multi-node CE site is being deployed. Since the site has two interfaces, two subnets are required for each node. One for SLI and the other for SLO.

For a multi-node site, the nodes will be deployed in three separate Availability Zones (AZs). An example would be us-west-1a, us-west-1b, and us-west-1c. You will need two subnets in each AZ (six subnets in total).

Note: The CE sites created in F5 Console also use a workload subnet, but it is not a requirement and can be ignored for manually created sites.

Create the CE Node Instances

Create the F5 CE EC2 instance using the previously created parameters. It is recommended that you create a table and fill it with the information before deploying the CE EC2 instance.

See the table below for an example parameters and explanations used for this procedure.

Launch an instance from the EC2 Console and follow the steps below to create the EC2 instance for node one in Availability Zone (AZ) a.

Note: This steps show nodes 1, 2, and 3 deployed in AZ a, b, and c as an example. However, the nodes can be deployed in any AZ as long as they are in separate AZs for redundancy.

Step 1: Configure tags.

After you have set the site name, click Add additional tags to add the required tags.

Note: Without the tags labeled as mandatory, the site will not register.

Figure: Add Tags

Figure: Add Tags

Step 2: Configure instance type.

• Select the AMI per the parameters in the table above.

• Select the instance type. Note that t3.xlarge is the minimum instance type required to run the F5 CE software.

• Select the key pair.

Figure: Add Key Pairs

Step 3: Configure network interfaces.

• Select the VPC and subnet (SLO-subnet-public-a), which are referenced in the parameters' table above. The subnet chosen is the subnet for Network interface 1 (SLO). Ensure that you are choosing the correct subnet so that the node one instance gets created in AZ a. Also, ensure that Auto-assign public IP is disabled as an Elastic IP address is used instead.

• From the Common security groups menu, select the f5-ce-security-group created previously.

• Under Advanced network configuration, for Network interface 1 enter a description for the interface as node1-slo.

Figure: Configure Network Interface 1

• Click Add network interface to create the SLI interface.

• For Network interface 2 enter a description of node1-sli.

• Select the subnet (SLI-subnet-private-a).

Figure: Configure Network Interface 2

Step 5: Configure the advanced details.

• Select the IAM instance profile (Role) per the parameters' table above.

• Copy and paste the modified user data file into the User data box.

Figure: User Data Information

Associate the Elastic IP Addresses to SLO Interface

After the EC2 instance is created, you need to allocate the previously created Elastic IP address to the SLO interface on the corresponding nodes.

Step 1: Note down SLO interface ID.

Note down the SLO interface ID by navigating to the Networking tab under the f5-ce node one instance and getting the elastic network interface (ENI) ID of the SLO. Keep this information as you will need it to assign the Elastic IP address to the interface.

Figure: Associate IP Address to Interface

Step 2: Associate IP addresses to interface.

• Select the Elastic IP address f5-ce-eip-1 and then navigate to Actions > Associate Elastic IP address.

• Select Resource type as Network interface.

• Enter the ENI ID.

• Click Associate.

Figure: Associate IP Address to Interface

Stop Source/Destination Checks

In AWS, the source/destination check is a feature that ensures that an EC2 instance is only responsible for traffic that it sends or receives. By default, this check is enabled for all EC2 instances, meaning that each instance is expected to handle only the network traffic that originates from or is destined to its own IP address.

In the case of an F5 CE Site, the instance is a Network Virtual Appliance (NVA) that outgoing and incoming traffic needs to transit through, and therefore we need to disable the source/destination check on the F5 CE EC2 instance.

• For you EC2 instance, navigate to Actions > Networking > Change source/destination check.

Figure: Networking

• Check the box for Stop.

Figure: Stop Checkbox

• Click Save.

Deploy Single-Node Site

Follow these steps to create a single-node Secure Mesh Site.

Create Elastic IP Address

Create an Elastic IP address to attach to the site.

Step 2: Configure Elastic IP address.

• Click Add new tag.

• Enter a key name and a name (for example, f5-ce-eip) for the value.

• Click Allocate.

Figure: Allocate IP Addresses

Existing VPC Details

In this example, a dual interface single-node CE site is being deployed. Since the site has two interfaces, two subnets are required for each node. One for SLI and the other for SLO.

For a single-node site, the node instance will be deployed in only one Availability Zone (AZ), and both subnets will be in the same AZ (for example, us-west-1a).

Note: The CE sites created in F5 Console also use a workload subnet, but it is not a requirement and can be ignored for manually created sites.

Create the CE Node Instance

Create the F5 CE EC2 instance using the previously created parameters. It is recommended that you create a table and fill it with the information before deploying the CE EC2 instance.

See the table below for an example parameters and explanations used for this procedure.

Launch an instance from the EC2 Console and follow the steps below to create the EC2 instance for the node.

Step 1: Configure tags.

After you have set the site name, click Add additional tags to add the required tags.

Note: Without the tags labeled as mandatory, the site will not register.

Figure: Add Tags

Figure: Add Tags

Step 2: Configure instance type.

• Select the AMI per the parameters in the table above.

• Select the instance type. Note that t3.xlarge is the minimum instance type required to run the F5 CE software.

• Select the key pair.

Figure: Add Key Pairs

Step 3: Configure network interfaces.

• Select the VPC and subnet (SLO-subnet-public), which are referenced in the parameters' table above. Ensure that Auto-assign public IP is disabled as an Elastic IP address is used instead.

• From the Common security groups menu, select the f5-ce-security-group created previously.

• Under Advanced network configuration, for Network interface 1 enter a description for the interface as slo.

Figure: Configure Network Interface 1

• Click Add network interface to create the SLI interface.

• For Network interface 2 enter a description of sli.

• Select the subnet (SLI-subnet-private).

Figure: Configure Network Interface 2

Step 5: Configure the advanced details.

• Select the IAM instance profile (Role) per the parameters' table above.

• Copy and paste the modified user data file into the User data box.

Figure: User Data Information

Associate the Elastic IP Address to SLO Interface

After the EC2 instance is created, you need to allocate the previously created Elastic IP address to the SLO interface on the CE node.

Step 1: Note down SLO interface ID.

Note down the SLO interface ID by navigating to the Networking tab under the f5-ce instance and getting the elastic network interface (ENI) ID of the SLO. Keep this information as you will need it to assign the Elastic IP address to the interface.

Figure: Associate IP Address to Interface

Step 2: Associate IP address to interface.

• Select the Elastic IP address f5-ce-eip and then navigate to Actions > Associate Elastic IP address.

• Select Resource type as Network interface.

• Enter the ENI ID.

• Click Associate.

Figure: Associate IP Address to Interface

Stop Source/Destination Checks

In AWS, the source/destination check is a feature that ensures that an EC2 instance is only responsible for traffic that it sends or receives. By default, this check is enabled for all EC2 instances, meaning that each instance is expected to handle only the network traffic that originates from or is destined to its own IP address.

In the case of an F5 CE Site, the instance is a Network Virtual Appliance (NVA) that outgoing and incoming traffic needs to transit through, and therefore we need to disable the source/destination check on the F5 CE EC2 instance.

• For you EC2 instance, navigate to Actions > Networking > Change source/destination check.

Figure: Networking

• Check the box for Stop.

Figure: Stop Checkbox

• Click Save.

Create and Register Site

Follow the Create Secure Mesh Site documentation to create a Secure Mesh Site object and register the nodes to the site.

Troubleshooting

For troubleshooting and opening a support case, see the Troubleshooting Manual Site Deployment Registration Issues guide. It provides step-by-step instructions to debug and resolve the issues that may arise due to Distributed Cloud published Terraform errors, networking and security misconfiguration, or CE internal process issues. The guide also provides instructions for contacting the F5 Distributed Cloud Support Team if you are unable to resolve the issue.

Concepts

• System Overview
• Core Concepts
• Networking
• F5 Distributed Cloud - Customer Edge
• F5 Distributed Cloud Site

References

• AWS VPC Permissions
• Create AWS Site
• Deploy AWS VPC Site with Terraform