Create Secure Mesh Site v2
Objective
This document provides instructions on how to deploy an F5 Distributed Cloud Customer Edge (CE) Site across all supported providers. For on-premises providers, this includes VMware, Nutanix, OpenStack, and more. For public cloud providers, this includes AWS, Azure, GCP, and OCI.
This new and simplified workflow also includes enhancements to remove certified hardware, a single endpoint for CE Site registration, and much more.
Important: The following providers are Generally Available (GA): VMware, AWS, Azure, GCP, OCI, Nutanix, OpenStack, Equinix, and Baremetal.
The following provider is Early Access (EA): KVM.
Planning
Read the following documents before deploying a Secure Mesh Site in any provider environment:
- Understanding F5 Distributed Cloud - Customer Edge (CE)
- CE Datasheet
- CE Supported Platforms Guide
- Customer Edge Site Sizing Reference
- CE Performance Guide: Contact your account representative on CE performance-related information.
- Proxy for CE Registration and Upgrades Reference
- Secure Mesh Sites v2 Frequently Asked Questions
- Customer Edge Registration and Upgrade Reference
- F5 Customer Edge IP Address and Domain Reference for Firewall or Proxy Settings
Important: After you deploy the CE Site, the IP address for the SLO interface cannot be changed. Also, the MAC address cannot be changed.
General Prerequisites
-
An F5 Distributed Cloud Account. If you do not have an account, see Getting Started with Console.
-
One or more devices or virtual machines (VMs) consisting of interfaces with Internet reachability for Site deployment.
-
Resources required per node: Minimum 8 vCPUs, 32 GB RAM, and 80 GB disk storage. For a full listing of the resources required, see the Customer Edge Site Sizing Reference guide. All the nodes in a given CE Site should have the same resources regarding the compute, memory, and disk storage. When deploying in cloud environments, these nodes should use the same instance flavor.
-
Internet Control Message Protocol (ICMP) needs to be opened between the CE nodes on the Site Local Outside (SLO) interfaces. This is needed to ensure intra-cluster communication checks.
-
Configure your firewall or proxy server to allow connections from and to the IP addresses listed in the F5 Customer Edge IP Address and Domain Reference for Firewall or Proxy Settings guide.
Configuration Overview
Use the following sequence of actions to deploy a CE Site in your provider's environment:
(1.) Choose the provider where your Secure Mesh Site is deployed. Configure additional parameters as required. Apart from the provider, all parameters are optional.
Important: It is important to review all optional parameters while configuring the Site to make sure the CE deployment is in adherence with your environment. There are a few properties which cannot be changed after the CE Site is created. If any changes are required to these, then the CE Site would need to be redeployed.
(2.) Prepare to launch nodes. Your options to launch depend on your provider environment:
- For VMware, OpenStack, Nutanix, KVM, Baremetal, and OCI: Download the CE node image from the F5 Distributed Cloud Console. Use the Download Image option.
- For AWS, Azure, GCP, and Equinix: Use the Launch Instance option to deploy your instance from the corresponding provider's marketplace.
(3.) Check out node token when launching a node. Each node you deploy requires a unique token generated in F5 Distributed Cloud Console.
Important: Tokens are ephemeral and expire within 7 days, so you should generate a node token while launching a node and not pre-stage them.
(4.) Launch nodes. If the high availability (HA) option is disabled, then your CE Site can only support one (1) node. If the HA option is enabled, then the CE Site requires three (3) nodes. Additional nodes can only be added to CE sites when the HA option is enabled.
Important: You cannot change the HA option after your CE Site is created.
(5.) Add additional network interfaces to each node as required.
Important: Adding a new network interface causes the data plane services to restart. Therefore, F5 strongly recommends that you perform this operation during maintenance windows. As data plane services restart, traffic drops are expected, as well as tunnels going down. When adding interfaces, it is important to make sure that the interfaces are added to each node in the cluster. Nodes with non-homogenous interfaces within a CE Site might cause issues. Therefore, each node in a given CE Site should have the same number of interfaces placed in the same VRFs.
Power off each node VM when adding new interfaces or modifying existing ones.
Create Secure Mesh Site
Log into F5 Distributed Cloud Console to create a Secure Mesh Site.
Step 1: Enter metadata information for Site.
-
In the Multi-Cloud Network Connect workspace, navigate to Manage > Site Management > Secure Mesh Sites v2.
-
Select Add Secure Mesh Site to open the configuration form.
-
In the Metadata section, enter a name for the Site.
-
Optionally, select labels and add a description.
Step 2: Select the infrastructure provider settings for Site.
-
From the Provider Name menu, select the infrastructure provider from the options available. Refer to provider-specific documentation links below to bring up infrastructure in that provider:
- For High Availability, choose an option. If it is Disabled, your CE Site can only support one node. If it is Enabled, then your CE Site requires three nodes. Additional nodes can only be added to CE sites when HA is Enabled.
Important: You cannot change the High Availability mode after your CE Site is created and deployed.
Step 3: Configure RE Site options.
Use the following steps to configure the regional edge (RE) Site settings in the Regional Edge section. Your CE Site connects to the RE Site for registration purposes.
Note: The configuration options in this section are set to standard default values. Therefore, it may not be necessary to customize any of the options unless you need customization only for advanced deployments.
-
From the Regional Edge Selection menu, select the RE geography to use. The default option is use the geographically-closest RE Site to where you are deploying your CE Site.
-
Optionally, select the Site-to-Site tunnel encryption type from the Tunnel Type menu. The default option is IPsec/SSL. When IPsec/SSL is used, IPsec takes priority.
-
Optionally, configure the timeout value for Site tunneling from the Tunnel Dead Timeout (msec) menu. The default option is zero (0) milliseconds.
-
Optionally, enable the offline survivability feature from the Offline Survivability Mode menu. For more information, see the Manage Site Offline Survivability guide.
Step 4: Configure Site networking options.
Use the following steps to configure the CE Site networking settings in the Site Networking section.
Note: The configuration options in this section are set to standard default values. Therefore, it may not be necessary to customize any of the options unless you need customization only for advanced deployments.
- Site Local Outside Network (SLO) is used to connect the CE node with the F5 Distributed Cloud Regional Edges (REs). It can also work as a public/WAN network. This network typically requires connectivity to the Internet (see note below for exception). If a custom DNS server or static routes need to be added into this network, then from the Site Local Outside Network menu, choose Configure Site Local Outside Network. Then click View Configuration. Here you can add custom static routes, common VIP for load balancers (this can be overridden on a per load balancer basis in the Advertisement Policy), or DNS servers for the SLO network.
Note: After you configure the SLO interface with a static IP address, DHCP displays in the Console. However, your static IP configuration is well taken into account. Also, remember that you cannot modify SLO parameters once the node is registered and deployed.
The CE Site can be connected to a private underlay that connects with F5 Distributed Cloud Regional Edges (REs) in which case the SLO need not have Internet-bound connectivity. Connectivity to the REs uses this private underlay.
- Site Local Inside Network (SLI) represents the internal network (LAN). If a custom DNS server or static routes need to be added into this network, then from the Site Local Inside Network menu, choose Configure Site Local Inside Network. Then click View Configuration. Here you can add custom static routes, common VIP for load balancers (this can be overridden on a per load balancer basis in the Advertisement Policy), or DNS servers for the SLI network.
Note: Site Local Inside is an optional network. Consider using Network Segments from Multi-Cloud Network Connect > Networking > Segments for internal networks. Network segments are flexible and can be used to keep networks isolated within an environment. In other words, they are restricted to a single CE Site or can be also used for seamless extension of networks across multiple hybrid/multi-cloud environments (across multiple CE sites).
- To enable virtual IP address (VIP) redundancy when operating load balancers advertised on a CE in L2 adjacency mode: From the Load Balancer Settings section, select Enable VRRP for VIP(s) from the VRRP Mode drop-down menu.
Step 5: Configure Site to Site connectivity options.
Use the following steps to configure the CE Site networking settings in the Site To Site Connectivity section.
Note: The configuration options in this section are set to standard default values. Therefore, it may not be necessary to customize any of the options unless you need customization only for advanced deployments.
-
To connect your Site to other sites using the SLO network: From the Connect using SLO Local VRF drop-down menu, select an option:
-
Site Mesh Group: This option connects your Site to other Sites in a mesh network. You can connect using a public IP or a private IP. For more information, see the Configure Site Mesh Group guide.
-
Member of DC Cluster Group: This option places your Site within a Direct Connect (DC) Cluster Group. For more information, see the Configure DC Cluster Group guide.
-
-
To connect your Site to other sites using the SLI network, from the Connect using SLI Local VRF menu, select Member of DC Cluster Group. For more information, see the Configure DC Cluster Group guide.
Step 6: Configure network security for Site.
Use the following steps to configure the CE Site networking security settings in the Network Security section.
Note: The configuration options in this section are set to standard default values. Therefore, it may not be necessary to customize any of the options unless you need customization only for advanced deployments.
-
From the Network Firewall menu, choose to enable an enhanced firewall. Select the firewall from the drop-down menu. Use Add Item to add more than one firewall. For more information, see the Create Network Firewall guide.
-
From the Forward Proxy menu, choose to enable a forward proxy. Select the policy from the drop-down menu. Use Add Item to add more than one policy. The network traffic is processed based on the order set. For more information, see the Create Forward Proxy Policies guide.
Step 7: Configure performance mode.
Use the following steps to configure the CE Site performance mode in the Services & Resources section.
Note: The configuration option in this section is set to standard default values. Therefore, it may not be necessary to customize it unless you need customization only for advanced deployments.
-
In the Services & Resources section, from the Performance Mode menu, select an option:
-
L7 Enhanced: This option optimizes the Site for Layer 7 traffic processing and is the default option.
-
L3 Enhanced: This option optimizes the Site for Layer 3 traffic processing. If you choose this option, then no L7 functionality is provided for your Site, such as load balancing. If you are using L3 Enhanced mode, select whether to use this feature with or without jumbo frames.
-
Important: If L3 Enhanced mode is not enabled on all CE sites in a Site Mesh Group, then the MTU configured on the Site-to-Site tunnel interfaces will not be consistent. Therefore, F5 recommends that you enable L3-focused performance mode on all sites participating in a Site Mesh Group.
Step 8: Configure Site management options.
Step 8.1: Configure software settings.
-
From the F5XC Software Version menu, keep the default selection of Latest SW Version or select F5XC Software Version to specify an older version number.
-
From the Operating System Version menu, keep the default selection of Latest OS Version or select Operating System Version to specify an older version number.
Step 8.2: Configure node upgrade settings.
From the Node by Node Upgrade menu, select how each worker node is upgraded. Note that this configuration does not apply to the control node(s). Optionally, configure Upgrade Wait Time, Node Batch Size, and Node Batch Size Count.
Step 8.3: Configure admin credentials.
-
Under Admin Password, click Configure. Configure the options for Secret Type, Action, and Policy Type. Enter your password in the Secret to Blindfold text box. Click Apply.
-
Enter your public SSH key.
Step 8.4: Configure node services, monitoring, and log streaming.
By default, the local web UI, SSH, and DNS services on each node in a CE Site are enabled.
-
To disable any of these services, from the Node Local Services menu, select Disable. Click Add Item for each service you want to disable. By default, these services are enabled to help with Site troubleshooting.
-
From the Logs Streaming menu, select Enable to configure a log receiver. Keep Disable selected if log streaming is not required.
Step 8.5: Configure enterprise proxy server settings.
By default, all CE sites use the F5 Enterprise Proxy, which is hosted by F5 in the F5 Global Network to register with F5 Distributed Cloud.
- If you want to use a custom proxy hosted in your enterprise environment:
- From the Enterprise Proxy menu, choose Custom Enterprise Proxy and provide your enterprise proxy settings, such as Proxy IPv4 Address, Proxy Port, Username, and Password. In addition, you can choose to use this custom enterprise for proxy-to-proxy tunnels from the nodes of this CE Site to the F5 Distributed Cloud Regional Edges (REs) by choosing to Enable from the Use for RE Tunnels menu.
Important: When Use for RE Tunnels is enabled, the CE Site always establishes a connection to the F5 Distributed Cloud Regional Edges (REs) using SSL tunnel encapsulation, even if the RE tunnel type is set to IPsec and SSL. After the Site comes online, the tunnel type setting in the RE section (Step 3 above) is automatically changed to SSL. When RE tunnels are formed via a custom proxy, IPsec cannot be supported because Internet Key Exchange (IKE), which is UDP-based, cannot be routed via a custom proxy. Therefore, the Site setting is changed to disable IPsec and only uses SSL.
Use for RE Tunnels cannot be changed after the CE Site is created. Make sure to set this field while creating the CE Site object. Changing this property requires the creation of a new CE Site and re-deployment of all nodes.
- From the Proxy Bypass Settings drop-down menu, choose Custom to add domains to bypass the forward proxy server.
Step 8.6: Configure DNS and NTP server settings.
-
Optionally, choose to configure custom DNS servers:
- From the DNS Servers menu, select Custom. Click Add Item and enter a server. Note that multiple DNS servers can be added.
-
Optionally, choose to configure custom NTP servers:
- From the NTP Servers menu, select Custom. Click Add Item and enter a server. Note that multiple NTP servers can be added.
Step 9: Complete the Site object creation.
Click Add Secure Mesh Site to complete creating the Site. The Status field for the Site object displays Validation in progress. After validation, the field displays Validation Succeeded.
Important: There are certain settings that cannot be changed after the CE Site object is created. Make sure that all settings for your CE Site are configured as required before clicking Add Secure Mesh Site to avoid re-creating the CE Site object and re-deploying the CE nodes.
Deploy and Register Site
Refer to provider-specific documentation links below to deploy and register Site:
Day 2 Operations
- To monitor your Site, see the Monitor Site guide.
- To manage your Site software and OS updates, see the Manage Site guide.
- For troubleshooting issues, see the Troubleshooting Guide for Secure Mesh Site v2 Deployment guide. It provides step-by-step instructions to debug and resolve the issues that may arise due to registration and provisioning errors.
- For the latest on Distributed Cloud Services releases, see Changelogs.
Related Guides
To create a load balancer on the CE Site, see the HTTP Load Balancer or the TCP Load Balancer guides.