Manage Site Offline Survivability

Published April 5, 2023 | Last modified December 10, 2024

Objective

This guide provides instructions on how to manage Offline Survivability for F5® Distributed Cloud Site. A Distributed Cloud site consists of one or more Distributed Cloud nodes. To learn more about sites, see Distributed Cloud Site.

Offline Survivability for a Customer Edge (CE) Site allows the Site to continue functioning normally without traffic loss during periods of connectivity loss to the Regional Edge (RE) or the Global Controller (GC).

Using the instructions provided in this guide, you can enable or disable the capability for Offline Survivability of a Site.

Note: The Offline Survivability functionality currently does not support the connectivity loss for sites within a Hub/Spoke Site Mesh Group. Also, this feature is only supported for cloud sites that are members of a DC Cluster Group.

Functional Overview

As part of Offline Survivability, there are three components in ensuring that the Site remains functional during connectivity loss. The following list summarizes the functional overview of the three components:

Routing: Routes are exchanged via BGP among nodes of a Site and among nodes across Sites in a Site Mesh Group or a DC Cluster Group. When Offline Survivability is enabled for a Site, a local control plane is activated, so that local traffic load balancing for the Site continues to function. Also, if two or more Sites in a Site Mesh Group have Offline Survivability enabled, and the Site Mesh Group is a Full Mesh type group with control plane enabled, load balancing across local and remote endpoints in those Sites continues to function, even when connectivity with the RE is lost. The same is also true for Sites within a DC Cluster Group.

Note: In case of DC Cluster Group, this functionality is supported regardless of whether Sites are connected to the DC Cluster Group via Site Local Outside (SLO) or Site Local Inside (SLI) network.

Identity Management: Certificates for services are issued via a Certificate Authority local to the Site when they start/restart without connectivity to the Global Controller. Therefore, if services restart, they get new certificates and continue functioning.
Secret Management: Secrets that are decrypted from F5 Distributed Cloud when connectivity to Global Controller was intact are cached locally on the site. This enables services to obtain decrypted secrets even when the connectivity is lost.

Important: The F5 Distributed Cloud does not learn the decrypted secrets.

Prerequisites

F5 Distributed Cloud Account is required. In case you do not have an account, see Create an Account.
Allow traffic from and to the Distributed Cloud public IP addresses to your network and allowlist related domain names. See Firewall and Proxy Server Allowlist Reference for the list of IP addresses and domain names.
Resources required per node: Minimum 4 vCPUs and 14 GB RAM. For a complete list of resource requirements, see the Customer Edge Site Sizing Reference guide.

Restrictions

When Offline Survivability is enabled and the Site is offline, the following apply:

The logs after five (5) minutes (or 8 MB worth of collection) are lost.
The metrics after two (2) hours are lost.

Enable Offline Survivability for a Site

You can enable Offline Survivability either during the Site creation or you can edit an existing Site configuration. In case of existing Site edit, enabling Offline Survivability results in restarting of several Site services.

This set of instructions covers enabling Offline Survivability for an existing Site.

Step 1: Navigate to your site and open the site edit form.

Select the Multi-Cloud Network Connect service.
Select Manage > Site Management and then select the type of site you want to edit.
Select ... > Manage Configuration to view your site's configuration.
Select Edit Configuration in the upper right corner to edit its configuration.

Step 2: Enable Offline Survivability and apply the changes.

Important: Enabling Offline Survivability for an existing Site restarts its pods and services. Wait for the Offline Survivability status in the Site dashboard to show as Enabled before resuming Site operations. For more information, see the Verify Offline Survivability chapter.

Go to Advanced Configuration section and enable the Show Advanced Fields option.

Note: Although the Offline Survivability mode option is the same for all cloud site view objects, the advanced configuration options might vary, depending on the cloud provider type.

Select Enable Offline Survivability Mode for the Offline Survivability Mode field.

Figure: Enable Offline Survivability Mode

Select Save and Exit.

Enable Offline Survivability for a Site in a Full Mesh Site Mesh Group

In case of a Site which is a member of a Full Mesh Site Mesh Group, perform the following to enable Offline Survivability:

Step 1: Enable Offline Survivability for all member sites in the Site Mesh Group.

Follow the instructions in the previous chapter to enable Offline Survivability Mode field for each member Site of the full Site Mesh Group.

Step 2: Navigate to your Site Mesh Group and open its edit form.

Go to Manage > Networking > Site Mesh Groups page.
Select ... > Manage Configuration for your Site Mesh Group object.
Select Edit Configuration in the upper right corner to edit its configuration.

Step 3: Enable Offline Survivability mode.

From the Mesh Choice menu, select Full Mesh.
From the Full Mesh Choice menu, select Control and Data Plane Mesh.

Figure: Site Mesh Group Control and Data Plane Mesh Choice

Select Save and Exit to save configuration.

Enable Offline Survivability for a Site in a DC Cluster Group

For a Site which is a member of a DC Cluster Group, perform the following to enable Offline Survivability:

Step 1: Enable Offline Survivability for all member sites.

Follow the instructions in the Enable Offline Survivability for a Site chapter to enable Offline Survivability Mode field for each member Site of the DC Cluster Group.

Step 2: Navigate to the DC Cluster Group.

Go to Manage > Networking > DC Cluster Groups page.
Select ... > Manage Configuration for your DC Cluster Group object.
Select Edit Configuration in the upper right corner to edit its configuration.

Step 3: Enable Offline Survivability mode.

From the Mesh Type menu, select Control and Data Plane Mesh.

Figure: DC Cluster Group Control and Data Plane Mesh Choice

Select Save and Exit to save configuration.

Verify Offline Survivability

You can verify the status of Offline Survivability using the Site monitoring and Site Mesh Group monitoring (in case it is enabled for Site Mesh Group). Do the following to verify that the Offline Survivability is functional for your Site:

Step 1: Verify Offline Survivability from Site monitoring.

In the Multi-Cloud Network Connect service, click Overview > Infrastructure > Sites.
Click on the Site (for which Offline Survivability is enabled) from the list of sites displayed.
In the dashboard, in the System Metrics section, check for the Local Control Plane Status metric. This indicates whether the control plane components are enabled and functioning normally.

Figure: Site Monitoring - Dashboard View with Offline Survivability

In the dashboard, inspect the Software Version tile located on the right to check the status of Offline Survivability. The value Enabled indicates that Offline Survivability is enabled.
Switch to the System Metrics page and select the Local Control Plane Status tile in the Status metrics section (on the bottom left). This metric shows the graphical trend for the local control plane.

Figure: Site Monitoring - System Metrics View for Local Control Plane

In addition, navigate to the Site Status page and check the F5 Software Status tile to view the status of Offline Survivability.

Step 2: Verify Offline Survivability from Site Mesh Group monitoring.

In the Multi-Cloud Network Connect service, click Overview > Networking > Topology.
Click on the SMG tile (for which Offline Survivability is enabled) from the list of SMG tiles displayed.
In the displayed connectivity topological view, place the mouse pointer over any of the nodes (representing Sites) or the edges (lines between the Sites).

Figure: Site Mesh Group Monitoring - Node Quickview

Figure: Site Mesh Group Monitoring - Edge Quickview

Verify that the Connection Status is shown as Up.
Click on any node (Site) enabled with Offline Survivability mode to view its quick details in a sliding window. Verify that the Offline Survivability field shows Enabled value.

Figure: Site Mesh Group Monitoring - Node Details View

Select the Health tab to view the system health metrics for that Site. Verify that the Local Control Plane Status field shows the Up value.

Figure: Site Mesh Group Monitoring - Node Details View

Note: Clicking on the value for local control plane status in the Health tab switches the view to the System Metrics page of Site monitoring.

Step 3: Verify Offline Survivability from DC Cluster Group monitoring.

In the Multi-Cloud Network Connect service, click Overview > Networking > Topology.
Click on the DCG tile (for which Offline Survivability is enabled) from the list of DCG tiles displayed.
In the displayed connectivity topological view, place the mouse pointer over any of the nodes (representing Sites) or the edges (lines between the Sites).
Verify that the Control Plane Status is shown as Up.

Figure: DC Cluster Group Monitoring View

Click on any node (Site) enabled with Offline Survivability mode to view its quick details in a sliding window. Verify that the Offline Survivability field shows Enabled value.
Select the Health tab to view the system health metrics for that Site. Verify that the Local Control Plane Status field shows the Up value.