Create Fleet
Objective
This guide provides instructions on how to create a Fleet using F5® Distributed Cloud Console (Console). See Fleet for more information.
Important: This is a legacy workflow for deploying Customer Edge (CE) sites and is not recommended to use. A new workflow for deploying Customer Edge (CE) sites has been introduced and is now Generally Available (GA). It is recommended that you use the new Secure Mesh Site (v2) workflow for all Customer Edge deployments. You can find this new workflow here.
Deprecation of this legacy workflow will be announced soon.
Fleet is used to configure infrastructure components (like nodes) in one or more F5® Distributed Cloud Services Customer Edge (CE) sites homogeneously. Fleet configuration includes the following information:
-
Software image release to be deployed on the Fleet
-
Virtual networks
-
List of interface and devices to be configured on every node
-
Connections between the virtual networks
-
Security policies applied in the Site
Associating Fleet with Site
Fleet has a field called fleet_label
. When a Fleet object is created, the system automatically creates a known_label
ves.io/fleet=
. The known_label
is created in the Shared
namespace for the tenant. A site is made a "member of Fleet" when this known_label
is added to the site. A site can have at most one known_label
of type ves.io/fleet
and hence belongs to exactly one Fleet at any given time.
When a Site becomes a member of a Fleet, all nodes in the site also become "members of that Fleet". The Fleet configuration is applied on all nodes that are members of the Fleet.
Note: You cannot add a fleet label to the following type of sites:
- F5® Distributed Cloud App Stack (App Stack) Site
- AWS VPC Site
- AWS TGW Site
- Azure VNet Site
- GCP VPC Site
Fleet and Virtual Site
Both Fleet and Virtual Site select a list of sites based on labels. But there is a major difference between Virtual Site and Fleet. Virtual sites are intersecting subsets of available sites. Fleet is a non-intersecting subset of available sites compared to other Fleets. As a result, at most one Fleet configuration is applied on a site. Also, while a Virtual Site
selects sites based on their labels (via the Virtual Site configured label selectors), the Fleet must be applied in the sites that you want to be part of that Fleet (it does not select them).
However, to enable other features like monitoring, deploying applications or jobs on sites represented by Fleet, the system will automatically create a Virtual Site in the Shared
namespace representing all sites in a Fleet.
Using the instructions provided in this guide, you will be able to create a Fleet, with all its elements, and apply to your site, making it part of this Fleet.
Prerequisites
-
A Distributed Cloud Services Account is required. If you do not have an account, see Getting Started with Console.
-
One or more sites.
Create Fleet
You can create and configure a Fleet label to apply to your sites, to perform software and operating system upgrades as well as other administrative tasks.
Step 1: Start creating Fleet in Console.
-
Log into Console.
-
Click
Multi-Cloud Network Connect
.

Figure: Console Homepage
-
Click
Manage
>Site Management
>Fleets
. -
Click
Add Fleet
.
Step 2: Add your Fleet name and metadata.
-
In the
Metadata
section, enter a mandatory Fleet name in theName
field. -
Optionally, select a label and enter a short description.

Figure: Add Fleet Name
Step 3: Configure your Fleet label, virtual networks, and software versions.
-
In the
Fleet Configuration
section, enter a mandatory value for the Fleet label in theFleet Label Value
field. -
Configure the following optional fields:
-
Select an outside virtual network from the
Outside (Site Local) Virtual Network
menu. To create a new network, clickAdd Item
. In the page that appears, configure the fields. After you finish, clickContinue
. -
Select an inside virtual network from the
Site Local Inside Virtual Network
menu. To create a new network, clickAdd Item
. In the page that appears, configure the fields. After you finish, clickContinue
. -
In the
Software Version
field, enter a software version. -
In the
Operating System Version
field, enter an operating system version.
-

Figure: Fleet Configuration
Step 4: Select or create network interfaces for the sites in your Fleet.
-
To add and configure a new bond device:
-
From the
Select Bond Configuration
menu, select if you want to configure a new bond interface withConfigure Bond Interfaces
. The option forNo Bond Devices
is set by default. -
Click
Add Item
. -
From the
Bond Device Name
menu, clickSee Common Values
. Select a value from the list displayed or type a custom name. -
In the
Member Ethernet Devices
section, clickSee Suggestions
from theMember Ethernet Devices
menu. Select a value from the options displayed or type a custom name. You can add more member devices using theAdd Item
option. -
From the
Select Bond Mode
menu, select how the bonding occurs. If you selectLACP (802.3ad)
, enter a packet interval value in theLACP Packet Interval
field. -
Click
Apply
.
-
-
Perform the following to configure non-bond interfaces:
-
From the
Select Interface Config
menu, select your interface configuration from the options available. TheDefault Interface Config
option does not require more configuration. -
If you select
List of Interfaces
, select as many interfaces as needed. To add more, clickAdd Item
. -
If you select
Legacy Device List
, clickAdd Item
and perform the following:-
Enter the required information for
Device Name
,Device Owner
,Device Instance
andUse
. -
Click
Apply
.
-
-

Figure: Fleet Network Interface
Step 5: Configure network connectors.
-
To select an existing network connector:
-
In the
Network Connectors
section, clickAdd Item
. -
From the list that appears, find and select the network connector, and then click
Select Network Connector
.
-
-
To create a new connector, click
Add Item
.

Figure: Fleet Network Connector
- Complete the configuration using the instructions provided at Network Connectors.
Step 6: Select or create a network firewall for your Fleet.
The Network Firewall object is used to configure network policies, forward and reverse proxies, and Distributed Cloud Services Fast Access Control Lists (ACLs). After this object is configured, it is applied to all sites within a Fleet and any associated virtual networks.
-
To select an existing network firewall, use the
Network Firewall
drop-down menu to search for the network firewall. -
To create a new network firewall, click
Add Item
.

Figure: Fleet Network Firewall
- Complete the configuration using the instructions provided at Network Firewall.
Step 7: Configure Fleet storage.
In the Storage Configuration
section, configure storage for your Fleet of sites using the instructions provided at Configure Storage in Fleet.
Step 8: Configure static routes and other advanced settings.
The settings in the Advanced Configuration
section enable you to configure more options for your Fleet.
Step 8.1: Configure new static route.
-
To configure a new static route:
-
From the
Select Storage Static Routes
menu, clickList of Storage Static Routes
. -
Click
Add Item
to display the form.
-

Figure: New Static Route
-
On the static route creation form, configure the following:
-
For the subnet configuration, click
Add Item
. Select the IP version for the new route, any prefixes and prefix lengths, and then clickApply
. -
In the
Nexthop
section, optionally perform configuration for route gateway, IP version, and network interface. -
Optionally, use
Add Label
to group routes together for network policies. -
Optionally, use the
Attributes
menu to select attributes for this new route. -
After you finish, click
Apply
.
-
Step 8.2: Configure other options.
-
To configure more options, click
Show Advanced Fields
and perform the following:-
To configure the cluster group, follow the instructions provided at Create App Stack Site.
-
To configure the GPU, follow the instructions provided at Deploy GPU Workload Using vK8s.
-
To configure log streaming, follow the instructions provided at Configure Streaming of Logs.
-
To configure USB device policy, follow the instructions provided at Manage USB Devices Using Fleet.
-
Step 9: Enable Fleet configuration download.
In the Advanced Configuration
section, optionally enable the default Fleet configuration download option with Enable Default Fleet Config Download
.
Note: You must enable the default Fleet configuration download if you configured the GPU and storage settings.

Figure: Enable Fleet Configuration Download
Step 10: Block services.
You can have your Fleet block services, like Web, DNS, and SSH.
In the Blocked Services
section, click Add Item
to configure blocked services.
Step 11: Complete Fleet configuration.
After you finish, click Save and Exit
.
Apply Fleet Label to Site
After your Fleet is configured, you will need to apply its label on your sites so that the sites are configured with the Fleet settings.
Step 1: Navigate to your site.
-
To edit a non-cloud site, click
Overview
, and then clickSites
. -
Find your site, click
...
and then clickManage Configuration
. -
Click
Edit Configuration
.
Step 2: Apply the Fleet label.
-
In the
Labels
box form, clickAdd Label
. Start by typing “fleet” and you will see the key-value pairves.io/fleet
. Select this key-value pair. -
Type the name of your Fleet label, and then select it.
Step 3: Save the configuration.
Click Save and Exit
.
Your site is now part of the Fleet that you previously configured.
Fleets Tab
The Fleets
tab shows a list of your fleets along with some metrics for the sites included in that fleet.
Fleets Details
- Each fleet in the list has its own section showing the number of sites in the fleet, some health statistics, and a chart showing the memory and CPU usage for each site in the fleet (Each dot represents a site).
- Hover over a site the graph to see site name, node name, and actual memory and CPU usage values.
- Click on the fleet name to see a site map of the sites in the fleet (the default) or a list of sites in the fleet (in the
Site List
tab).
Site in a Fleet
- From the
Overview
>Infrastructure
>Sites
page, go to theFleets
tab. - Find the fleet containing your site and click the fleet name.
- Use either the
Site Map
orSite List
tabs to find and click on your site, as described above.
Upgrade Sites Using Fleet
A fleet is used to perform upgrades to the site software and operating system software. You can also use fleet to add or delete devices such as network interfaces or storage devices to the fleet of sites. This document covers information on how to perform software upgrades using the fleet.
Fleet upgrades are not supported for clouds sites, such as AWS VPC, AWS TGW, Azure VNet, GCP Credentials, or F5® Distributed Cloud App Stack sites.
Note: The outcome of attempts to perform upgrades using the Fleet are dependent on the
Site Software Version Override
setting of the site that is part of the fleet. The following list describes the behavior:
If the set value of override is
Site Version Overrides
, then the version set in site always takes precedence.If the set value of override is
New Version Overrides
, then the newer version among site and fleet takes precedence.If the set value of override is
Fleet Version Overrides
, then the version set in fleet always takes precedence.
Upgrade a Fleet
The upgrade feature enables you to upgrade the software and operating system software versions for your Fleet of sites.
Step 1: Navigate to your fleet in Console.
- Log into Console, and then click
Multi-Cloud Network Connect
.

Figure: Console Homepage
- Click
Manage
>Site Management
>Fleets
.

Figure: Navigate to Fleets
- Click
...
and then clickManage Configuration
on the fleet object to which your sites are associated with.

Figure: Edit Fleet
- In the form that appears, click
Edit Configuration
from the top right corner.
Step 2: Set the site software version and operating system version.
- In the
Fleet Configuration
section, enter the software version or operating system version to which you want the sites to be upgraded in theSoftware Version
andOperating System Version
fields, respectively.

Figure: Fleet Configuration Versions
Note: Navigate to
Sites
>Site List
. Click...
and then clickEdit
for your site and ensure that theFleet Version Overrides
option is set for theSite Software Version Override
field.
-
Click
Save and Exit
to save the updated fleet configuration. -
Navigate to
Sites
>Site List
to display the available upgrades to your sites under theSW version (Current / Status)
andOS version (Current / Status)
fields.
Step 3: Verify your fleet was upgraded.
-
Navigate to
Fleets
>Overview
. -
Click on the entry for the fleet for which you set the software and operating system versions.
-
Click the
Site List
tab to display the list of sites that are part of your fleet. -
In the
SW version (Current / Status)
andOS version (Current / Status)
fields, confirm that the values match the versions you applied to your fleet configuration. The values with messagesScheduled
orIn progress
indicate that the upgrade is planned or triggered. The messageSuccessful
indicates that the upgrade was completed successfully.
Configure Multi-Node Site Network Using Fleet
This guide provides instructions on how to configure the networking configuration for each node in a multi-node site using F5® Distributed Cloud Services Fleet. All the multi-node sites belonging to the Fleet will be configured equally, and you can further enhance the security by adding a network firewall to the Fleet, to ensure consistent security policy across all the sites in the Fleet. These instructions cover the following:
-
IP address management of outgoing interfaces (toward the Internet).
-
IP address management of interfaces toward inside networks.
-
Segmentation of subnets using virtual local area networks (VLANs).
Configuration Diagram
The following diagram shows an example topology for the use case presented in this guide:

Figure: Sample Network Topology
The example in this topology sets up a multi-node site as a network gateway for inside networks. The gateway has three master nodes and a Layer 2 switch with three VLANs for segmentation of employee workstations, development servers, and test servers. To set up this topology, you need to configure the following in the Fleet object:
-
Two virtual networks: one outside network and one inside network. This example creates inside and outside networks as part of Fleet.
-
Set up the following network interfaces:
- Dedicated Interfaces on the Site Local Outside (WAN).
Note: The IP addressing must be static or a fixed DHCP lease from an upstream DHCP server. Changing of SLO IP addresses in a multi-cluster setup is currently not supported.
-
Ethernet Interface for the Site Local Inside Network. We will configure Static Node Address, a DHCP Server, and DHCP Fixed Leases for internal hosts such as printers.
-
Set up a site local breakout from the Inside to Outside network using the network connector with SNAT.
-
Finally, apply the Fleet with the above objects to your multi-node site to enable the network connectivity.
Configure Storage in Fleet
This guide provides instructions on how to set storage for your applications using the F5® Distributed Cloud Services fleet of sites. Storage configuration set using fleet gets applied to all the sites that are part of the fleet and will be accessible to the workloads on those sites. For more information on fleets, see Fleet.
Using the instructions provided in the document, you can define the following storage configuration using fleet:
-
Storage devices: Configuration for a specific hardware appliance.
-
Storage classes: Class to which the appliance belongs to and this is dependent on the appliance provider. For example, a class can be a fast pool or slow pool of devices.
-
Storage interfaces: Interfaces for storage devices.
Procedure
Use the following steps to configure storage for your applications.
Note: By default, a Distributed Cloud site deployed in AWS will support Elastic Block Store (EBC), GCP supports Google Cloud Storage, and Azure supports Azure Disk Storage.
Step 1: Navigate to the Fleets page.
-
Select the
Multi-Cloud Network Connect
service. -
Navigate to
Manage
>Site Management
>Fleets
.

Figure: Fleet list
Step 2: Edit fleet set interfaces for storage.
-
Select
...
>Manage
for the fleet object you want to define storage configuration. -
Click
Edit Configuration
in the upper right corner. -
Navigate to the
Storage Configuration
section. -
From the
Select Storage Interface Configuration
menu, selectList of Storage Interface
. -
Select an interface from the displayed list or click
Add Item
. You can add multiple interfaces using theAdd item
option.

Figure: Interfaces for Storage Devices
Step 3: Configure storage devices.
- From the
Select Storage Device Configuration
menu, selectList of Storage Devices

Figure: Storage Devices Option
-
Under the
List of Storage Devices
field, clickAdd Item
. -
Enter a name in the
Storage Device
field. Ensure that this name corresponds to the class in which the storage device falls. The classes are used by vK8s for storage related actions.

Figure: Storage Devices Parameters
- From the
Select Storage Device to Configure
menu, select an option and perform one of the following based on that option:
NetApp Trident

Figure: NetApp Device Backend LIFs
-
From the
Select NetApp Trident Backend
menu, select an option. TheONTAP NAS
is selected by default. -
Select an option from the
Backend Management LIF
menu. TheBackend Management LIF IP Address
is selected by default. Enter an IP address for the backend management logical interface in theBackend Management LIF IP Address
field. In case you select the name option, enter the backend management interface name. -
Select an option from the
Backend Data LIF
menu. TheBackend Data LIF IP Address
is selected by default. Enter an IP address for the backend data interface in theBackend Data LIF IP Address
field. In case you select the name option, enter the backend data interface name. -
Enter a username in the
Username
field. ClickConfigure
for thePassword
field. Enter your password in theSecret
page and clickBlindfold
. Wait for the Blindfold to complete encrypting your password and clickApply
. -
Enter a certificate in the
Client Certificate
field. ClickConfigure
for theClient Certificate
field. Enter your text for your secret in theSecret
page and clickBlindfold
. Wait for the Blindfold to complete encrypting your password and clickApply
. -
Enter a certificate in the
Trusted CA Certificate
field. ClickConfigure
for theTrusted CA Certificate
field. Enter your text for your secret in theSecret
page and clickBlindfold
. Wait for the Blindfold to complete encrypting your password and clickApply
.

Figure: NetApp Device Password
- Enter CIDR for your K8s nodes in the
Auto Export CIDRs
field in case auto export policy is enabled for your storage device.

Figure: NetApp Device AutoExport CIDRs
-
If you are configuring virtual storage, then in the
Virtual Storage Pools
section, enter a label and region for the storage, and clickAdd Item
one or more times to add pool labels and pool zones. -
Click
Apply
.
Pure Storage Service Orchestrator
- Enter a cluster identifier in the
Cluster ID
field. This is used to identify the volumes used by the datastore. Alphanumeric characters and underscores are allowed.
Note: Unique cluster ID is required for multiple K8s cluster using the same storage device.

Figure: Pure Storage Orchestrator Device
-
Click
Configure
under theFlash Arrays
field. -
Click
Add Item
to add a flash array endpoint.

Figure: Pure Storage Flash Arrays
- Enter an IP address in the
Management Endpoint IP Address
field.

Figure: Pure Storage Flash Array endpoint
-
Click
Configure
under theAPI Token
field. Enter the token in the secret field and clickBlindfold
. ClickApply
after the Blindfold encryption is completed. -
Optionally, select labels for this endpoint.
-
Click
Apply
. -
Click
Configure
under theFlash Blade
field. -
Click
Add Item
to add a flash blade endpoint.

Figure: Pure Storage Flash Blade
- Enter the IP address in the
Management Endpoint IP Address
field.

Figure: Pure Storage Flash Blade Endpoint
-
Click
Configure
under theAPI Token
field. Enter the token in the secret field and clickBlindfold
. ClickApply
after the Blindfold encryption is completed. -
Enter the IP address in the
NFS IP Address
field. -
Optionally, add labels for this endpoint.
-
Click
Apply
.
Note: You can change the management or NFS endpoints to specify management endpoint name or NFS DNS name.
- Click
Apply
.
Custom Storage
The custom storage classes option is used for storage devices or external storages which are deployed outside F5 Distributed Cloud Services. For example, the option allows you to configure custom storage classes for AWS, GCP, etc.
-
Select
Custom Storage
for theSelect Storage Device to Configure
field. -
Click
Add Item
.

Figure: Custom Storage Device
-
Optionally, in the
Parameter Name
andParameter Value
fields, enter the required information. -
Click
Apply
.
HPE Storage
-
Select
HPE Storage
for theSelect Storage Device to Configure
field. -
In the
Storage Name
field, enter a name. -
In the
Storage Server IP address
field, enter an IP address. -
In the
Storage server Port
field, enter a port number. -
In the
Username
field, enter the username used to connect to the HPE storage device. -
To configure the password, click
Configure
. Then perform the following:-
For the
Blindfolded Secret
option, complete the configuration by entering the secret text to blindfold. -
For the
Clear Secret
option, enter the secret text in plaintext format or Base64. -
Click
Apply
. -
Click
Apply
again to complete configuration.
-
Note: You can add multiple devices using the
Add item
option.
Step 4: Configure storage classes.
You can use default storage classes supported in K8s or you can customize the classes. If you are using default classes, ensure that the storage device names correspond to the K8s classes.
NetApp Trident

Figure: NetApp Class
- Click
Add Item
.
Pure Storage Service Orchestrator

Figure: Pure Storage Orchestrator Class
-
Select an option from the
Backend
menu. Theblock
option is selected by default. -
Optionally, enter IOPS and bandwidth limits in their respective fields.
-
Click
Add Item
.
Custom Storage Class

Figure: Storage class parameters
-
Select
Add Custom Storage Class
for theSelect Configuration for Storage Classes
field. -
Click
Add Item
under theList of Storage Classes
field. This opens theStorage Class Parameters
page. -
Enter a name for the
Storage Class Name
field. This name will appear in K8s. -
Enter a name in the
Storage Device
field. This is the storage device that will be used by this class, as entered in Step 3. -
Optionally, enter a storage class description.
-
Optionally, check the
Default Storage Class
box to make this storage class the default for the K8s cluster. -
Select
Custom Storage
for theSelect Storage Class Configuration
field. -
Enter the storage class YAML. It must have the configuration of
apiVersion: storage.k8s.io/v1
kind: StorageClass
...
-
Enter a
Reclaim Policy
. -
Optionally, check the
Allow Volume Expansion
box. -
Optionally, enter generic/advanced parameters.
-
Click
Add Item
.
HPE Storage

Figure: Storage class parameters
Complete the required parameters.
Note: You can add multiple classes using the
Add item
option.
Step 5: Complete applying storage configuration to fleet.
Click Save and Exit
to apply the storage configuration fleet.
Manage USB Devices Using Fleet
This guide provides instructions on how to manage USB devices for your sites using fleet configuration in the F5® Distributed Cloud Console (Console). See Fleet for conceptual details of the fleet.
You can enable or disable access to USB devices for your sites via the fleet configuration. By default, all USB devices are denied in the fleet, and you can allow all or attach USB device policies to manage specific devices. You can also create a USB device policy from within the fleet configuration.
Using the instructions in this guide, you can update fleet configuration to enable or disable access to all or specific USB devices for your sites that are part of the fleet.
Apply USB Device Policy Using Fleet
Log into Console and perform the following:
Step 1: Navigate to fleets and start editing fleet configuration.
- In Console, click
Multi-Cloud Network Connect
.

Figure: Console Homepage
- Click
Manage
>Site Management
>Fleets
.

Figure: Fleets
-
Click
...
>Manage Configuration
for the fleet you want to manage the USB devices. -
Click
Edit Configuration
.
Step 2: Start configuring the USB device policy.
- In the
Advanced Configuration
section, enable theShow Advanced Fields
option.

Figure: Enable Show Advanced Fields
-
Select an option from the
USB Device Policy
menu and perform configuration per the following guidelines:-
Select
Deny All USB Devices
to deny all USB devices for your sites. This option is selected by default. -
Select
Allow All USB Devices
to allow all USB devices for your sites. -
Select
USB Device Policy
to apply an existing USB device policy or create a new one.
-
This example creates a new USB device policy from within fleet configuration and applies it to a fleet.

Figure: Fleet USB Policy
-
Select
USB Device Policy
from the drop-down list. -
From the
USB Device Policy
menu, selectAdd Item
. This opens the USB device policy creation form.
Step 3: Create a USB device policy.
-
In the
Metadata
section, enter a name for the USB device policy. -
Optionally, add a description and labels.
-
In the
Allowed USB devices
section, clickConfigure
. -
Click
Add Item
. -
Configure the fields per the following guidelines:
-
In the
iSerialNumber
field, enter an index for the serial number for your device. -
In the
Vendor ID
field, enter the device vendor identifier. -
In the
Product ID
field, enter the device product identifier assigned by the manufacturer. -
In the
Class
field, enter the device class. -
In the
Subclass
field, enter the device subclass. -
In the
Protocol
field, enter the protocol for this device. -
After you finish, click
Add Item
.
-
-
Click
Apply
. You can add more than one device using theAdd Item
option. -
Click
Continue
. -
Click
Save and Exit
. -
To obtain the USB device properties using the site dashboard:
-
Navigate to
Overview
>Sites
. -
Click on your site to open its dashboard.
-
Click on the
Tools
tab. -
Select
Show USB devices
option from theSelect tool
field. -
Select a node from the list under the
Node name
field.
-
Step 4: Complete updating the fleet configuration.
-
In the
Enable Default Fleet Config Download
section, enable theShow Advanced Fields
option. -
Click on the
Enable Default Fleet Config Download
checkbox. -
Click
Save and Exit
to apply the USB device policy to the sites that are part of the fleet.
Concepts
On this page:
- Objective
- Associating Fleet with Site
- Fleet and Virtual Site
- Prerequisites
- Create Fleet
- Apply Fleet Label to Site
- Fleets Tab
- Upgrade Sites Using Fleet
- Upgrade a Fleet
- Configure Multi-Node Site Network Using Fleet
- Configuration Diagram
- Configure Storage in Fleet
- Procedure
- Manage USB Devices Using Fleet
- Apply USB Device Policy Using Fleet
- Concepts