Multi-Cloud Networking
Objective
This guide provides instructions on how to seamlessly connect and secure applications between multiple cloud networks using F5® Distributed Cloud Mesh and F5 Distributed Cloud Console.
The steps to connect and secure applications between multiple cloud networks are:
Figure: Multi-Cloud Networking and Security Setup Steps
The following images shows the topology of the example for the use case provided in this document:
Figure: Multi-Cloud Networking and Security Sample Topology
Using the instructions provided in this guide, you can setup Amazon Virtual Private Cloud (Amazon VPC) site, data center cloud gateway, setup secure networking between the 2 clouds, and setup end-to-end monitoring.
Prerequisites
-
Distributed Cloud Console SaaS account.
Note: If you do not have an account, see Create F5 Distributed Cloud Services Account.
-
Amazon Web Services (AWS) account.
Note: This is required to deploy an site.
-
Private cloud environment (data center) with networking connectivity to internet and TOR from the hardware.
Note: The management IP address for your hardware is required.
Configuration
The use case provided in this guide sets up F5 Distributed Cloud Services sites as gateways for the ingress and egress traffic for the two cloud networks. The datacenter gateway site is on a physical hardware in an on-premise datacenter location. This datacenter also has TOR behind which we have VM based hosts sitting on two different subnets.
The following actions outline the activities in setting up secure networking between the AWS VPC and private data center cloud.
-
Distributed Cloud Services AWS VPC Site is deployed using the Console.
-
Distributed Cloud Services VMware site is deployed on the ESXi host using the OVA template.
-
The two cloud environments are connected using the Distributed Cloud Services global network and secured using the network policies.
-
Local-breakout for hosts on the VMware site is configured. This allows inside network hosts to access the internet using SNAT. This is achieved by configuring a Fleet and related objects - Virtual Networks, Network Interfaces, and Network Connectors on the VMware site.
Note: Ensure that you keep the Amazon Elastic IP VIPs ready for later use in configuration.
Step 1: Deploy Site (Public Cloud)
The following video shows the site deployment workflow:
Perform the following steps to deploy a Site in your VPC:
Step 1.1: Start creating AWS VPC site object.
- Select the
Multi-Cloud Network Connect
service. - Select
Manage
>Site Management
>AWS VPC Sites
in the configuration menu. SelectAdd AWS VPC Site
. - Enter a name for your VPC site in the metadata section.
Step 1.1.1: Configure site type selection.
- Go to the
Site Type Selection
section and perform the following:- Select a region in the
AWS Region
drop-down field. This example selectsus-west-2
. - Select
New VPC Parameters
for theVPC
field. If you want a specific name for the AWS VPC, selectChoose VPC Name
in theAWS VPC Name
field and then enter your preferred name. Otherwise, leave the drop-down selection set toAutogenerate VPC Name
. Enter the CIDR in thePrimary IPv4 CIDR blocks
field. This example sets 192.168.32.0/22 as the CIDR. - Select
Ingress/Egress Gateway (Two Interface)
for theSelect Ingress Gateway or Ingress/Egress Gateway
field.
- Select a region in the
Figure: AWS VPC Site Configuration of Site Type
Step 1.1.2: Configure ingress/egress gateway nodes.
-
Select
Configure
to open the two-interface node configuration wizard. -
Select
Add Item
in theIngress/Egress Gateway (two Interface) Nodes in AZ
section to add theTwo Interface Node
.- Select an option for the
AWS AZ name
field that matches the configuredAWS Region
. This example selectsus-west-2a
. - Select
New Subnet
for theWorkload Subnet
field, and then enter the subnet address in theIPv4 Subnet
field. - Similarly configure a subnet address for the
Subnet for Outside Interface
section.
Figure: Two Interface Node Configuration
Note: This example sets 192.168.32.128/25 as the workload subnet and 192.168.32.0/25 as the outside interface subnet.
- Select an option for the
-
Then select
Apply
to return to the two-interface node configuration wizard.
Figure: Ingress/Egress Gateway Nodes Configuration
Note: The
Site Network Firewall
configuration will be done as part of step 4 of this quick start guide.
- Select
Apply
to complete the two-interface node configuration and return to the AWS VPC site object creation.
Step 1.1.3: Complete AWS VPC site object creation.
- Select the AWS credentials object under the
Automatic Deployment
field.
Note: Select
Add Item
in theCloud Credentials
drop-down menu to create the credentials. You will need an AWS access key ID and AWS secret access key. See Step 1 of the Secure Kubernetes Gateway quick start for more information.
- Make a selection in the
AWS Instance Type for Node
field. - Enter the public key for remote SSH if necessary.
- Select
Save and Exit
to complete creating the AWS VPC object. The AWS VPC site object gets displayed.
Figure: Automatic Deployment and Site Node Parameters
Step 1.2: Deploy AWS VPC site.
- Select the
Apply
button for the created AWS VPC site object. This will create the VPC site.
Figure: Terraform Apply for the VPC Object
- After a few minutes, select the
Refresh
button to verify that the site is created. TheSite Admin State
should beOnline
, and theStatus
should beApplied
.
Figure: VPC Object Online
- Verify that the site is created and ready to use. Select
Sites
in the left navigation pane and then select your site from the list of sites at the bottom of theSites
page. Navigate toSite Connectivity
>PoP (RE) Connectivity
and hover over your newly created site to see basic health and connectivity information. - Select the site to see a slide-out panel with more details.
- Select
Explore Site
in the panel to see the site's full dashboard.
Figure: Site Dashboard and Health Details
- Select the
Interfaces
tab to check the interface status and details such as throughput. You can view inside and outside interfaces using theInside
andOutside
options.
Figure: Site Dashboard Interfaces View
Step 2: Deploy Site (Private DC)
Deploying site in your private data center consists of downloading the Distributed Cloud Services site image and installing gateway site on the data center.
Note: Refer to the Prerequisites chapter for data center site deployment prerequisites.
The following video shows the data center site deployment workflow:
Perform the following steps for deploying gateway site on the data center:
Step 2.1: Get/Create site token.
-
Log into the Console and select the
Multi-Cloud Network Connect
service. -
Navigate to
Manage
>Site Management
>Site Tokens
. -
Save the site token (the value in the UID column) for use in VMware when creating a Virtual Machine (VM)
Figure: Create site token
Note: If you have not already created a site token, use
Add site token
to create a token.
Step 2.2: Download and install the VMware site image on your data center.
- See VMware Images page for downloading
OVA image
.
Step 2.3: Install the VM with the OVA template.
-
Log into VSphere webclient, ESXi console to create a VM from the OVA template
-
Select
Deploy a virtual machine from an OVF or OVA file
and selectNext
.Figure: Create New VM from OVA Template
-
Select OVA file:
- Enter a name for the virtual site.
- Select or drag-and-drop the OVA file.
- Select
Next
.
Figure: Import OVA Template
-
Select an appropriate amount of storage for the VM and select
Next
.
Figure: Select VM Storage
-
Enter the deployment options:
- Select the
OUTSIDE
andREGULAR
(inside) network mappings. - Ensure that
Power on automatically
is checked. - Press `Next'.
Figure: VM Deployment Options
- Select the
-
Enter additional settings:
- Expand the
Options
section. - Enter the
Hostname
. This should match the name of the virtual site. - Enter the token you saved from step 2.1.
- Enter the
Cluster Name
, which should also match the name of the virtual site. - Enter the
Name for outside network interface
. - Enter
vmware-regular-nic-voltmesh
for theCertified Hardware
field. - Enter the
Latitude
andLongitude
for your virtual site. - Select
Next
.
Figure: VM Deployment Options
- Expand the
-
Validate the settings and select
Finish
. The VM gets booted up.
Step 2.4: Perform site registration in Console.
- Log into the Console, select the
Multi-Cloud Network Connect
service, and navigate toManage
>Site Management
>Registrations
. - Select
Pending Registrations
tab. Find the registration request for your site and accept the registration. Validate the information shown and then select the checkbox icon. TheRegistration acceptance
sliding sidebar will show information about your new VM. - Select
Save and Exit
. - Wait for the registration to complete and the site to come up. You can find the site in the
Sites
>Site List
view. Select your site to open the site dashboard and ensure that its healthscore is 100 and its interfaces are up in theInterfaces
tab.
Step 2.5: Create a fleet.
- In the
Multi-Cloud Network Connect
service, navigate toManage
>Site Management
>Fleets
. - Select
Add fleet
. - Enter a name for your fleet and enter a label in the
Fleet Label Value
field. This label is later used to apply to the site.
Figure: Fleet Name and Label
Step 2.5.1: Configure virtual networks.
- Use the
Outside (Site Local) Virtual Network
pull-down to see and select theAdd Item
option. - Enter a name for your outside network and select
Site Local(Outside) Network
for theSelect Type of Network
field. SelectContinue
to create the network and add it to the fleet configuration.
Figure: Outside Virtual Network
- In the
Fleet Configuration
screen, create the inside virtual network in the same way. SelectSelect inside virtual network object
and selectAdd Item
. Enter a name for your inside network and selectSite Local Inside Network
for theSelect Type of Network
field. SelectContinue
to create network and add to the fleet configuration.
Step 2.5.2: Configure network interfaces.
Go to the Network Interfaces
section and configure the following:
- Use the
Select Interface Config
pulldown to selectList of Interfaces
. - Use the
List of Interfaces
pulldown to select theAdd Item
option. - Enter a name for the interface, select the
Ethernet Interface
for theInterface Type
field, and then selectConfigure
to setup the ethernet interface. - Select
eth1
in theEthernet Device
field. - Go to the
IP Configuration
section and selectDHCP Server
for theSelect Interface Address Method
field. SelectConfigure
under theDHCP Server
option to open DHCP server configuration. Perform the configuration using the following guidelines:- Select
Configure
in theDHCP Networks
section to open DHCP server configuration. - Enter a prefix in the
Network Prefix
field. - Select
Add Item
in theDHCP Pools
section and enter the starting and ending IP addresses. - Select
Apply
at the bottom of the form to add the settings to the DHCP server configuration. This sets the DHCP pool, default gateway, and DNS server address.
- Select
Figure: DHCP Network Configuration
- Select
Apply
in theDHCP Server
configuration to apply the DHCP server to the ethernet interface configuration. - Select
Site Local Network Inside
in theSelect Virtual Network
field in the ethernet interface configuration.
Figure: Ethernet Interface Configuration
- Select
Apply
to set the ethernet interface to the network interface configuration.
Figure: Network Interface Configuration
- Select
Continue
to create and add the network interface to the fleet.
Step 2.5.3: Configure network connector and complete fleet creation.
This step creates a network connector with one in SNAT mode and the other in the direct mode to the global network.
- Select
Add Item
in theNetwork Connectors
section and then use theNetwork Connectors
drop-down menu to selectAdd Item
. - Enter a name for the network connector and select
Continue
and thenSelect Network Connector
to add the network connector to the fleet. This sets the network connector to function in the default SNAT mode that connects site local inside network to site local outside network. This is used for the data center private cloud for establishing connectivity from inside subnets to outside network through the Site deployed on the VMware VM.
Figure: Network Connector for Private DC
- Scroll down and select
Save and Exit
in the fleet configuration screen to create the fleet.
At this point, you can verify that the inside subnets can communicate with each other but accessing outside of their networks is not possible. You can use ping
command to verify the same.
Step 2.5.4: Add VMware site to the fleet.
- Select
Manage
>Site Management
>AWS VPC Sites
. - Select
...
>Manage Configuration
for your VMware site and then selectEdit Configuration
in the upper right corner to open its configuration edit form. - Select in the
Labels
field and addves.io/fleet
with the value of fleet label you created in previous step.
Figure: Add Fleet Label to Site
-
Select
Save and Exit
to apply fleet settings to the site. -
Verify that the fleet interfaces are applied to the site. Check the site local UI dashboard for ethernet interfaces section. The interface
Eth1
gets IP address assigned by the DHCP server configured in the fleet.
Figure: Ethernet Interface Details in Local UI Dashboard
At this point, you can verify that the inside subnets can access outside networks via the Site by means of SNAT. You can verify the same with the ping
command.
Note: To check connectivity over internet, you can execute
ping 8.8.8.8
to Google DNS server.
Step 3: Connect Networks
Connecting networks includes configuring local-breakout for hosts on the VMware site. That is, allowing inside network hosts to access the Internet using SNAT. This is done by configuring a Fleet and related objects - Virtual Networks, Network Interfaces, and Network Connectors. This includes creating network connectors with one in SNAT mode and other in the direct mode to the global network.
After that, connect both the VMware and AWS inside networks using Distributed Cloud Services ADN.
The following video shows the workflow of connecting and securing the two networks:
Perform the following to connect and secure the two cloud networks:
Step 3.1: Create global network.
-
Log into the Console and select the
Multi-Cloud Network Connect
service. -
Navigate to
Manage
>Networking
>Virtual Networks
and selectAdd virtual network
. -
Enter a name and select
Global Network
in theSelect Type of Network
field. -
Select
Save and Exit
.Figure: Setup Global Network
Step 3.2: Connect the global network with the VMware site.
-
Navigate to
Manage
>Site Management
>Fleets
to see a list of your fleets. -
Select
...
>Manage Configuration
for the fleet you created in step 2. Then selectEdit Configuration
in the upper right corner to edit the fleet. -
Scroll down to the
Network Connectors
section and selectAdd Item
and then selectAdd Item
from the newly createdNetwork Connectors
drop-down menu.-
Enter a name for the network connector.
-
Select
Direct, Site Local Inside to a Global Network
in theSelect Network Connector Type
field. -
In the
Global Virtual Network
field, select the global network you created in the previous step.Figure: Network Connector Configuration
-
-
Select
Continue
to save the network connector and thenSave and Exit
to save the updated fleet configuration.
Step 3.3: Connect the global network with the AWS VPC site.
-
Navigate to
Manage
>Site Management
>AWS VPC Sites
. -
Select
...
>Manage Configuration
for the site you created in step 1. Then selectEdit Configuration
in the upper right corner to edit the fleet. -
Scroll down to the
Networking Config
section and selectEdit Configuration
underIngress/Egress Gateway (Two Interface)
. -
Select
Connect Global Networks
in theSelect Global Networks to Connect
field and then selectAdd Item
.-
Select the global network you created previously in the
Global Virtual Network
field. -
Select
Apply
to save the changes.Figure: Global Network Connections
-
-
Select
Manage Static Routes
in theManage Static Routes for Inside Network
field. -
In the
List of Static Routes
section, selectAdd Item
.-
Enter the network prefix in the
Simple Static Route
field. -
Select
Add Item
.Figure: Site Static Route Configuration
-
-
Select
Apply
to add the network configuration. -
Select
Apply
to save the Ingress/Egress Gateway configuration. -
Select
Save and Exit
to save updates to the site configuration.
Now you can verify that the connectivity is enabled between the VMware subnets and the AWS cloud EC2 instances. You can use ping
to verify the same.
Step 4: Secure Networks
Securing networks includes applying firewall policies to restrict the network accesses for chosen networks. It also includes applying forward proxy policies to allow access to chosen URLs. This is achieved by means of creating a network firewall with the policies and applying to the fleet.
This example creates a firewall policy that allows access only from one subnet of the private DC to the AWS cloud and blocks access for all other subnets. It also creates a forward proxy policy that blocks access to a specific domain and allows everything else.
Perform the following steps to setup secure networks.
Step 4.1 Create and add network firewall to the fleet.
- Log into the Console and select the
Multi-Cloud Network Connect
service. - Select
Manage
>Site Management
>Fleets
. Find your fleet from the displayed list and select...
>Manage Configuration
to open its configuration form, and then selectEdit Configuration
in the upper right. - Scroll down to the
Network Firewall
section and use theNetwork Firewall
pull-down menu to selectAdd Item
. Enter a name for the firewall.
Step 4.1.1 Create and add firewall policies to the fleet.
- Scroll down to the
Firewall Policy
section and selectActive Firewall Policies
. Use theSelect Item
pull-down menu and selectAdd Item
. This policy will allow all traffic for the server1 prefix.
Figure: Network and Policy for Network Firewall
-
Enter a name for the policy and add the prefix of the server1 subnet (for which you want to allow access) in the
IPv4 Prefix List
field. -
Select
Configure
on theConnections From Policy Endpoints
section to configure an egress rule. SelectAdd Item
to create the egress rule.-
Set a name for the egress rule and select
Allow
for theAction
field. -
Select
Apply
to add the egress rule to the list of egress rules.Figure: Egress Allow Rules
-
-
Select
Apply
to save the egress rules list. -
Select
Continue
to save the server1 firewall policy
-
Select
Add item
in theFirewall policy
section to add another policy for the server2 prefix. This one will deny access to a subnet. -
Select the
Select Item
field and selectAdd Item
. -
Enter a name for the policy and add the prefix of the server2 subnet (for which you want to block access) in the
IPv4 Prefix List
field. -
Select
Configure
on theConnections From Policy Endpoints
section to configure an egress rule. SelectAdd Item
to create the egress rule that denies traffic to a subnet.-
Set a name for the egress rule and leave
Deny
for theAction
field. -
Select
List IP Prefix Set
in theSelect Other Endpoint
field. -
Select in the reference field and then select
Add Item
. -
Enter a name for the prefix set, enter the subnet prefix, and press
Continue
. -
Select
Apply
to add the egress rule to the list of egress rules.Figure: Egress Deny Rule for a Subnet
-
-
Select
Add Item
to create a second egress rule that allows all other traffic.- Set a name for the egress rule and select
Allow
for theAction
field. - Select
Add item
to add the egress rule to the list of egress rules.
- Set a name for the egress rule and select
-
Select
Apply
to save the egress rule list, and selectContinue
to complete the server2 policy.
- Select
Add Item
to add a third policy slot for the local internet breakout. Then use theSelect Item
pull-down menu to selectAdd Item
. - Enter a name for the new firewall policy.
- Select
Any Endpoint
for theEndpoint(s)
field in thePolicy For Endpoints
section. - Select
Configure
on theConnections To Policy Endpoints
section to configure an ingress rule that allows all traffic, and then selectAdd Item
to create the ingress rule.- Enter a name for the ingress rule.
- Select
Allow
for theAction
field. - Select
Apply
to add the ingress rule to the list of ingress rules. - Select
Apply
to add the list to the network policy.
- Select
Configure
on theConnections From Policy Endpoints
section to configure an egress rule for this network policy, and then selectAdd Item
to create the egress rule.- Set a name for the egress rule.
- Select
Allow
for theAction
field. - Select
Apply
to add the egress rule to the list of egress rules.
- Select
Apply
to save the egress rule list, selectContinue
to complete the third firewall policy, selectContinue
to add the network firewall to the fleet configuration, and finally selectSave and Exit
in the fleet configuration to save changes to fleet.
Step 4.1.2 Verify the policy operation.
-
Verify that access from only one subnet is allowed to the EC2 instances of AWS. Also, verify that the site local breakout and internet access is still allowed. Enter
ping
command to an EC2 instance IP address from both subnets and only one is allowed. -
You can also verify the policy and rule hits from Console. Navigate to
Manage
>Firewall
>Firewall Policies
. Check theHits
field for your policy.
Figure: Policy Hits
- Select the value in the
Hits
column for your policy to view the rule hits.
Step 4.2 Add a forward proxy rule that blocks a specific URL.
To enable URL filtering, update the firewall created in step 4.1.
-
Go to
Manage
>Firewall
>Network Firewalls
. -
Select
...
>Manage Configuration
for the firewall created earlier, and then selectEdit Configuration
. -
Scroll down to
Forward Proxy Policy
and selectActive Forward Proxy Policies
. -
Use the
Select Item
pull-down menu to selectCreate new Forward Proxy Policy
.-
Enter a name for the new policy.
-
Select
All forward Proxies on Site
in theProxy
section so that it gets activated everywhere. -
Select
Denied connections
in theRules
section. -
Select
Add Item
in theTLS Domains
section, enter the URL you wish to block in theExact Value
field, and selectApply
to add this URL to your denied TLS domains.Figure: Enable Forward Proxy for facebook.com
-
-
Select
Continue
and thenSave and Exit
to save the changes to your firewall.
Step 4.3 Enable forward proxy for network connector of the private DC.
- Go to
Manage
>Networking
>Network Connectors
. Select...
>Manage Configuration
for the VMware site network connector you created, and then selectEdit Configuration
.
Figure: Enable Forward Proxy for VMware Networks
- Select
Enable Forward Proxy
for theSelect Forward Proxy
field. SelectSave and Exit
.
This is required to apply the forward proxy policies.