Multi-Cloud Networking

Objective

This guide provides instructions on how to seamlessly connect and secure applications between multiple cloud networks using F5® Distributed Cloud Mesh and F5 Distributed Cloud Console.

The steps to connect and secure applications between multiple cloud networks are:

Seq
Figure: Multi-Cloud Networking and Security Setup Steps

The following images shows the topology of the example for the use case provided in this document:

Top
Figure: Multi-Cloud Networking and Security Sample Topology

Using the instructions provided in this guide, you can setup Amazon Virtual Private Cloud (Amazon VPC) site, data center cloud gateway, setup secure networking between the 2 clouds, and setup end-to-end monitoring.


Prerequisites

  • Distributed Cloud Console SaaS account.

    Note: If you do not have an account, see Create F5 Distributed Cloud Services Account.

  • Amazon Web Services (AWS) account.

    Note: This is required to deploy an site.

  • Private cloud environment (data center) with networking connectivity to internet and TOR from the hardware.

    Note: The management IP address for your hardware is required.


Configuration

The use case provided in this guide sets up F5 Distributed Cloud Services sites as gateways for the ingress and egress traffic for the two cloud networks. The datacenter gateway site is on a physical hardware in an on-premise datacenter location. This datacenter also has TOR behind which we have VM based hosts sitting on two different subnets.

The following actions outline the activities in setting up secure networking between the AWS VPC and private data center cloud.

  • Distributed Cloud Services AWS VPC Site is deployed using the Console.

  • Distributed Cloud Services VMware site is deployed on the ESXi host using the OVA template.

  • The two cloud environments are connected using the Distributed Cloud Services global network and secured using the network policies.

  • Local-breakout for hosts on the VMware site is configured. This allows inside network hosts to access the internet using SNAT. This is achieved by configuring a Fleet and related objects - Virtual Networks, Network Interfaces, and Network Connectors on the VMware site.

Note: Ensure that you keep the Amazon Elastic IP VIPs ready for later use in configuration.

Step 1: Deploy Site (Public Cloud)

The following video shows the site deployment workflow:

Perform the following steps to deploy a Site in your VPC:

Step 1.1: Start creating AWS VPC site object.
  • Select the Cloud and Edge Sites service.
  • Select Manage -> Site Management -> AWS VPC Sites in the configuration menu. Click Add AWS VPC Site.
  • Enter a name for your VPC site in the metadata section.
Step 1.1.1: Configure site type selection.
  • Go to Site Type Selection section and perform the following:
    • Select a region in the AWS Region drop-down field. This example selects us-west-2.
    • Select New VPC Parameters for the Select existing VPC or create new VPC field. Enter the name in the AWS VPC Name field and enter the CIDR in the Primary IPv4 CIDR blocks field. This example sets 192.168.32.0/22 as the CIDR.
    • Select Ingress/Egress Gateway (Two Interface) for the Select Ingress Gateway or Ingress/Egress Gateway field.

aws vpc basic
Figure: AWS VPC Site Configuration of Site Type

Step 1.1.2: Configure ingress/egress gateway nodes.
  • Click Configure to open the two-interface node configuration wizard.

  • Click Add Item in the Ingress/Egress Gateway (two Interface) Nodes in AZ section to add the Two Interface Node.

    • Select an option for the AWS AZ name field that matches the configured AWS Region. This example selects us-west-2a.
    • Select New Subnet for the Workload Subnet field, and then enter the subnet address in the IPv4 Subnet field.
    • Similarly configure a subnet address for the Subnet for Outside Interface section.

    two interface node
    Figure: Two Interface Node Configuration

    Note: This example sets 192.168.32.128/25 as the workload subnet and 192.168.32.0/25 as the outside interface subnet.

  • Then click Add Item to return to the two-interface node configuration wizard.

inside outside cidr
Figure: Ingress/Egress Gateway Nodes Configuration

Note: The Site Network Firewall configuration will be done as part of step 4 of this quick start guide.

  • Click Apply to complete the two-interface node configuration and return to the AWS VPC site object creation.
Step 1.1.3: Complete AWS VPC site object creation.
  • Select Automatic Deployment for the Select Automatic or Assisted Deployment field.
  • Select the AWS credentials object for the Automatic Deployment field.

Note: Select Create new cloud credentials to create the credentials. You will need an AWS access key ID and AWS secret access key. See Step 1 of the Secure Kubernetes Gateway quick start for more information.

  • Make a selection in the AWS Instance Type for Node field.
  • Enter the public key for remote SSH if necessary.
  • Click Save and Exit to complete creating the AWS VPC object. The AWS VPC site object gets displayed.

autodep ssh
Figure: Automatic Deployment and Site Node Parameters

Step 1.2: Deploy AWS VPC site.
  • Click the Apply button for the created AWS VPC site object. This will create the VPC site.

tf apply
Figure: Terraform Apply for the VPC Object

  • After a few minutes, click the Refresh button to verify that the site is created. The Site Admin State should be Online, and the Status should be Applied.

tf applied
Figure: VPC Object Online

  • Verify that the site is created and ready to use. Navigate to Site Connectivity -> PoP (RE) Connectivity and hover over your newly created site to see basic health and connectivity information.
  • Click on the site to see a slide-out panel with more details.
  • Click Explore Site in the panel to see the site's full dashboard.

site db
Figure: Site Dashboard and Health Details

  • Click on the Interfaces tab to check the interface status and details such as throughput. You can view inside and outside interfaces using the Inside and Outside options.

int in out
Figure: Site Dashboard Interfaces View


Step 2: Deploy Site (Private DC)

Deploying site in your private data center consists of downloading the Distributed Cloud Services site image and installing gateway site on the data center.

Note: Refer to the Prerequisites chapter for data center site deployment prerequisites.

The following video shows the data center site deployment workflow:

Perform the following steps for deploying gateway site on the data center:

Step 2.1: Get/Create site token.
  • Log into the Console and select the Cloud and Edge Sites service.

  • Navigate to Manage -> Site Management -> Site Tokens.

  • Save the site token (the value in the UID column) for use in VMware when creating a Virtual Machine (VM)

    SiteToken
    Figure: Create site token

    Note: If you have not already created a site token, use Add site token to create a token.

Step 2.2: Download and install the VMware site image on your data center.
Step 2.3: Install the VM with the OVA template.
  • Log into VSphere webclient, ESXi console to create a VM from the OVA template

  • Select Deploy a virtual machine from an OVF or OVA file and click Next.

    vm from ova
    Figure: Create New VM from OVA Template

  • Select OVA file:

    • Enter a name for the virtual site.
    • Select or drag-and-drop the OVA file.
    • Click Next.

    import ova
    Figure: Import OVA Template

  • Select an appropriate amount of storage for the VM and click Next.

select vm storage
Figure: Select VM Storage

  • Enter the deployment options:

    • Select the OUTSIDE and REGULAR (inside) network mappings.
    • Ensure that Power on automatically is checked.
    • Press `Next'.

    vm deplmt options
    Figure: VM Deployment Options

  • Enter additional settings:

    • Expand the Options section.
    • Enter the Hostname. This should match the name of the virtual site.
    • Enter the token you saved from step 2.1.
    • Enter the Cluster Name, which should also match the name of the virtual site.
    • Enter the Name for outside network interface.
    • Enter vmware-regular-nic-voltmesh for the Certified Hardware field.
    • Enter the Latitude and Longitude for your virtual site.
    • Click Next.

    vm addnl settings
    Figure: VM Deployment Options

  • Validate the settings and click Finish. The VM gets booted up.

Step 2.4: Perform site registration in Console.
  • Log into the Console, select the Cloud and Edge Sites service, and navigate to Manage -> Site Management -> Registrations.
  • Click Pending Registrations tab. Find the registration request for your site and accept the registration. Validate the information shown and then click the checkbox icon. The Registration acceptance sliding sidebar will show information about your new VM.
  • Click Save and Exit.
  • Wait for the registration to complete and the site to come up. You can find the site in the Sites -> Site List view. Click on your site to open the site dashboard and ensure that its healthscore is 100 and its interfaces are up in the Interfaces tab.
Step 2.5: Create a fleet.
  • In the Cloud and Edge Sites service, navigate to Manage -> Site Management->Fleets.
  • Click Add fleet.
  • Enter a name for your fleet and enter a label in the Fleet Label Value field. This label is later used to apply to the site.

fleet basic
Figure: Fleet Name and Label

Step 2.5.1: Configure virtual networks.
  • Use the Outside (Site Local) Virtual Network pull-down to see and click the Create new virtual network button.
  • Enter a name for your outside network and select Site Local(Outside) Network for the Select Type of Network field. Click Continue to create the network and add it to the fleet configuration.

vn outside
Figure: Outside Virtual Network

  • In the fleet configuration screen, create the inside virtual network in the same way. Click Select inside virtual network object and click Add new virtual network. Enter a name for your inside network and select Site Local Inside Network for the Select Type of Network field. Click Continue to create network and add to the fleet configuration.
Step 2.5.2: Configure network interfaces.

Go to the Network Interfaces section and configure the following:

  • Use the Select Interface Configuration pulldown to select List of Interfaces.
  • Use the Select network interface pulldown to press the Create new network interface button.
  • Enter a name for the interface, select the Ethernet Interface for the Interface Type field, and then click Configure to setup the ethernet interface.
  • Select eth1 in the Ethernet Device field.
  • Go to the IP Configuration section and select DHCP Server for the Select Interface Address Method field. Click Configure under the DHCP Server option to open DHCP server configuration. Perform the configuration as per the following guidelines:
    • Click Add Item in the DHCP Networks section to open DHCP server configuration.
    • Enter a prefix in the Network Prefix field.
    • Click Add Item in the DHCP Pools section and enter the starting and ending IP addresses.
    • Click Add Item at the bottom of the form to add the settings to the DHCP server configuration. This sets the DHCP pool, default gateway, and DNS server address.

dhcp net
Figure: DHCP Network Configuration

  • Click Apply in the DHCP Server configuration to apply the DHCP server to the ethernet interface configuration.
  • Select Site Local Network Inside in the Select Virtual Network field in the ethernet interface configuration.

eth int
Figure: Ethernet Interface Configuration

  • Click Apply to set the ethernet interface to the network interface configuration.

ni final
Figure: Network Interface Configuration

  • Click Continue to create and add the network interface to the fleet.
Step 2.5.3: Configure network connector and complete fleet creation.

This step creates a network connector with one in SNAT mode and the other in the direct mode to the global network.

  • Click Select Network Connector and click Add new Network Connector.
  • Enter a name for the network connector and click Continue and then Select Network Connector to add the network connector to the fleet. This sets the network connector to function in the default SNAT mode that connects site local inside network to site local outside network. This is used for the data center private cloud for establishing connectivity from inside subnets to outside network through the Site deployed on the VMware VM.

vmw nc
Figure: Network Connector for Private DC

  • Scroll down and click Save and Exit in the fleet configuration screen to create the fleet.

At this point, you can verify that the inside subnets can communicate with each other but accessing outside of their networks is not possible. You can use ping command to verify the same.

Step 2.5.4: Add VMware site to the fleet.
  • Click Sites -> Site List. Click ...->Manage Configuration for your VMware site to open its configuration edit form.
  • Click in the Labels field and add ves.io/fleet with the value of fleet label you created in previous step.

site to fleet
Figure: Add Fleet Label to Site

  • Click Save changes to apply fleet settings to the site.

  • Verify that the fleet interfaces are applied to the site. Check the site local UI dashboard for ethernet interfaces section. The interface Eth1 gets IP address assigned by the DHCP server configured in the fleet.

local ui eth1
Figure: Ethernet Interface Details in Local UI Dashboard

At this point, you can verify that the inside subnets can access outside networks via the Site by means of SNAT. You can verify the same with the ping command.

Note: To check connectivity over internet, you can execute ping 8.8.8.8 to Google DNS server.


Step 3: Connect Networks

Connecting networks includes configuring local-breakout for hosts on the VMware site. That is, allowing inside network hosts to access the Internet using SNAT. This is done by configuring a Fleet and related objects - Virtual Networks, Network Interfaces, and Network Connectors. This includes creating network connectors with one in SNAT mode and other in the direct mode to the global network.

After that, connect both the VMware and AWS inside networks using Distributed Cloud Services ADN.

The following video shows the workflow of connecting and securing the two networks:

Perform the following to connect and secure the two cloud networks:

Step 3.1: Create global network.
  • Log into the Console and select the Cloud and Edge Sites service.

  • Navigate to Manage -> Networking -> Virtual Networks and click Add virtual network.

  • Enter a name and select Global Network in the Select Type of Network field.

  • Click Save and Exit.

    global network
    Figure: Setup Global Network

Step 3.2: Connect the global network with the VMware site.
  • Navigate to Manage -> Site Management -> Fleets to see a list of your fleets.

  • Click ... -> Manage Configuration for the fleet you created in step 2. Then click Edit Configuration in the upper right corner to edit the fleet.

  • Scroll down to the Network Connectors section and click Select Network Connector and then Add new Network Connector.

    • Enter a name for the network connector.

    • Select Direct, Site Local Inside to a Global Network in the Select Network Connector Type field.

    • In the Global Virtual Network field, select the global network you created in the previous step.

      net connect config
      Figure: Network Connector Configuration

  • Click Continue followed by Select Network Connector to save the network connector and then Save and Exit to save the updated fleet configuration.

Step 3.3: Connect the global network with the AWS VPC site.
  • Navigate to Manage -> Site Management -> AWS VPC Sites.

  • Click ... -> Manage Configuration for the site you created in step 1. Then click Edit Configuration in the upper right corner to edit the fleet.

  • Scroll down to the Networking Config section and click Edit Configuration under Ingress/Egress Gateway (Two Interface).

  • Click the Show Advanced Fields toggle in the Advanced Options section.

  • Select Connect Global Networks in the Select Global Networks to Connect field and then click Add Item.

    • Select the global network you created previously in the Global Virtual Network field.

    • Click Add Item to save the changes.

      global netwk conn
      Figure: Global Network Connections

  • Select Manage Static Routes in the Manage Static Routes for Inside Network field.

  • In the List of Static Routes section, click Add Item.

    • Enter the network prefix in the Simple Static Route field.

    • Click Add Item.

      static route conf
      Figure: Site Static Route Configuration

  • Click Apply to add the network configuration. Click Save and Exit to save updates to the site configuration.

Now you can verify that the connectivity is enabled between the VMware subnets and the AWS cloud EC2 instances. You can use ping to verify the same.


Step 4: Secure Networks

Securing networks includes applying network policies to restrict the network accesses for chosen networks. It also includes applying forward proxy policies to allow access to chosen URLs. This is achieved by means of creating a network firewall with the policies and applying to the fleet.

This example creates a network policy that allows access only from one subnet of the private DC to the AWS cloud and blocks access for all other subnets. It also creates a forward proxy policy that blocks access to a specific domain and allows everything else.

Perform the following steps to setup secure networks.

Step 4.1 Create and add network firewall to the fleet.
  • Log into the Console and select the Cloud and Edge Sites service.
  • Click Manage -> Site Management -> Fleets. Find your fleet from the displayed list and click ... -> Edit to open its configuration form, and then click Edit Configuration.
  • Scroll down to the Network Firewall section and use the Network Firewall pull-down menu to selectCreate new network firewall. Enter a name for the firewall.
Step 4.1.1 Create and add network policies to the fleet.
  • Scroll down to the Network Policy section and select Active Network Policies. Click on the Select network policy view field and select Create new network policy view. This policy will allow all traffic.

net pol fw
Figure: Network and Policy for Network Firewall

  • Enter a name for the policy and add the prefix of a subnet (for which you want to allow access) in the IPv4 Prefix List field.

  • Click Configure on the Connections from Policy Endpoints section to configure an egress rule. Click Add Item to create the egress rule.

    • Set a name for the egress rule and select Allow for the Action field.

    • Click Add item to add the egress rule to the list of egress rules.

      allow egress rules
      Figure: Egress Allow Rules

  • Click Apply to save the changes to the Connections from Policy Endpoints section and then click Continue to return to the firewall configuration.

  • Click Add item in the List of Network policy section to add another policy. This one will deny access to a subnet.

  • Click on the Select network policy view field and select Create new network policy view.

  • Enter a name for the policy and add the prefix of a subnet (for which you want to block access) in the IPv4 Prefix List field. Click Configure on the Connections From Policy Endpoints section to configure an egress rule. Click Add Item to create the egress rule that denies traffic to a subnet.

    • Set a name for the egress rule and leave Deny for the Action field.

    • Select List IP Prefix Set in the Select Other Endpoint field.

    • Click in the reference field and then click Create new IP prefix set.

    • Enter a name for the prefix set, enter the subnet prefix, and press Continue.

    • Click Add item to add the egress rule to the list of egress rules.

      egress deny rule
      Figure: Egress Deny Rule for a Subnet

  • Click Add Item to create a second egress rule that allows all other traffic.

    • Set a name for the egress rule and select Allow for the Action field.
    • Click Add item to add the egress rule to the list of egress rules.
  • Click Apply to save the egress rule list, and click Continue to complete the second network policy.

  • Click Add Item for the local internet breakout. Then click on the Select network policy view field and select Create new network policy view.

  • Enter a name for the policy.

  • Select Any Endpoint for the Endpoint(s) field in the Policy For Endpoints section.

  • Click Configure on the Connections To Policy Endpoints section to configure an ingress rule that allows all traffic, and then clickAdd Item to create the ingress rule.

    • Set a name for the ingress rule.
    • Select Allow for the Action field.
    • Click Add item to add the ingress rule to the list of ingress rules.
    • Click Apply to add the list to the network policy.
  • Click Configure on the Connections From Policy Endpoints section to configure an egress rule for this network policy, and then clickAdd Item to create the egress rule.

    • Set a name for the egress rule.
    • Select Allow for the Action field.
    • Click Add item to add the egress rule to the list of egress rules.
  • Click Apply to save the egress rule list, click Continue to complete the third network policy, click Continue to add the network firewall to the fleet configuration, and finally click Save and Exit in the fleet configuration to save changes to fleet.

Step 4.1.2 Verify the policy operation.
  • Verify that access from only one subnet is allowed to the EC2 instances of AWS. Also, verify that the site local breakout and internet access is still allowed. Enter ping command to an EC2 instance IP address from both subnets and only one is allowed.

  • You can also verify the policy and rule hits from Console. Navigate to Manage -> Firewall -> Network Policies. Check the Hits field for your policy.

pol hits
Figure: Policy Hits

  • Click on the value in the Hits column for your policy to view the rule hits.
Step 4.2 Add a forward proxy rule that blocks a specific URL.

To enable URL filtering, update the firewall created in step 4.1.

  • Go to Manage -> Firewall -> Network Firewalls.

  • Click ... -> Manage Configuration for the firewall created earlier, and then click Edit Configuration.

  • Scroll down to Forward Proxy Policy and select Active Forward Proxy Policies.

  • Click in Select forward proxy policy and then click Create new forward proxy policy.

    • Enter a name for the new policy.

    • Select All forward Proxies on Site in the Proxy section so that it gets activated everywhere.

    • Select Denied connections in the Rules section.

    • Click Add Item in the TLS Domains section, enter the URL you wish to block in the Exact Value field, and click Add Item to add this URL to your denied TLS domains.

      fwdprx blk fb
      Figure: Enable Forward Proxy for facebook.com

  • Click Continue and then Save and Exit to save the changes to your firewall.

Step 4.3 Enable forward proxy for network connector of the private DC.
  • Go to Manage -> Networking -> Network Connectors. Click ... -> Manage Configuration for the VMware site network connector you created, and then click Edit Configuration.

nc en fprx
Figure: Enable Forward Proxy for VMware Networks

  • Click Enable Forward Proxy for the Select Forward Proxy field. Click Save and Exit.

This is required to apply the forward proxy policies.


Concepts