Networking

Virtual Networks and Network Interfaces

Virtual network is a collection of endpoints that have unique ip address, can talk to each other, and are isolated from others. In the networking world, virtual network is also referred to as a VRF (Virtual Routing and Forwarding). In some cases, the virtual network may only be present on one site and in other cases, it may need to be extended across multiple sites as shown in the diagram below. In order to extend the virtual network across multiple sites, the system leverages F5® Distributed Cloud Fabric to extend the network across sites. Multiple physical and logical interfaces in a site can belong to a given virtual network.

Conceptually, each site has an independent virtual L3 router that is responsible for routing and forwarding for that virtual network.

Local Virtual Network

A local virtual network is only present locally on the site. In case of managing many sites together as fleet, it is advantageous to configure a virtual network once for all sites. However, we may not want to connect all of these sites directly. So in configuration we can create one network and mark it as type “local”.

For example, if the sites represent a retail store, then one can create a fleet of retail store. In this fleet a virtual network of type local network is created, that represents inside network of a given store. Same config is replicated across every store and since the network is assigned to be of type local, it is not automatically extended across the fabric and available across multiple sites.

Global Virtual Network

In some cases, it may be necessary to create a virtual network that lets some endpoints on multiple sites talk to each other. In this case virtual network is created with type global. Global networks come with inherent security risks. As connectivity without carefully crafted policy can create large attack surface area. For example. in the same example of retail store, if the enterprise wants to create a virtual backbone connecting all stores together they can create a global network and the network will be extended across all sites.

Configuring Virtual Networks

There are many types of virtual networks that are present on each site. Some of them are bootstrap networks and some are configured by the user. Following types of virtual networks may be present in the system -

• Per-Site - Even though this network has been configured once globally, it is instantiated as an independent network on each of the site. As a result, this network that is instantiated on one site cannot talk to same network instantiated on another site.

• Global - This network is exactly the same across all sites on which it is instantiated. Any endpoint that is a member of this network can talk to other members irrespective of what site they belong.

• Site-Local : There can only be one network of this type on a particular site and is automatically configured by the system during bootstrap. This can also be considered as a site local outside network and needs access to public internet for registration during the site bring up process.

• Site-Local-Inside: There can only be one network of this type on a particular site and is automatically configured during bootstrap for sites with two interfaces (default gateway).

• Public: This is a conceptual virtual network that represents the Public Internet. Its is only present on F5 Distributed Cloud Regional Edge sites and not on customer sites.

• Site-Local-Service: This is another conceptual virtual network and it represents service network of kubernetes within the site. This is the network from which cluster IP for k8s service configuration is allocated.

• VER-Internal: This is another conceptual virtual network and it represents where all control plane services for the site exist. This is only used if control plane needs access to tenant services. For example, a site needs access to customer’s secrets that are stored in Hashicorp Vault running in another site.

Details on all the parameters that can be configured for this object is covered in the API specification.

Configuring Network Interfaces

There are two types of interfaces - “physical” and “logical” that can be configured on a fleet:

1. For every networking device (eg. eth0, eth1) on a node within a site, you have the option to configure the network interface, this type of interface object is considered as a “physical” interface.

2. “Logical” interfaces are child interfaces of a physical interface, for example, VLANs

Both logical and physical interfaces need to be configured with a minimum of the following two things:

1. Membership to a virtual network
2. IP Address - Static IP or DHCP. If the interface is configured for static ip, then optionally it can be configured to serve DHCP request on that interface.

There is no need to configure interfaces that are present in a certified hardware. However, there is a need for an object to be present if there is a need to attach labels to these bootstrap interface. The reason one would attach label to an interface is to enable network firewall.

Details on all the parameters that can be configured for this object is covered in the API specification.

Configuring Virtual Networks to a Site

There are two methods to configure virtual networks in a site:

1. Physical and logical Interfaces can be added to a virtual network and all endpoints that can be accessed using these interfaces will automatically become members of this virtual network. For example, if eth0 on a particular site was added to VN1 and eth0 was configured with subnet1 and a default gateway on subnet1, then all endpoints that are accessible by eth0 will become members of VN1. Also, please note that all endpoints need to have unique IP addresses for them to be a member of virtual network.

2. Another method to configure a global virtual network on a site is to connect it to an existing virtual network on the site. This connection is achieved using a network-connector object that is defined in the next section.

Details on all the parameters that can be configured for this object is covered in the API specification.

---

Network Connectors

There are many cases where there is a need to connect virtual networks. There are three scenarios that need to be considered:

1. Two-way connector - When two networks, virtual-network-1 and virtual-network-2, don't have overlapping IP addresses and are unique, they can be connected directly. Virtual routers of these networks exchange full routing information with each other and endpoints within the network can talk to each other directly. If the local network on a site is more complex than a simple subnet on the interface, dynamic routing will need to be configured on the site.

2. One-way connector - When two virtual networks need connection and one of the networks cannot guarantee unique ip-addresses (inside-network), then it needs to be connected using one-way connection to another virtual network (outside-network). In order to guarantee unique ip-address on the outside-network, this connection will be made using SNAT or forward proxy to prevent overlapping addresses.

   Sometimes, it is very hard to coordinate and maintain unique IP address assignments across teams or large number of sites. Consider virtual-network-1 in each of a site that has the same IP address subnet assigned as another site. Since these two sites have the subnet, IP addresses may be overlapping and these two sites cannot talk to each other. In this case, virtual-network-2 can be created which is “global” network that is present on each site. Virtual-network-1 can be connected to virtual-network-2 with one way connector. Endpoints in virtual-network-1 can initiate connection to endpoints in virtual-network-2, but not the other way around. This is achieved by doing SNAT or forward proxy. In this case, virtual-network-1 will be considered inside-network and virtual-network-2 will be outside-network.

3. Reverse Proxy connector - If the IP addresses are overlapping on both virtual-network-1 and virtual-network-2, then the only way to connect endpoints in these two networks is to use a reverse proxy. This functionality and its configuration will be explained in the section on proxies.

Configuring Network Connectors

A fleet object can have a list of network connectors objects. Network connector object is used to configure connectivity between two networks - to connect an inside-network to the outside-network. However any virtual network that is not of the type conceptual can be connected by network connector object.

There are a few types of connectors that can be configured:

• Direct: inside-network and outside-network are connected directly. Endpoints in each virtual network can initiate connections to other. If inside or outside are connected to other external networks, then they can be configured with dynamic routing to exchange routes (currently, only BGP is supported for dynamic routing)

• SNAT: inside-network is connected using SNAT to outside-network. Endpoints from inside-network can initiate connections to outside-network. However endpoints from outside-network cannot initiate connections to the inside-network.

• Forward-Proxy: Along with SNAT, a forward proxy can be configured. This enables the capability of inspection of HTTP and TLS connections. Using forward proxy, URL filtering and visibility of different host accessed from the inside network can be configured.

Details on all the parameters that can be configured for this object is covered in the API specification.