Check out the new F5 Distributed Cloud Technical Knowledge Hub

Objective

This guide provides instructions on how to create multiple interfaces for virtual machine (VM), container, or pod workloads running in a Kubernetes cluster within an F5® Distributed Cloud App Stack site.

Overview

In your App Stack site configuration form, you enable this feature which auto-installs KubeVirt in your cluster. The first interface of the virtual machine or container is always connected to the default pod network. You can create subnets and attach them to additional interfaces of a virtual machine or container. Virtual machines and containers within the same site (across master and worker nodes) are connected only within a subnet, not across subnets. This allows your virtual machine and container workloads to run as pods inside a Kubernetes cluster. You can manage virtual machine workloads with Kubernetes, without having to convert them to containers.

You can also add VLAN Layer 2 interfaces to VMs and pods. These Layer 2 interfaces act as subnets and have access to the underlay network and bypass the F5 Distributed Cloud Services node data plane. The Layer 2 IP addresses are allocated using an external Dynamic Host Configuration Protocol (DHCP) server or can be static within the VM or pod itself. Different types of subnets can be attached to VMs and Pods running on Layer 2 interfaces.

The Subnets and Network Interfaces configuration objects in Distributed Cloud Services create a Kubernetes Network Attachment Definition on an F5 Distributed Cloud App Stack site using the bridge CNI plug-in. Layer 2 network interfaces must not have Subnet DHCP Networks configured in the Subnet object.

Prerequisites

The following prerequisites apply:

A Distributed Cloud Services Account. If you do not have an account, see Create an Account.
An App Stack site with managed Kubernetes. For more information, see App Stack Site.
Predefined subnet and network interface objects. For more information, see the Create Subnet and Network Interfaces guides.

Enable Virtual Machine Support for Site

Enable VM support in order to auto-install KubeVirt on your site.

To confirm your existing site has VM support enable:
- Select Multi-Cloud Network Connect.
- Select Manage > Site Management > App Stack Sites.
- Find your site and then select ... > Manage Configuration.
- Confirm your site has VM support enabled under the Enable/Disable VMs support field in the Advanced Configuration section.

If you are creating a new site, select the VMs support Enabled option from the Enable/Disable VMs support drop-down menu.

Download Kubeconfig File

Navigate to your App Stack site and download the kubeconfig file based on your managed K8s configuration.

Create and Apply the Manifest File

Create a manifest file that refers to the multiple subnets for the interfaces, and then apply the VM or container configuration.

Create a Virtual Machine

In the .yml file, configure the fields per your desired configuration.
Ensure that the networkName: ves-system/<subnet-name> match the subnet(s) you created previously in Console. In the example below, networkName: ves-system/test-subnet-1 corresponds to a subnet test-subnet-1 created in Console.

Note: In the example manifest below, multus-1 and multus-2 are the additional networks that the VM interfaces will be on. Also, the example pins the VM to a specific node (master-0 in this case), but that is optional.

          apiVersion: kubevirt.io/v1alpha3
kind: VirtualMachine
metadata:
  name: test-vm-1
spec:
  running: true
  template:
    metadata:
      labels:
        kubevirt.io/name: test-vm-1
    spec:
      nodeSelector:
        kubernetes.io/hostname: master-0
      domain:
        cpu:
          threads: 1
          cores: 1
          sockets: 1
        memory:
          guest: "1G"
        devices:
          autoattachPodInterface: true # true by default, explicitly setting for clarity
          disks:
          - name: bootdisk
            disk:
              bus: virtio
          - name: cloud-init
            disk:
              bus: virtio
          interfaces:
          - name: default
            masquerade: {}
          - name: multus-1
            bridge: {}
          - name: multus-2
            bridge: {}
      terminationGracePeriodSeconds: 0
      networks:
      - name: default
        pod: {}
      - multus:
          networkName: ves-system/test-subnet-1
        name: multus-1
      - multus:
          networkName: ves-system/test-subnet-2
        name: multus-2
      volumes:
      - name: bootdisk
        containerDisk:
          image: quay.io/containerdisks/ubuntu:20.04
      - name: cloud-init
        cloudInitNoCloud:
          userData: |-
            #cloud-config
            users:
              - name: root
                ssh-authorized-keys:
                  # provide ssh public key
            ssh_pwauth: True
            password: ubuntu
            chpasswd:
              expire: False
              list: |-
                 root:ubuntu
          networkData: |
            network:
              version: 1
              config:
              - type: physical
                name: enp1s0
                subnets:
                  - type: dhcp
              - type: physical
                name: enp2s0
                subnets:
                  - type: dhcp
              - type: physical
                name: enp3s0
                subnets:
                  - type: dhcp

To apply the configuration and create the VM/container, enter the following command: kubectl apply -f <name-of-file.yml>.
To confirm the VM/container was created, enter the following command: kubectl get vms.

This example shows three virtual machines created:

          \$ kubectl get vms
NAME      AGE    STATUS    READY
fedora1   5d9h   Running   True
fedora2   5d9h   Running   True
fedora3   5d9h   Running   True

Create Pods Connected to Layer 2 VLAN

In the .yml file, configure the fields per your desired configuration. See example deployment below for a manifest file that creates two pods with Layer 2 VLAN interfaces.

          apiVersion: v1
kind: Pod
metadata:
  name: colo-a1
  annotations:
    k8s.v1.cni.cncf.io/networks: ves-system/marcel-colo-vlan10
spec:
  nodeSelector:
    kubernetes.io/hostname: ryzen1
  containers:
  - image: alpine
    command: ["sleep", "infinity"]
    imagePullPolicy: IfNotPresent
    name: alpine
    securityContext:
      privileged: true
      runAsUser: 0
---
apiVersion: v1
kind: Pod
metadata:
  name: colo-a2
  annotations:
    k8s.v1.cni.cncf.io/networks: ves-system/marcel-colo-vlan10
spec:
  nodeSelector:
    kubernetes.io/hostname: ryzen2
  containers:
  - image: alpine
    command: ["sleep", "infinity"]
    imagePullPolicy: IfNotPresent
    name: alpine
    securityContext:
      privileged: true
      runAsUser: 0

To apply the configuration and create the pods, enter kubectl apply -f <name-of-file.yml>.
To confirm the pods were created, enter kubectl get pods -o wide.

This example shows two pods created (colo-a1 and colo-a2):

          \$ kubectl get pods -o wide
NAME                          READY   STATUS    RESTARTS   AGE    IP           NODE     NOMINATED NODE   READINESS GATES
colo-a1                       2/2     Running   0          18h    10.1.3.118   ryzen1   <none>           <none>
colo-a2                       2/2     Running   0          18h    10.1.3.119   ryzen2   <none>           <none>
virt-launcher-colo-f1-fpl5z   3/3     Running   0          24h    10.1.3.109   ryzen1   <none>           1/1
virt-launcher-colo-f2-gfhkv   3/3     Running   0          24h    10.1.3.110   ryzen2   <none>           1/1

Note: You can use a CLI tool called virtctl to manage Kubernetes with your VM/container. See Manage PK8s with VMs for more information.

View Virtual Machine Interfaces

To view the interface(s) for the VM/container from a terminal, enter the following command: kubectl virt console <name>.
To view the interface(s) for the VM/container from Console, navigate to Manage K8s Overview:
- Select your site from the list.
- Select the Virtual Machines tab.

Create Workloads with Multiple Network Interfaces

On This Page: