Create KVM Site

• Log into F5® Distributed Cloud Console.

• Click Multi-Cloud Network Connect.

• Select Manage > Site Management > Site Tokens.

• Click Add Site Token to create a new token.

• In the Name field, enter the token name.

• In the Description field, enter a description for the token.

• Click Save and Exit.

• Find the token previously created or choose an existing token from the list of tokens displayed.

• Click > to expand the token details in JSON format and note down the value of the uid field.

Enter egrep '(vmx|svm)' /proc/cpuinfo | wc -l. This command returns a non-zero number if virtualization is enabled.

          egrep '(vmx|svm)' /proc/cpuinfo | wc -l
        

Note: For Ubuntu, you may need to run the sudo apt install cpu-checker command prior to continuing. This action will install the cpu-checker package.

Enter sudo kvm-ok. This command returns a message that confirms that KVM acceleration can be used.

          sudo kvm-ok
        

          KVM acceleration can be used
        

• Enter sudo apt update.

          sudo apt update
        

• Enter sudo apt install qemu qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager.

          sudo apt install qemu qemu-kvm libvirt-daemon-system libvirt-clients bridge-utils virt-manager
        

• Enter sudo yum update.

          sudo yum update
        

• Confirm with y to perform the update.

• Enter sudo yum install qemu-kvm qemu-img virt-manager libvirt libvirt-client virt-install virt-viewer libvirt-python bridge-utils.

          sudo yum install qemu-kvm qemu-img virt-manager libvirt libvirt-client virt-install virt-viewer libvirt-python bridge-utils
        

• Confirm with y to perform the installation.

• For the hypervisor host on Ubuntu, add the following in the Grub configuration file (/etc/default/grub):

  • GRUB_CMDLINE_LINUX="default_hugepagesz=2M hugepagesz=2M hugepages=1200"

          GRUB_CMDLINE_LINUX="default_hugepagesz=2M hugepagesz=2M hugepages=1200"
        

• For the hypervisor host on RHEL, add the following to the /etc/sysctl.conf file:

  • vm.nr_hugepages = 1200

          vm.nr_hugepages = 1200
        

• For Ubuntu, run the following commands in the terminal:

  • sudo update-grub

  • sudo grub-mkconfig -o /boot/grub/grub.cfg

          sudo grub-mkconfig -o /boot/grub/grub.cfg
        

• Reboot your system.

• For RHEL, run the following commands in the terminal:

  • sysctl -p.

  • Confirm the output is correct.

  • Reboot your system.

• Enter cat /proc/meminfo | grep Huge.

          cat /proc/meminfo | grep Huge
        

• Confirm the configuration settings are correct.

          AnonHugePages:   6100992 kB
ShmemHugePages:        0 kB
HugePages_Total:    1200
HugePages_Free:     1200
HugePages_Rsvd:        0
HugePages_Surp:        0
HugePagesize:       2048 kB
        

• In the terminal, use the touch command to create an XML file with a .xml extension.

• Open the XML file using vim or nano text editors.

• Populate the network fields. Set a different subnet other than the underlay network.

This example provides sample file entries:

          <network>
  <name>virtualnetwork1</name>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='bridge1' stp='on' delay='0'/>
  <ip address='192.168.122.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.122.2' end='192.168.122.254'/>
    </dhcp>
  </ip>
</network>
        

• Save your information and close the text editor.

In the terminal, enter sudo virsh net-define <virtual-network-name>.xml.

          sudo virsh net-define <virtual-network-name>.xml
        

• To start the network, enter sudo virsh net-start <virtual-network-name>.

          sudo virsh net-start <virtual-network-name>
        

• To enable autostart, enter sudo virsh net-autostart <virtual-network-name>.

          sudo virsh net-autostart <virtual-network-name>
        

To list the libvirt networks, enter sudo virsh net-list.

          sudo virsh net-list
        

This example provides sample output:

          Name                 State      Autostart     Persistent
----------------------------------------------------------------
default            active        yes                 yes
virtualnetwork1    active        yes                 yes
        

Enter brctl show.

          brctl show
        

This example provides sample output:

          bridge name     bridge id               STP enabled     interfaces
virbr0          8000.5254003339b3       yes             virbr0-nic
virbr1          8000.52540060f86e       yes             virbr1-nic
        

• In the terminal, enter sudo qemu-img create /var/lib/libvirt/images/volterra.qcow2 45G.

This example creates a virtual disk file with name volterra.qcow2 45G:

          sudo qemu-img create /var/lib/libvirt/images/volterra.qcow2 45G
        

This list provides the option descriptions for the virt-install command:

• name: Name of the guest virtual machine.

• ram: RAM allocation for guest virtual machine (minimum 8 GB, recommended 16 GB).

• vcpus: The CPU allocation for guest virtual machine (minimum 4).

• network: The name of the virtual network that the Customer Edge (CE) server should be attached to. You can also specify bridge for network.

• disk path: The virtual disk path.

• cache: Ensure that the cache=none option is specified in the command.

Note: KVM installation is supported only for the SCSI disk type and virtio model for network type.

• Run the virt-install command with the following options set:

          sudo virt-install --name Volterra --ram 14000 --vcpus=4 --network network=virtualnetwork1 model=virtio --accelerate -–disk path=/var/lib/libvirt/images/volterra.qcow2 bus=virtio cache=none size=64 --cdrom /scratch/RHEL-9.2023.29-Installer.iso  --noautoconsole --noreboot
        

Start the virtual machine using the virsh command. Note that in some cases the virtual machine may not start yet.

• Enter sudo virsh start <node-name>.

          sudo virsh start <node-name>
        

• Enter sudo virsh list –all.

          sudo virsh list --all
        

This example provides the sample output of the command:

           Id    Name                           State
----------------------------------------------------
 31    ce-volterra                  running
        

• Enter sudo virsh console 31.

          sudo virsh console 31
        

This example provides the sample output of the command:

          Connected to domain ce-volterra
Escape character is ^]

This is localhost (Linux x86_64 4.19.56-coreos-r1) 22:53:47
SSH host key: SHA256:aIFHRiKcjFWaF47+9AAd6IGzCQ5M6suHzL0xH3QUaS4 (ECDSA)
SSH host key: SHA256:hmWTu9M5GYKDQcTgks7LDTkQK9LXKhQjSodVO14JICo (ED25519)
SSH host key: SHA256:ewojrrFELDNOZWIW+jp2UgSaQVikpnGupdbu9+ObgZw (RSA)
SSH host key: SHA256:dM0wUNcFqfJxVRGwwzOJBLtTY1JdptmtjyHXthnBKqY (DSA)
eth0: 192.168.122.197 fe80::5054:ff:fe89:abb8
        

Enter sudo virsh domifaddr ce-volterra.

          sudo virsh domifaddr ce-volterra
        

This example provides the sample output of the command:

           Name       MAC address          Protocol     Address
-------------------------------------------------------------------------------
 vnet0      52:54:00:89:ab:b8    ipv4         192.168.122.197/24
        

• For RHEL: Launch the application from Applications > System Tools > Virtual Machine Manager.

• For Ubuntu: Launch the application from Applications > Virtual Machine Manager.

• From a terminal, enter virt-manager.

• Click Edit > Connection Details.

• Follow the instructions at CREATING A VIRTUAL NETWORK to complete the virtual network set up.

• Click Create a new virtual machine.

• Select Import existing disk image.

• Confirm the installation information is correct, and then click Forward.

• Click Browse to navigate to and then select the file.

• Click Forward.

• In the Memory field, use the + or – buttons to set the memory amount.

• In the CPUs field, use the + or – buttons to set the number of CPU cores.

• Click Forward.

• Confirm the option for Enable storage for this virtual machine is enabled.

• Under the Create a disk image for the virtual machine field, use the + or – buttons to set the amount of storage.

• Click Forward.

• From the Network selection drop-down menu, select Virtual network ‘default’ : NAT.

• Confirm all other settings are correct.

• Click Finish.

• For the NIC driver, confirm the settings are correct, or adjust them and then click Apply.

• For the CPU(s), confirm the settings are correct, or adjust them and then click Apply.

• Click Finish.

• Start the virtual machine from the Virtual Machine Manager GUI options.

• Log into the node using the default credentials of admin for the username and Volterra123 for the password. If you did not sign in previously, you will be prompted to update the default password for the admin username. Follow the instructions to update the default password.

• Press the Tab key to select parameters to configure. The login terminal loads with different options to select for configuration.

• Select get-config.

• Confirm the settings are correct.

• Select configure-network. Follow the prompts to configure network settings.

Note: You cannot change an IP address for a registered node for a multi-node site. You must use fixed IP addresses or DHCP addresses with a fixed lease. When configuring an HTTP proxy server, IPSec tunneling is not supported.

• Optionally, configure your Wi-Fi network. If you want to apply static configuration, then this option can be used. Perform the following:

  • For the Do you want to configure wifi? option, enter Y and follow the on-screen instructions.

You must first perform network configuration using the configure-network option before setting the other fields using the configure option, in case you are applying static configuration for the network. Also, note that changing the assigned IP address after the successful registration of the node is not supported in cases of multi-node sites.

• Press the Tab key to select the configure option.

• Enter the registration token.

• Enter a site name.

• Enter a hostname. Ensure that hostnames are unique if you are installing nodes for a multi-node site.

• Enter the longitude and latitude information.

• Optionally, enter a fleet name.

• Select kvm-voltmesh for the certified hardware, if not done so previously.

Certified Hardware: The supported options are as follows.

• kvm-voltmesh: Single service level objective eth0 interface hardware.

• kvm-voltstack-combo: For managed k8s and vk8s use cases.

• kvm-regular-nic-voltmesh: Two virtual NICs, when eth0 is SLO dedicated and eth1 is binded as regular interface, which can be optionally configured from Console post registration.

Note: The default option is kvm-regular-nic-voltmesh, and this supports 2 interfaces (ensure that both eht0 and eth1 are available). However, this is loaded by default with eth0 interface set as a dedicated site local outside interface. The other interface is an optional regular interface and can be only configured from Console using fleet after the site is provisioned. For instructions on configuring interfaces using fleet, see Create a Fleet document.

Note: For more information on choosing an appropriate image for your certified hardware, see How to choose an image for your site deployment?.

Enter Y to confirm configuration.

• Press the Tab key to select the health option. Use the Enter key to scroll down for more information.

• Verify your network configuration and registration status.

Note: You can select the factory-reset option to perform a configuration reset and repeat the registration process again per the instructions below.

Note: If you use an NTP server, ensure that the server is reachable. Else, leave the NTP server configuration empty so that the F5 Distributed Cloud NTP servers are used.

• Use the q key to exit the health information page.

• Log into Console.

• Click Multi-Cloud Network Connect.

• Click Manage > Site Management > Registrations.

• Under Pending Registrations, find your node name and then click the blue checkmark.

• In the form that appears, fill in all required fields with the asterisk symbol (*).

• If you did not previously, enter a latitude value and a longitude value.

• Enter other configuration information, if needed.

• Click Save and Exit.

It may take a few minutes for the site registration information to update.

• Click Sites.

• Click on your site name. The Dashboard tab appears, along with many other tabs to inspect your site.

• Click the Site Status tab to verify the following:

  • The Update Status field has a Successful value for the F5 OS Status section.

  • The Update Status field has a Successful value for the F5 Software Status section.

  • The Tunnel status and Control Plane fields under the RE Connectivity section have up values.

• Log into Console.

• Click Multi-Cloud Network Connect.

• Click Manage > Site Management > Registrations.

Registration requests are displayed in the Pending Registrations tab.

• Click Accept to accept the registration requests from the master-0, master-1, and master-2 nodes. Your node names will differ.

• Enter the same values for the following parameters for all the registration requests:

  • In the Cluster name field, enter a name for the cluster. Ensure that all master nodes have the same name.

  • In the Cluster size field, enter 3. Ensure that all master nodes have the same cluster size.

• Enter all mandatory fields marked with the asterisk (*) character.

It may take a few minutes for the site health and connectivity score information to update.

• Click Sites.

• Click on each of your site names. The Dashboard tab appears, along with many other tabs to inspect your site.

• Click the Site Status tab to verify the following:

  • The Update Status field has a Successful value for the F5 OS Status section.

  • The Update Status field has a Successful value for the F5 Software Status section.

  • The Tunnel status and Control Plane fields under the RE Connectivity section have up values.

Terraform can be located on a different node than the host running KVM. In the main.tf file cloned in Step 4, you can change the uri variable under provider to point to your KVM location.

In a terminal, enter terraform version. If you need to install, follow the instructions at the official guide.

• To confirm hardware virtualization is supported by your CPU, enter egrep -c '(vmx|svm)' /proc/cpuinfo. This command returns a non-zero number if virtualization is enabled. For example, if a 2 is returned, then your system supports virtualization. If a 0 is returned, then your system does not support virtualization.

• To confirm KVM acceleration for Ubuntu-based systems only, enter sudo kvm-ok. Confirm the output is the following:

          INFO: /dev/kvm exists
KVM acceleration can be used
        

• For Ubuntu only:

  • If sudo kvm-ok does not work, enter sudo apt install cpu-checker. Then rerun the sudo kvm-ok command.

For your specific Linux distribution, the installation will be different.

For information on Ubuntu, see KVM Installation.

For information on Red Hat, see Installing KVM packages on an existing Red Hat Enterprise Linux system.

          sudo adduser \`id -un\` libvirt
        

          sudo adduser \`id -un\` kvm
        

• Navigate to KVM-CE and use Git to clone the repository folder to your desired location.

• In a terminal, navigate to the directory where the cloned repository folder is.

• Enter cd kvm-ce-main/terraform/.

• Confirm that the following folder and three files now exist in the current directory with ls -lh:

  • cloudinit

  • cpu-pinning.xsl

  • main.tf

  • var.tf

Open the var.tf file and provide all necessary values for the variables. For information on the values, see KVM CE Deployment.

• Add another interface (eth1 - inside) in the main.tf (kvm-ce libvirt_domain resource).

              network_interface {
        network_name = "default"
    }
     network_interface {
        network_name = "inside"
    }
        

• In Console, configure your Fleet with interface assignment to outside and inside. See example below.

• Use the wget command to download the KVM CE qcow2 image.

• Enter qemu-img resize command to resize the virtual disk storage. For example: qemu-img resize rhel-9.2023.29-20231212011947.qcow2 50G.

• In the var.tf file, update the kvm-ce-qcow2 variable to point to the local qcow2 image downloaded previously.

• Enter terraform init.

• Enter terraform apply.

In Console, navigate to the list of sites to register and confirm your site is on the list.