Deploy GPU Workload Using vK8s

You can enable GPU capability for a new App Stack site or new Fleet, or edit an existing site or Fleet object.

App Stack Site: Enable GPU capability in App Stack Site.

This step shows only how to enable the GPU in App Stack site configuration. For detailed information on App Stack site creation, see Site Management.

• From the Home screen, select Multi-Cloud Network Connect.

• Navigate to Manage > Site Management and select App Stack Sites. Select Add App Stack Site and enter a name and other required fields for your site. To edit an existing site, click ... > Manage Configuration > Edit Configuration.

• Go to the Advanced Configuration section and enable the Show Advanced Fields option.

• Select GPU Enabled option for the Enable/Disable GPU field.

Figure: App Stack Site GPU Setting

• Select Save and Exit.

Fleet: Enable GPU capability in Fleet.

For detailed information on Fleet configuration and operation, see Fleets and Vsites.

• From the Home screen, select Multi-Cloud Network Connect.

• Navigate to Manage > Site Management and select Fleets from the options. To edit an existing Fleet, click ... > Manage Configuration > Edit Configuration.

• Select Add Fleet and enter a name for your fleet in the metadata section.

• Enter a label in the Fleet Label Value field in the Fleet Configuration section.

• Go to Advanced Configuration section and enable the Show Advanced Fields option.

• Select GPU Enabled for the Enable/Disable GPU field.

• Go to Enable Default Fleet Config Download section and enable the Show Advanced Fields option.

• Select the Enable Default Fleet Config Download checkbox.

Figure: Fleet Configuration Enabled with GPU

• Select Save and Exit to create the fleet.

For detailed instructions on fleets, see Create Fleet document.

• In the Multi-Cloud Network Connect service, navigate to Manage > Site Management and select the option (AWS VPC Sites, AWS TGW Sites, GCP VPC Sites, or App Stack Sites) that applies to the site where you want to deploy the GPU application.

• For that site, select ... > Edit.

• Select the Labels field and then select ves.io/fleet from the list of keys and assign the value created in the previous step.

• Select Save and Exit.

• Repeat the above steps for all the sites of your choice.

Note: In case you apply a fleet label to a site which does not have hardware support for GPU, the GPU applications do not get deployed on that site.

Note: Ensure that the vK8s object is associated with virtual site that groups all the App Stack sites enabled with GPU or sites that are part of the fleet created in previous steps.

• From the Home screen, select Distributed Apps under All Services.

• Select your application namespace from the namespace dropdown list to change to that namespace.

• Select Applications in the configuration menu and Virtual K8s in the options pane.

• Select ... > Kubeconfig for the created vK8s object to download its kubeconfig file.

• Enter an expiration date for the credential and select download Credential.

• Set environment variable for the downloaded kubeconfig in your local machine.

export KUBECONFIG=<vK8s-kubeconfig>

Note: In case of deployment using kubectl, setting KUBECONFIG variable deploys the resources to the vK8s.

You can deploy the application workload from Console or using Kubectl. This example shows deploying using Kubectl.

• Create a file in your local machine and enter the GPU application manifest in JSON or YAML format. Ensure that the manifest resources should specify that a GPU is required. See the following resources sample for NVIDIA GPU:

spec:
containers:
- name: nvidia-pytorch
image: "nvcr.io/nvidia/pytorch:18.05-py3"
resources:
requests:
nvidia.com/gpu: 1

• Deploy the workload using the application manifest file and the kubeconfig downloaded in previous step. Use the following sample command:

kubectl apply -f k8s-app-manifest.yaml --kubeconfig vk8s-kubecfg.yaml

Note: In case of continuous use of GPU such as video monitoring applications, it is recommended to use Kubernetes deployment. In other cases, it is recommended to use a Kubernetes job so that the GPU is released after the task is completed.

You can verify that the sites are enabled with GPU and the application processes are consuming the enabled GPU resources.

• Log into your node and check the GPU processes. This example shows command to monitor the GPU devices of Nvidia EGX.

nvidia-smi

Note: The nvidia-smi command displays the information on the GPU devices and the running processes for that GPU.

• Log into Console and navigate to the Sites > Site List page. Select a site that is part of the fleet you created. This opens the site dashboard. Select the Nodes tab and then select a node to open its dashboard. Select Metrics to monitor the GPU usage, GPU temperature, and GPU throughput metrics.