Deploy vGPU Apps Using vK8s

• Log onto your vSphere platform using the vSphere client.

• Select your server under the Datacenter in the left menu. Select Hardware > Graphics under the Configure tab.

• Select the graphics device under the Host Graphics tab and click Edit.

Figure: Graphics Devices

• Select Shared Direct and Spread VMs Across GPUs (best performance) options and click Ok.

Figure: Set GPU to vGPU

Note: Restart the host or the xorg service to enforce these settings.

• Right-click on your VM and select Edit Settings.
• Click Add NEW DEVICE > New PCI Device and select the type as NVIDIA GRID vGPU.
• Select a profile for the NVIDIA GRID vGPU Profile field as per your requirement. This example selects grid_t4-4q.

Figure: Add vGPU to VM

• Click OK to add the vGPU software.

• Select the Multi-Cloud Network Connect workspace.
• Navigate to Manage > Site Management > Fleets.
• Click ... > Edit for the fleet in which your VMware site is a member.
• Go to Advanced Configuration section and enable Show Advanced Fields option.
• Select vGPU Enabled for the Enable/Disable GPU field.
• Enter the license server IP address or FQDN in the License Server Address field.
• Enter license server port number in the License Server Port Number field. Port 7070 is populated by default.
• Select a type for the Feature Type field. This example sets NVIDIA vGPU as the feature type.

Figure: Enable vGPU in Fleet

Note: You can set Unlicensed for the feature type field. However, unlicensed usage of NVIDIA vGPU software results in degraded performance based on time elapsed since the boot time. See Grid Licensing Guide for more information.

• Go to Enable Default Fleet Config Download section and enable the Show Advanced Fields option. Select Enable Default Fleet Config Download option.

• Click Save and Exit.

You can deploy the vGPU app using the vK8s workloads or jobs. This example shows deploying a vGPU app using the vK8s jobs.

Prepare a vGPU application manifest in JSON or YAML format. Ensure that the manifest resources should specify that a vGPU is required. See the following resources sample for NVIDIA vGPU:

apiVersion: batch/v1
kind: Job
metadata:
  name: vgpu-test1
spec:
  template:
    spec:
      containers:
      - name: cuda-container
        image: nvidia/cuda:11.0-base
        command: ["nvidia-smi"]
        resources:
          limits:
            nvidia.com/gpu: 1
            cpu: 200m
            memory: 600Mi
          requests:
            cpu: 0m
            memory: 100Mi
      restartPolicy: Never
  backoffLimits: 2

The following is a JSON version of the deployment:

{
  "apiVersion": "batch/v1",
  "kind": "Job",
  "metadata": {
    "name": "vgpu-test1"
  },
  "spec": {
    "template": {
      "spec": {
        "containers": [
          {
            "name": "cuda-container",
            "image": "nvidia/cuda:11.0-base",
            "command": [
              "nvidia-smi"
            ],
            "resources": {
              "limits": {
                "nvidia.com/gpu": 1,
                "cpu": "200m",
                "memory": "600Mi"
              },
              "requests": {
                "cpu": "0m",
                "memory": "100Mi"
              }
            }
          }
        ],
        "restartPolicy": "Never"
      }
    },
    "backoffLimits": 2
  }
}

Note: In case of continuous use of vGPU such as video monitoring applications, it is recommended to use Kubernetes deployment and in other cases, it is recommended to use a Kubernetes job so that the vGPU is released after the task is completed.

Note: Ensure that the vK8s object is associated with virtual site that groups all the VMware sites enabled with vGPU or sites that are part of the fleet created in previous steps.

• Select the Multi-Cloud Network Connect workspace.
• Select your application namespace from the namespace dropdown list to change to that namespace.
• Navigate to Applications > Virtual K8s.
• Click the name of the virtual K8 object where you want to deploy the vGPU app.
• Click on the Jobs tab and click Add Job.
• Paste the YAML prepared in previous step and click Save.

You can verify that the sites are enabled with vGPU and the application processes are consuming the enabled vGPU resources.

• Log into your node and check the vGPU processes. This example shows command to monitor the vGPU devices of Nvidia.

nvidia-smi

Note: The nvidia-smi command displays the information on the GPU devices and the running processes for that GPU.

• Log into F5 Distributed Cloud Console and navigate to the Sites > App Site List page. Click on a site that is part of the fleet you created. This opens the site dashboard. Click Nodes tab and click on a node to open its dashboard. Click on Metrics to monitor the GPU usage, GPU temperature, and GPU throughput metrics.