Hardware & Network
Server: Dell T3640, i7-10700, 64G , 1T NVME, nvidia Tesla P4-8G
Network: DHCP Service is enable on network.
Install Proxmox 7.4
- Download Proxmox ISO image
https://www.proxmox.com/en/downloads
- Boot from USB or CD/DVD
Use rufus create a bootable usb key.
- Configure via GUI
https://<Proxmox_ipaddr>:8006
Install nvidia vGPU driver on Proxmox
- Install PVE-headers & dkms
CODENAME=`cat /etc/os-release |grep PRETTY_NAME |cut -f 2 -d "(" |cut -f 1 -d ")"`
echo "deb http://download.proxmox.com/debian/pve $CODENAME pve-no-subscription" | tee -a /etc/apt/sources.list
rm /etc/apt/sources.list.d/pve-enterprise.list
apt update && apt dist-upgrade -y
Install pve-headers & dkms
apt install pve-headers dkms mdevctl -y
- Enable IOMMU
# /etc/default/grub , GRUB_CMDLINE_LINUX_DEFAULT,add intel_iommu=on iommu=pt
sed -i 's/GRUB_CMDLINE_LINUX_DEFAULT="quiet"/GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"/g' /etc/default/grub
update-grub
# Loading vfio vfio_iommu_type1 vfio_pci vfio_virqfd four Modules
echo vfio >> /etc/modules
echo vfio_iommu_type1 >> /etc/modules
echo vfio_pci >> /etc/modules
echo vfio_virqfd >> /etc/modules
echo "options vfio_iommu_type1 allow_unsafe_interrupts=1" > /etc/modprobe.d/iommu_unsafe_interrupts.conf
echo "options kvm ignore_msrs=1 report_ignored_msrs=0" > /etc/modprobe.d/kvm.conf
echo "blacklist nouveau" >>/etc/modprobe.d/disable-nouveau.conf
echo "options nouveau modeset=0" >>/etc/modprobe.d/disable-nouveau.conf
update-initramfs -u
reboot
Checking the IOMMU is enabled
root@pveserver:~# dmesg | grep -e DMAR -e IOMMU
[ 0.000000] ACPI: DMAR 0x0000000079A48648 0000A8 (v01 INTEL EDK2 00000002 01000013)
[ 0.000000] DMAR: IOMMU enabled
[ 0.004000] DMAR: Host address width 39
[ 0.004000] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[ 0.004000] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[ 0.004000] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[ 0.004000] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
- Install nvidia vGPU Drivers
#download nvidia drivers
wget https://dl.deskpool.com/NVIDIA-Linux-x86_64-470.161.02-vgpu-kvm.run
chmod +x NVIDIA-Linux-x86_64-470.161.02-vgpu-kvm.run
#install nvidia vgpu driver for kvm hypervisor
./NVIDIA-Linux-x86_64-470.161.02-vgpu-kvm.run -dkms -s
Reboot PVE Server
systemctl daemon-reload
reboot
Check P4 is working
root@pveserver:~# nvidia-smi vgpu
Thu Feb 29 13:39:00 2024
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 470.161.02 Driver Version: 470.161.02 |
|---------------------------------+------------------------------+------------+
| GPU Name | Bus-Id | GPU-Util |
| vGPU ID Name | VM ID VM Name | vGPU-Util |
|=================================+==============================+============|
| 0 Tesla P4 | 00000000:01:00.0 | 0% |
+---------------------------------+------------------------------+------------+
Install Deskpool on Proxmox
Please refer to Deskpool for Proxmox quick start
In the step of Download Image from Remote Repository, select win21h1gpu.
In the step of Pool configure Pool, Select the GPU and vGPU type.