22

[GUIDE] GPU Passthrough for Laptop with Fedora

 1 year ago
source link: https://gist.github.com/firelightning13/e530aec3e3a4e15885a10f6c4b7ae021
Go to the source link to view the article. You can view the picture content, updated content and better typesetting reading experience. If the link is broken, please click the button below to view the snapshot at that time.

[GUIDE] GPU Passthrough for Laptop with Fedora · GitHub

[GUIDE] GPU Passthrough for Laptop with Fedora

Abstract

This is a full guide for people who wanted to setup Windows 10/11 VM with QEMU/KVM hypervisor for laptops that configured with hybrid graphics card like Intel/AMD + NVIDIA. This process will take about 1 to 2 hours, depending on your system's performance.

Before we proceed:

  • This guide is exclusively for Fedora because this distro is quite different to set up than other distro such as Arch. I would say Arch is easier to setup than Fedora, but sometimes your prefer Fedora than Arch in terms of its usage & features.
  • This tutorial mostly focus on laptops with NVIDIA Optimus MUXed configuration when dGPU (dedicated GPU) can connect directly to HDMI/DP output. If in doubt, search "VR-Ready laptops", as those are definitely MUXed.
  • For MUXless configuration which the dGPU share display alongside with iGPU (integrated GPU), you may need to extract your system's vBIOS for your card to avoid getting no output from your screen. Refer here first before proceeding.
  • How to check in terminal: lspci -k | grep -A 2 -E "(VGA|3D)"; If it's display 2 VGA = likely MUXed, else 1 VGA and 1 3D = highly likely MUXless.
  • This tutorial also doesn't cover AMD iGPU + AMD dGPU configuration (or AMD Advantage laptops) as I personally don't have them. The process might be the same, but some steps needs slight alteration. Do this at your own risk.
Tested on Lenovo Ideapad Gaming 3 15ARH05 (AMD Ryzen 5 4600H + NVIDIA Geforce 1650 Ti) with Fedora 36 (previously on ArchLinux), using NVIDIA driver

**UPDATE 10/30/2022: Supergfxctl has updated to version 5.0.0. Rebootless mode is enabled by default, and Xorg is no longer supported (but still can be used however).

Index

Prerequisites

  • CPU requirements: Intel Core or AMD Ryzen, at least 6-core CPU with hyper-threading/SMT (12 threads in total) and has support for virtualization extension (VT-d/AMD-Vi)
  • GPU requirements: Intel/AMD iGPU+ NVIDIA GPU w/ MUXed (or MUXless) configuration
  • At least have 12GB of RAM. Even better if you have more.
  • Have more disk space depending on your use case (eg: gaming, creative work, etc.). For me, minimum space per virtual disk should be 256GB or 512GB. So your system should have at least 1TB of total space for bigger headroom. Even better if you are using SSD.
  • You have installed Fedora Workstation . Should work with any desktop environment, and with display server like Xorg/Wayland.
  • You have installed NVIDIA driver. If you haven't installed it yet, get it here, unless you don't want your host to use your dGPU at all. In this case, this doesn't matter.
  • Things to display your Windows 10/11 VM. Either:
    • Have a separate monitor to display Windows 10/11 VM (highly recommended)
    • Use Looking Glass (a good alternative)
    • Use any remote display client (NVIDIA Stream, Moonlight, etc., not recommended)
  • Basic know how to use terminal. This guide requires you to type certain terminal commands to achieve something that is not possible with only just GUI. (tips: use Ctrl+Shift+C to copy and Ctrl+Shift+V to paste from inside the terminal, and use Tab to autocomplete commands or filenames)

Pre-setup VM

You need to prepare some tools before setting up your virtual machine. If you already set everything up, go here.

Enable Intel VT-d/AMD Vi virtualization extension

Most laptops nowadays already have this extension build into their chipset. To enable it, you need to go your UEFI/BIOS settings and search for the option that relates to virtualization which they may have different naming scheme in different settings tab, depending on your hardware. (example from the picture below)

Usually you can hit function keys (F10, F2, Delete, etc.) when rebooting. Consult with your laptop manufacturer or read their user guides on how to boot to BIOS/UEFI.

68747470733a2f2f312e62702e626c6f6773706f742e636f6d2f2d34703465686172736d646f2f5855624f705a54686266492f4141414141414142694f672f68664b62573236464c6c773158774c644b34632d39634461457836454544797067434c63424741732f73313630302f32303137303430365f3133353330385f4844522e6a7067

Install virtualization tools

Install virtualization tools package by typing:

sudo dnf update # update your system first
sudo dnf install @virtualization

The virtual machine tools such as virt-manager, QEMU, and libvirt will be installed to your system, which is required to run Windows 10/11 VM with GPU passthrough.

Then, start libvirtd service:

sudo systemctl enable libvirtd
sudo systemctl start libvirtd

Enable IOMMU grouping

IOMMU is part of Intel/AMD virtualization technology (VT-d/AMD-Vi) that can be used to passthrough PCI devices, in this case your dGPU. IOMMU groups PCI devices to ensure passthrough work properly. NVIDIA has two PCI devices: the VGA controller (or 3D controller for MUXless), and audio device. You can check by typing lspci | grep NVIDIA or lspci -s 01:00..

If you have AMD CPU, you don't need to do anything because it is already enabled by the kernel itself.

However, if you have Intel CPU, it isn't enable by default. Edit your kernel parameter through GRUB2 configuration, by typing:

sudo nano /etc/default/grub # you can use vim/emacs if you want

Then, in the line GRUB_CMDLINE_LINUX, insert/append:

intel_iommu=on # insert before the end of the quotation mark (")

Then, apply your GRUB2 config:

# UEFI, for most modern laptops shipped with Windows 8+
sudo grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg 
# BIOS, if you have older laptop
sudo grub2-mkconfig -o /boot/grub2/grub.cfg

To confirm that you have enable IOMMU, copy and paste this to your terminal:

#!/bin/bash
shopt -s nullglob
for g in $(find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V); do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;

If you see something like this:

IOMMU Group 9:
	01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU117M [GeForce GTX 1650 Ti Mobile] [10de:1f95] (rev a1)
	01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10fa] (rev a1)

These means that your IOMMU grouping is working. If you don't see anything like these, you may need to redo those steps above properly.

If, for some reason the IOMMU isn't properly group, for example some random PCI device grouped with NVIDIA out of nowhere, you should return the device immediately and blame the laptop manufacturer for shipping a broken motherboard. I doubt this will happen with your laptop, though there is ACS patch that can be used but that mostly applies only for PCs.

Setting up Windows 10/11 VM

In this section, you need to set up and configure Windows 10/11 VM. Then, you need to install it and go through the OOBE (Windows Out of Box Experience) setup before doing the passthrough process. You can setup the VM entirely through command line by using QEMU, but I'll be using virt-manager for this tutorial.

*If you know about these steps already, you can skip ahead to the next section

Before launching virt-manager, you need to download Windows ISO file first. You can get here: Windows 10 ISO or Windows 11 ISO. Then, create a new folder in your home directory to store the file. Name the folder something like "VM". This will be useful to store the installation file and also creating virtual hard disk later.

192543229-cc6bd3c0-d5dc-4ecd-bf63-3af12d7f3f82.png

(Optional) You also need to download virtio-win drivers from Fedora if you want to increase your VM image disk and virtual network card performance. You can visit here and select Latest virtio-win ISO and it will download the file. Don't forget to put it into your VM folder.

Initial setup

  • Launch virt-manager in the application menu. You may need to enter your user password if prompted. Then, create a new VM.
  • Then, choose Local install media.

192543714-fb9a7b7f-dd7f-471b-97ca-067a432b862f.png
  • Select your Windows installation file. Click Browse.

192544096-c14de2f4-67fb-430f-8a5a-eca7689effc4.png
  • Click the + button at the bottom left, then choose the pool's name (eg: myVM), and then locate the folder you just made in the home directory.

192544431-1785ca04-8103-48c9-8f95-56829f30d3fc.png
  • Then, choose the pool that you've created, select the Windows ISO file and click Choose Volume. Virt-manager will detect the OS of the iso file. (If you're using Windows 11, it might says "Microsoft Windows 10". This is normal as Windows 10 and 11 are not that different)

192545491-e67db8fd-3b56-4665-8252-738b6b1163e1.png
  • The dialogue box might be appeared, something to do with search permission. Just check the box and click Yes.

192546105-b211dd4b-dc91-431d-8510-46dd03bcc227.png

  • Configure the amount of memory and CPU core depending on how much available in your system. You can put at least 4GB of memory which is minimum system requirement for Windows 10/11, with 2 or 4 CPU core. Don't worry, you can configure these again later.

192547708-8ab6b00c-4a1a-4567-8649-1c5dea2b1e3d.png

Setting up virtual disk

Create a virtual hard disk. By default, it will create the hard disk in the default directory /var/lib/libvirt/images, but you can create it inside your own directory you just made.

  • Select the "Select or create a custom image" option, then click Manage

192546529-c3729f9d-1a7f-468a-9418-d3228d4f1817.png
  • Select your pool, then press + at the top center besides "Volume"
  • Name your disk, and enter number of capacity that you want. Make sure the format qcow2 is selected. *You can use raw too, some people say it actually increase performance, but I personally haven't compared between these two formats

192548186-aee0a1a2-dd63-4c5b-b9e4-b00df18927e3.png
  • Select the disk that you've just created. Then click Choose Volume. Make sure the path is right in the main window.

192549419-9632d9fe-46aa-40d3-b6bd-5ab97d21eb69.png
  • Finally, name your virtual machine anything you like (no whitespaces). There are some configurations that you need to change later, so tick the checkbox Customize configuration before install. Then, click Finish.

192549735-97694cf3-526c-4ec8-9e86-3eacd73f710c.png

More VM configuration

  • In the Overview tab, you can fill out basic details of your VM. Make sure to select Q35 as your chipset, and change the firmware to UEFI x86_64: /../OVMF_CODE.fd, or OVMF_CODE.secboot.fd if you install Windows 11 to enable secure boot. Click Apply if you're done.

192550540-e1039cdc-061f-4efe-9835-8536cf0bb7ca.png
  • In the CPUs tab, you have to manually set the CPU. Tick the checkbox Manually set CPU topology, and edit your sockets, cores and threads. You shouldn't use all of the CPU cores and threads, and leave only one core for your host system. Click Apply if you're done.

192551047-648f1731-2276-4f6e-b19b-e38dd5308f97.png

If you don't how to specify those properties, you can type in your terminal: lscpu, and check under the VendorID. This is the result in my terminal:

Vendor ID:               AuthenticAMD
  Model name:            AMD Ryzen 5 4600H with Radeon Graphics
    CPU family:          23
    Model:               96
    Thread(s) per core:  2
    Core(s) per socket:  6
    Socket(s):           1
  • In the Memory tab, make sure the amount of memory is enough for your VM and also watch your total system memory. Decrease or increase the amount if necessary. "Enable shared memory" is necessary if you plan to use Looking Glass, which you'll enable it later.

192551531-72fdd3e5-d04f-4b43-ae5b-9b0c439454b3.png
  • In the SATA Disk 1 tab, this is your virtual disk that you've created. Check if the path points to the right direction. Make sure "Disk bus" is SATA. Also check in the SATA CDROM 1 section, make sure it is read-only under "Advanced options".

192551973-7791433c-32cd-44b2-8e4e-cf4c6b4c3d6c.png

*If you want to increase read/write performance of the virtual disk, change the disk bus to virtio and Advanced options > Cache mode: none and Discard mode: unmap. (requires virtio-win drivers)

  • In the NIC tab, you can leave device model as default, e1000e, or you can change it to virtio to increase network performance (requires virtio-win drivers). Make sure the link state is active (you can disable this to trick Windows 11 to setup offline account).

192552542-70045a37-3062-4070-9372-073f89bead8b.png
  • In Video QXL tab, make sure QXL is selected.

  • (Optional) Add virtio-win ISO file as a secondary CDROM. Click Add Hardware in the bottom left corner, select Storage in the left list. Select virtio-win ISO file (same as selecting your Windows ISO), then select Device Type to CDROM device and finally enable read-only under advanced options. Click Finish to add the hardware.

192553716-9e8a059c-a815-4602-b839-5226889c632f.png
  • (For Windows 11 installation) To emulate TPM, click Add Hardware, then select TPM. The type should be Emulated, and under advanced option, select CRB as the model and choose version 2.0 as stated from Windows 11 requirement. Click Finish to add the hardware.

192554050-9962009e-8a7f-4480-9728-4782704d92de.png
  • Make sure everything is configured properly. If you satisfied with the configuration, click Begin Installation at the top left corner. At this point, the screen should be popped up, and hit any key when asked. If there is an error while starting the VM, you should see the log and try to troubleshoot for yourself.

193087489-7e942d6e-6783-49a2-8543-114e61657acc.png

Windows Installation

The windows installation is pretty much just like a normal installation in a bare metal hardware. You need to use the screen provided by virt-manager to interact with the installation process.

However, if the display is too small for you (for folks who uses GNOME fractional scaling or 4k display), you can use Remote Viewer in the Application menu. Enter spice://localhost:5900 then press Connect.

192554464-8b696aff-ca31-48aa-8cee-2f4487296d15.png

I chose Windows Pro edition to enable Hyper-V virtualization so I can spin up VM inside this VM setup, but you can choose Home edition if you like.

NOTE: I don't recommend enter any product key as you are likely to messed up your VM or you might find yourself constantly delete/create this VM and configure it to your liking. There is a way to copy your product key embedded to your laptop and paste it into this VM, but I won't cover here in this tutorial. You can check here if you are interested.

(Optional) If you configured your virtual hard disk bus to use virtio, you may find that the disk is missing after you choose Custom: Install Windows only (advanced). In this case you need to do the following:

  • Click Load driver at the bottom left. Then click Browse.

192554938-11a4fbd6-2e09-40f2-bcc5-3273b201c800.png
  • Under This PC > select the folder under CD Drive named virtio-win > viostor > w10/w11 > amd64. Click OK. (w11 if you are using Windows 11, w10 if you are using Windows 10)

192555947-a508239d-eabf-46cc-9eb0-940a7a9bee9d.png
  • The Red Hat VirtIO SCSI Controller should be selected. Click Next.

192556246-07443a56-09e2-4998-a449-76b40b4b956b.png
  • Now, the Drive 0 Unallocated Space should appear.

  • If you also select your NIC type to virtio, you should install driver from here too. Just like before, select Load driver > Browse > CD Drive named virtio-win > NetKVM > w10/w11 > amd64 and click OK. The Red Hat VirtIO Ethernet Adapter should show up.

193088309-a9193e8c-6fa4-48a5-98ee-735b46a5d26b.png
  • Select Drive 0 Unallocated Space and click Next to begin installation.

  • After the installation, it will reboot by itself (sometimes even reboot several times). Now you have to go through the OOBE process to initialize Windows operating system. After everything is done initializing, you can shutdown the VM to begin GPU passthrough process.

192557048-b918cd9e-78d5-4aa9-a306-c37275600b35.png

Using supergfxctl to bind vfio to GPU

Supergfxctl is a very useful tool to bind vfio drivers to GPU on demand, which also double as graphics mode switching tools. However, when switching modes, it requires you to restart your system which can be daunting, however it can be configured as rebootless. You can also switch mode during boot by edit kernel parameter via GRUB menu.

NOTE: This section is for version 4.0.5 which is outdated, new version is out now (v5.0.0) as of writing this note. The most notable update is rebootless mode is enabled by default. Xorg is now unsupported by the dev, however it can still works. I will update this section later.

This tool is incompatible with other graphics mode switching such as Envycontrol. If you have one, refer here to completely uninstall it.

Installing supergfxctl

Since Fedora doesn't have this tool in their repository, you have to build it from source by yourself.

  • Get development tools by copy & paste this line of code in your terminal:
sudo dnf install curl git @development_tools
  • This tool also requires Rust as dependency:
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs
source ~/.cargo/env
  • Make a folder called build in your home directory, or any name that you want. This is just a good practice to reserve a folder to any tools that you want to build from source. Then, clone supergfxctl's repository.
mkdir ~/build && cd ~/build
git clone https://gitlab.com/asus-linux/supergfxctl.git
  • Proceed to install supergfxctl to your system:
cd supergfxctl
make && sudo make install
  • Then enable the service:
sudo systemctl enable supergfxd.service --now

(Optional for GNOME user) Add GUI mode switch

If you are using GNOME as your desktop environment, you can add GNOME extension as GUI to use this tool. You can check them out here.

  • To install, get Extension Manager from Sofware app if you don't have one.

193086287-8b29293a-5448-4a61-b526-93987e2c12db.png
  • After installing Extension Manager, click Browse at the top center, then search "supergfx". It will automatically show up the first result, which is Super Graphics Control by krst. Click Install.

193087090-9d99aea7-2d49-4c69-ab24-ffe1353a88a1.png

The UI button "H" at your toolbar should appear at the top right. If you can't see it, make sure to install AppIndicator and KStatusNotifierItem Support as well.

193088624-75265a5c-c501-4bd2-8018-d9648819737f.png

Add vfio mode to supergfxctl

Now, you need to configure the tool to add vfio mode. To do that, you need to edit /etc/supergfxd.conf.

  • Edit the config as super user:
sudo nano /etc/supergfxd.conf # or vim if you like
  • Change vfio_enable value to true. If you want persistent vfio mode on reboot, you can also set vfio_save to true. The final config should look like this:
{
  "mode": "Hybrid",
  "vfio_enable": true,
  "vfio_save": true,
  "compute_save": false,
  "always_reboot": false,
  "no_logind": false,
  "logout_timeout_s": 180
}
  • Save the configuration. You may need to restart the service to make changes:
sudo systemctl restart supergfxd.service
  • You can check whether "vfio" mode is now changeable:
supergfxctl -s

It should returned [Integrated,Hybrid, Dedicated, Compute, Vfio].

Switch to vfio mode

  • Make sure you switch to integrated mode before set to vfio mode, as per tool's instruction (or switch via GUI):
supergfxctl -m integrated

*NOTE: If you are curious about the message requires nvidia-drm.modeset=0, click here to find out.

  • This switch mode from hybrid > integrated requires reboot. After fully reboot, make sure you don't have NVIDIA card running in your system by typing in your terminal:
nvidia-smi # if you have xorg-x11-drv-nvidia-cuda installed 

Should return this line (this error is normal if you don't have NVIDIA card running, which means it worked):

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.
  • Or, check your nvidia card via lspci | grep NVIDIA, should return nothing.

*NOTE: My laptop has a bug where restarting once doesn't work. If you have this bug as well, make sure to restart your system again for integrated mode to work

  • Now, switch to "vfio" mode, either via GUI or in terminal and reboot:
supergfxctl -m vfio
sudo reboot
  • To confirm that the vfio drivers has binded your GPU, type lspci -ks 01:00.. The result would be like this:
01:00.0 VGA compatible controller: NVIDIA Corporation TU117M [GeForce GTX 1650 Ti Mobile] (rev a1)
	Subsystem: Lenovo Device 3a44
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau, nvidia_drm, nvidia
01:00.1 Audio device: NVIDIA Corporation Device 10fa (rev a1)
	Subsystem: Lenovo Device 3a44
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

If the line under Kernel driver in use is vfio-pci, then it is success!

  • To go back from normal, you can switch to hybrid if you want to use your system without VM.

*To switch mode during boot instead of using command line or GUI, see here.

Post VM setup with GPU passthrough

Now you can add your NVIDIA GPU PCI devices into your VM, then install NVIDIA drivers in Windows 10/11. Before doing this though, you need to edit your VM's XML config first.

Add dGPU to the hardware list in VM

  • Make sure to check your libvirt version because some version requires different code snippet which I'll show you later. As of writing this, my version is 8.1.0 which is a little bit older than the latest version. To check your version, open your terminal and type virsh --version.

  • You also need to get your dGPU's sub-vendor/device ID by typing lspci -nks 01:00.0 , find it under your Subsystem:.

lspci -nks 01:00.0
# Results below
01:00.0 0300: 10de:1f95 (rev a1)
	Subsystem: 17aa:3a44 # your id here
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau, nvidia_drm, nvidia

For example, mine have 17aa:3a44. My vendor ID is 17aa, and my device ID is 3a44

  • In the main window, at the top left, select Edit > Preferences. In the General tab, check the box Enable XML editing, and click Close.

192558230-b4c9afe2-622c-4cdb-99ca-1a05572395dd.png
  • Select the your VM, click the bulb icon at the top left then at the bottom left corner, then click Add Hardware. In the left list, select PCI Host Device and select both PCI devices named NVIDIA Corporation. After that, click Finish.

192558907-69f6b632-d101-483f-975c-45f10be886da.png
  • To edit full XML, select Overview in the left list, and click XML tab at the top center. In the editing window, change the content of the domain tag at the top line to this line of code:
<domain xmlns:qemu="http://libvirt.org/schemas/domain/qemu/1.0" type="kvm">

192559299-d3c10a9e-2ec9-4487-93e3-7773c1583172.png
  • (For libvirt version lower than 8.2.0) Scroll down until at the bottom, and between the line </devices> and </domain>, copy and paste this block of codes and edit the value ID:
  <qemu:commandline>
    <qemu:arg value="-set"/>
    <qemu:arg value="device.hostdev0.x-pci-sub-vendor-id=0x17aa"/>
    <qemu:arg value="-set"/>
    <qemu:arg value="device.hostdev0.x-pci-sub-device-id=0x3a44"/>
  </qemu:commandline>
  <qemu:capabilities>
    <qemu:del capability="device.json"/>
  </qemu:capabilities>

Derived from my example above, the sub-vendor-id should be 0x17aa, and sub-device-id would be 0x3a44.

  • (For libvirt version 8.2.0 and above) Scroll down until at the bottom, and between the line </devices> and </domain>, copy and paste this block of codes and edit the value:
  <qemu:override>
    <qemu:device alias="hostdev0">
      <qemu:frontend>
        <qemu:property name="x-pci-sub-vendor-id" type="unsigned" value="6058"/>
        <qemu:property name="x-pci-sub-device-id" type="unsigned" value="14916"/>
      </qemu:frontend>
    </qemu:device>
  </qemu:override>

To get value from the two property, your subsystem ID should be converted from hexadecimal to decimal value. You can go to online converter for this. For example, my vendor id 17aa converts to 6058, and for my device ID 3a44 converts to 14916.

192559959-631f4734-13f1-4fd5-802e-1d667a53526b.png
  • Then, click Apply.

Test your VM and install NVIDIA driver

  • Connect to your external monitor and start your VM. If you don't have external monitor, you can temporarily use the screen provided by virt-manager or Remote Viewer (after installing NVIDIA, you should install Looking Glass).

  • Check if you have another Display adapter in Device Manager with error sign on it. That means you have successfully passthrough your GPU. (tips: Right Click on Windows button > Device Manager)

  • After boot and login to your Windows 10/11, go to official NVIDIA website and download the NVIDIA driver for your appropriate dGPU. And then install it just like you would normally do on your Windows laptop.

  • After installing NVIDIA, go to Device Manager and under Display adapter, check if you have error code 43. If you have one, you can refer here on how to fix it.

VM Enhancements

If you made it here, congratz! You just sucessfully managed to do GPU passthrough in your laptop. You can check out guide below on how to enhance your VM by giving you more performance and some tools that can help you to get a better experience. These steps are entirely optional and you can just start using your VM as it is.

Use Looking-Glass

Looking-Glass is a great tool to have if you don't have external monitor. Unlike using live capture/remote display client such as NVIDIA Stream, Moonlight or FreeRDP; Looking Glass have insanely low latency and no compression/artifacts because the video memory is shared from Linux to VM directly. More info here.

Even if you have an external monitor, you can take advantage of Looking-Glass as a video capture card that can be used to recorded your Windows VM screen via OBS on Linux by using OBS plugin.

Before doing this, you need HDMI dummy plug for it to work. Or else it will not be at a full resolution that you've set. There is an alternative solution later on, so just follow along this section first.

Build from source

To install Looking-Glass, you need to build it from source as Fedora repository unfortunately doesn't have this package.

  • Firstly, download the dependencies for Looking Glass. In your terminal, copy & paste this line of code:
dnf install cmake gcc gcc-c++ libglvnd-devel fontconfig-devel spice-protocol make nettle-devel \
            pkgconf-pkg-config binutils-devel libXi-devel libXinerama-devel libXcursor-devel \
            libXpresent-devel libxkbcommon-x11-devel wayland-devel wayland-protocols-devel \
            libXScrnSaver-devel libXrandr-devel dejavu-sans-mono-fonts dkms kernel-devel kernel-headers

For up to date dependency packages, check here.

  • Download the package from the website here. Under Official/Stable version, choose Source.

  • Make a directory called build in your home directory if you don't have it, or any name that you want. This is just a good practice to reserve a folder to any tools that you want to build from source.

mkdir ~/build
  • Extract the downloaded package to your build folder. (tips: install GNOME File Roller to easily manage package files)

  • In your terminal, go to client/build directory from your extracted files.

# depending on your Looking Glass version
cd ~/build/looking-glass-B5.0.1/client/build
  • Then, run cmake. This will take some times to finish.
# run this if you are non-GNOME wayland user
cmake ../

If you're GNOME Wayland user, add an additional parameter as example below (Note: you need to install libdecor to use this).

# run this if you are GNOME wayland user
sudo dnf install libdecor # do this first
cmake -DENABLE_LIBDECOR=ON ../
  • Then finally run make to build the Looking Glass client. This will take some time to finish.
make
  • The looking-glass-client should appeared if you look inside client/build directory. You can type ls to make sure it's there. Copy the file to /usr/local/bin to easily access the application with terminal:
sudo cp looking-glass-client /usr/local/bin
  • To confirm the client is working, type looking-glass-client -h. It should show up useful commands that you can do. List of commands can be found here as well if you want to know more about them.

Edit VM configuration

  • If you have virt-manager still opened, go to your Windows VM settings, and in the Memory tab, check the box Enabled shared memory.

  • In your XML config, scroll down until at the bottom. Between the line <memballoon/> and </devices>, copy paste this block of codes:

<shmem name='looking-glass'>
  <model type='ivshmem-plain'/>
  <size unit='M'>32</size>
</shmem>

Depending on the size, 32MB is plenty enough to display 1080p. If you want to have 2K or 4K resolution, you might need to increase it. See this guide to determine your memory size.

  • Speaking of <memballoon/> line, you need to change the model type to none for performance improvement:
<memballoon model="none"/>
  • You need to give permission for the Looking Glass shared memory file found in /dev/shm/looking-glass. Create a new file in your terminal:
sudo nano /etc/tmpfiles.d/10-looking-glass.conf # use vim if you like

And copy and paste this block of codes (change user to your username):

#Type Path               Mode UID  GID Age Argument

f /dev/shm/looking-glass 0660 user kvm -
  • Try start your VM first before proceeding. If you have permission problem like "Permission denied", see here to know how to fix it.

Installing Looking Glass in Windows VM

  • When booting, you need to go to BIOS settings before installing a driver by pressing F2 repeatedly.

  • Go to Device Manager > Secure Boot Configuration > uncheck Attempt Secure Boot. Press F10 to save configuration. Go back to the main menu by ESC, then press Continue.

192560806-08bd075d-c0df-4885-b0fa-f2d8d982fc5a.png
  • Now, go to the Looking Glass website and download the Windows Host Binary from here. Make sure the version is the same as your client.

  • You should also download the IVSHMEM driver. This is important for Looking Glass to work. Follow the guide here for the download link. (use version 0.1.161 as stated from the website)

  • Right click Windows home button at the bottom and select Device Manager. Under System Devices, scroll down and look for PCI Standard RAM Controller. If you found it, right click and select Update driver...

  • Select Browse my computers for drivers > Browse. In the file browser, under This PC > Downloads > -Your extracted driver folder- > Win10 > amd64. Then, click Next and wait for it to install. After install, it is done.

  • Extract this zip file that you have downloaded and start installing Looking Glass.

  • After this step, you might need a dummy HDMI plug for Looking Glass to work. There is an alternative solution if you dont have one, which you can refer here. Come back here if you're done.

  • Shut down your VM. Go to the VM settings, under Video tab, change the model type to None

  • Everything should be set. Start your VM now, and open your terminal and type:

looking-glass-client # you can use more command option, but this is just a test
  • Plug in your dummy HDMI (not relevant if you have installed virtual display driver), and it should appear. You can do fullscreen by pressing ScrollLk + F. You can see all keybindings here.
  • ScrollLk is a "master" key for those bindings (also capture your mouse to VM directly). To change it, refer number keys here and type looking-glass-client -m <number>. e.g: Right Ctrl is 97, so you should type looking-glass-client -m 97.

That should be it for setting up Looking Glass. You can check out other enhancements here if you want to futher improve your experience.

Use Remote Desktop client (not recommended)

I do not recommend any network based desktop client as the quality is a bit low and introduce artifacts compares to Looking Glass, more reason that I won't cover this step. I don't know any program that is good enough to do this, and you have to search on your own.

Audio passthrough

By default, the virt-manager/QEMU uses HD audio (ICH9) sound driver which can be piped through SPICE server (the screen you use in virt-manager or remote viewer), but the audio quality is terrible and produces high latency. If your monitor supports HDMI audio, you won't need additional steps to do this and just remove that Sound driver.

However if you are using Looking Glass, it does not support audio passthrough (except if you are using version B6-rc1, more info here). There are a few options that you can do to achieve this besides HDMI audio driver:

  • Use USB sound card (best option, very low latency/overhead)
  • Use "Scream" audio driver (a good alternative, easy to setup for Windows guest)
  • Passthrough Pipewire directly (better than pulseaudio, easier to setup for other OS guest)
  • Passthrough Pulseaudio directly (not a good solution, CPU heavy and poor bitrate)

USB Sound Card

This is the best option if you have a spare sound card. Note that this can be only used for Windows VM and cannot be shared through your host. Most laptops have their own on-board sound card (your built-in headphone jack) and use that for your host.

  • Plug in your USB Sound Card, from your VM settings, click Add Hardware > USB Host Device, and choose your sound card. Once added, remove the sound driver (HDA ICH9).

SCREAM audio driver

This is a virtual audio driver in Windows VM which sends through your VM virtual network to passthrough the sound it produce. The latency is pretty low and produce cleanest sound you can get for virtual audio, which can be piped through your host audio system. The benefit is that the audio can be shared with your host sound as well if you want with no sound quality loss unlike using USB Sound card.

  • Install dependency packages for scream:
sudo dnf install pulseaudio-libs-devel
  • Make a directory called build in your home directory if you don't have it, or any name that you want. This is just a good practice to reserve a folder to any tools that you want to build from source.
mkdir ~/build
  • Download Scream git repo:
cd ~/build # go here first
git clone https://github.com/duncanthrax/scream.git
  • Go to receiver's build directory to start cmake them. This won't take long to build.
cd scream/Receivers/unix/
mkdir build && cd build
cmake ..
make
  • Copy scream to /usr/local/bin:
sudo cp scream /usr/local/bin
scream -h # check if it's installed in your system
  • Start your Windows VM if you haven't already. If you didn't install Looking Glass, you might need to disable secure boot first in your VM. (spam F2 while boot > Device Manager > Secure Boot Configuration > uncheck Attempt Secure Boot > F10 to save)

  • Download the zip file audio driver in the scream's release page.

  • Extract the zip file, run Install_x86_64.bat. This will install scream audio driver. If an install prompt showed up, click Install.

  • The scream driver should be installed. You can check it via sound settings in the toolbar at bottom right.

  • In your linux host, open your terminal and just type:

scream -i virbr0

virbr0 is the default name for virtual network for your VM. If in doubt, type ip link show to know your network interface name.

  • If something shows up, it means that you have sucessfully passthrough audio from the VM. If you don't hear anything, this might be because of firewall blocking the connection. To remedy this, add a port to the firewall rule:
firewall-cmd --zone=libvirt --add-port=4010/udp
sudo firewall-cmd --runtime-to-permanent # make sure the rule is permanent

Start testing your audio by going to Youtube and play a video, or play a game to see if it works for you.

PipeWire and JACK

Using PipeWire & JACK as a passthrough is also good if you don't want so much hassle installing scream. This is also useful if you plan to spin other VM like linux distro or Mac OS using this method. Pipewire and JACK is installed natively in Fedora, so you don't need to do anything besides setting up few things in your VM.

  • Before anything, make sure QEMU is running as user. Edit qemu.conf:
sudo nano /etc/libvirt/qemu.conf
  • Find this line, uncomment and change it to your username then save the file. (tips for nano user: Press Ctrl + W and search "user =")
user = "bob"
  • Go to VM settings, open your XML window. Scroll down until you find <audio id="1" type="spice"/> line. Replace it with these codes (change win11 to win10 or anything, doesn't matter):
  <audio id="1" type="jack">
    <input clientName="vm-win11" connectPorts="Family 17h/19h HD Audio Controller Analog Stereo"/>
    <output clientName="vm-win11" connectPorts="Family 17h/19h HD Audio Controller Analog Stereo"/>
  </audio>
  • The Family 17h/19h HD Audio Controller Analog Stereo is my port. To know what port name in your system, install either Carla or Audacity. For me, Audacity is the easiest to identify my ports. At the leftmost toolbar, switch from ALSA to JACK Audio Connection Kit. Observe the the name of the input and output:

192561285-6507598a-5787-4092-ac2d-a7743ead7b4a.png
  • Then, under </devices>, add these:
  <qemu:commandline>
    <qemu:env name="PIPEWIRE_RUNTIME_DIR" value="/run/user/1000"/>
    <qemu:env name="PIPEWIRE_LATENCY" value="512/48000"/>
  </qemu:commandline>
  • The /run/user/1000 (1000) is your user ID. If you are not sure what is your user ID, you can check it by typing id in your terminal.

  • The latency is set at 512/48000 (512 is sample size, 48000 is sample rate) which is enough for most laptop. You probably need to change it depending on your audio card. If you have cracking audio, increase the sample size or change the sample rate according to your pipewire system.

  • If you don't hear any sound, SELinux might block it from connecting through pipewire. Refer here for the fix.

Use Pulseaudio (not recomended)

I will not cover how to passthrough audio via Pulseaudio because the sound quality is quite poor with constant audio cracking. If you are still insist on doing this, I recommend this guide from ArchWiki. (the explanation is quite technical, so proceed at your own risk)

CPU Pinning

CPU pinning improves performance massively because it gives number of cores in your host CPU to run exclusively on VM. You may not passed every cores as you need one to reserve for your host, or else everything will not be responding and smoothly.

Know your CPU topology

Intel and AMD CPUs have different topology if they support hyper-threading for Intel, or SMT for AMD (1 core each gives 2 threads). Below are examples of 6 cores with 12 threads in total.

  • In your terminal, type lscpu -e. The output should be like this (my AMD Ryzen 5 4600H):
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE    MAXMHZ    MINMHZ       MHZ
  0    0      0    0 0:0:0:0          yes 3000.0000 1400.0000 1400.0000
  1    0      0    0 0:0:0:0          yes 3000.0000 1400.0000 1397.3250
  2    0      0    1 1:1:1:0          yes 3000.0000 1400.0000 1396.9139
  3    0      0    1 1:1:1:0          yes 3000.0000 1400.0000 1400.0000
  4    0      0    2 2:2:2:0          yes 3000.0000 1400.0000 1397.3600
  5    0      0    2 2:2:2:0          yes 3000.0000 1400.0000 1396.5480
  6    0      0    3 4:4:4:1          yes 3000.0000 1400.0000 1397.3590
  7    0      0    3 4:4:4:1          yes 3000.0000 1400.0000 1397.2980
  8    0      0    4 5:5:5:1          yes 3000.0000 1400.0000 1394.4351
  9    0      0    4 5:5:5:1          yes 3000.0000 1400.0000 3000.0000
 10    0      0    5 6:6:6:1          yes 3000.0000 1400.0000 1397.4510
 11    0      0    5 6:6:6:1          yes 3000.0000 1400.0000 1397.0450
  • If you're using Intel, it might be like this (Intel Core i7-8700k):
CPU NODE SOCKET CORE L1d:L1i:L2:L3 ONLINE MAXMHZ    MINMHZ
0   0    0      0    0:0:0:0       yes    4600.0000 800.0000
1   0    0      1    1:1:1:0       yes    4600.0000 800.0000
2   0    0      2    2:2:2:0       yes    4600.0000 800.0000
3   0    0      3    3:3:3:0       yes    4600.0000 800.0000
4   0    0      4    4:4:4:0       yes    4600.0000 800.0000
5   0    0      5    5:5:5:0       yes    4600.0000 800.0000
6   0    0      0    0:0:0:0       yes    4600.0000 800.0000
7   0    0      1    1:1:1:0       yes    4600.0000 800.0000
8   0    0      2    2:2:2:0       yes    4600.0000 800.0000
9   0    0      3    3:3:3:0       yes    4600.0000 800.0000
10  0    0      4    4:4:4:0       yes    4600.0000 800.0000
11  0    0      5    5:5:5:0       yes    4600.0000 800.0000

*Note: Intel 12th Gen processors and up have different topography than the generations before it, with the presence of p-cores and e-cores. Someone made a post here sucessfully CPU pinned the processor.

  • Note that the link between "ID" of CPU and CORE. If I use the example above, AMD/Intel links their core with threads would be like this:
AMD Intel (n)th Core
0,1 0,6 1
2,3 1,7 2
4,5 2,8 3
6,7 3,9 4
8,9 4,10 5
10,11 5,11 6
  • If I want to isolate them, I would use 2nd-6th cores for VM, and 1st core reserves for host. Remember these setups, as you need to configure them later on:

    • To isolate AMD would be: 2-11, leave 0-1
    • To isolate Intel would be: 1-5, 7-11, leave 0,6
    • To config AMD in XML: 2,3,4,5,6,7,8,9,10,11
    • To config Intel in XML: 1,7,2,8,3,9,4,10,5,11
  • If you want a visualization of how your CPU topology looks like, run lstopo:

sudo dnf install hwloc-gui # install this first
lstopo

Mine looks like this:

192474386-b50d91f0-cafa-4a67-91e7-9fb3211eb82a.png

Edit your VM configuration

  • If you have VM settings windows open, under CPUs tab, make sure that you define your CPU topology according to how many cores you wanted to pass through.
  • Go to XML editing window, find the <vcpu/> line. Replace it with one of snippets below:
  • For AMD Ryzen 5 4600H, passing the last 5 cores to VM:
  <vcpu placement='static'>10</vcpu>
  <iothreads>1</iothreads>
  <cputune>
    <vcpupin vcpu='0' cpuset='2'/>
    <vcpupin vcpu='1' cpuset='3'/>
    <vcpupin vcpu='2' cpuset='4'/>
    <vcpupin vcpu='3' cpuset='5'/>
    <vcpupin vcpu='4' cpuset='6'/>
    <vcpupin vcpu='5' cpuset='7'/>
    <vcpupin vcpu='6' cpuset='8'/>
    <vcpupin vcpu='7' cpuset='9'/>
    <vcpupin vcpu='8' cpuset='10'/>
    <vcpupin vcpu='9' cpuset='11'/>
    <emulatorpin cpuset='0-1'/>
    <iothreadpin iothread='1' cpuset='0-1'/>
  </cputune>
  • For Intel Core i7-8700k, passing the last 5 cores to VM:
  <vcpu placement='static'>10</vcpu>
  <iothreads>1</iothreads>
  <cputune>
    <vcpupin vcpu='0' cpuset='1'/>
    <vcpupin vcpu='1' cpuset='7'/>
    <vcpupin vcpu='2' cpuset='2'/>
    <vcpupin vcpu='3' cpuset='8'/>
    <vcpupin vcpu='4' cpuset='3'/>
    <vcpupin vcpu='5' cpuset='9'/>
    <vcpupin vcpu='6' cpuset='4'/>
    <vcpupin vcpu='7' cpuset='10'/>
    <vcpupin vcpu='8' cpuset='5'/>
    <vcpupin vcpu='9' cpuset='11'/>
    <emulatorpin cpuset='0,6'/>
    <iothreadpin iothread='1' cpuset='0,6'/>
  </cputune>
  • Configure only cpuset depending on how much core you wanted to pass through this VM. vcpu number increases incrementally if you pass more cores. Arrangement is important!
  • <iothreads/> and <iothreadpin/> is useful when your storage is using virtio driver. Do not use this when you don't configured it!
  • Now try starting your VM. If it doesn't complain anything, that means it is working! You can inspect your CPU process by using htop if you have it installed, or use built-in system monitor if you're using GNOME/KDE.

Isolate CPU cores

Isolating CPU cores is another major performance improvement to go along with CPU pinning. This is done to make sure that the cores that you want to use for VM will not be used in your host so there are no hiccups in your system.

This can be done either by configure your kernel parameter or via basic libvirt hook, which is an automation tool that can be used when starting/stopping your VM. I recommend choosing latter if you are planning to use your host extensively. If you want total isolation, you can refer here instead.

Create libvirt hook script

  • Open your terminal, and create a libvirt hooks directory:
sudo mkdir -p /etc/libvirt/hooks
  • Create a file called qemu:
sudo nano /etc/libvirt/hooks/qemu # or vim if you like
  • Copy and paste this lines of codes to the file (for AMD use 0-1, Intel use 0,6 depending on how much core you have pass through):
#!/bin/sh

command=$2

# When VM starts, the core from 0 to 1 will be used for host (0,6 if Intel)
if [ "$command" = "started" ]; then
    systemctl set-property --runtime -- system.slice AllowedCPUs=0-1
    systemctl set-property --runtime -- user.slice AllowedCPUs=0-1
    systemctl set-property --runtime -- init.scope AllowedCPUs=0-1
# If VM stops, the host will reclaim all cores
elif [ "$command" = "release" ]; then
    systemctl set-property --runtime -- system.slice AllowedCPUs=0-11
    systemctl set-property --runtime -- user.slice AllowedCPUs=0-11
    systemctl set-property --runtime -- init.scope AllowedCPUs=0-11
fi
  • Make it executable and restart libvirtd service to apply:
sudo chmod +x /etc/libvirt/hooks/qemu
sudo systemctl restart libvirtd
  • Try starting your VM and see your htop or system monitor. If it works, there will be some cores uses 0% utilization at Windows startup. That means it's working.

Miscellaneous optimization

Some optimizations that may or may not improve your VM performance.

Switch to performance mode for your CPU

Your CPU governor will use their settings by default, which can lower the performance if you wanted to run your VM.

I recommend using auto-cpufreq by @AdnanHodzic to change your CPU mode on the fly. Refer to the instruction on how to install them and create a config for the tool. I recommend set it to performance mode when charged and use it while spinning up your VM, and powersave mode if unplugged to save more battery for your laptop.

Enable SMT for AMD CPUs in VM

This is for AMD CPUs only if you want hyper threading performance uplift. Inside your XML, find the tag <cpu>. Add a TOPOEXT feature like below:

<cpu mode='host-passthrough' check='none'>
  <topology sockets='1' cores='5' threads='2'/>
  <feature policy='require' name='topoext'/>
</cpu>

Total CPU isolation

Sometimes dynamic isolation via libvirt hook does give a bit lower performance than using static isolation via kernel parameter. I recommend use your system GRUB menu (when you boot up your laptop) to add them instead of permanently isolate them which host can only use one core all the time.

To edit your kernel parameter, reboot your laptop and press Right-Shift (or Esc if it didn't work) repeatedly to force GRUB menu to show up. Select latest linux kernel in the option, then press E.

Under the line linux /boot/..., append this line of code (the number below refer to cores that you wanted to use for virtualization, not host):

... isolcpus=2-11 nohz_full=2-11

This will isolate your cores completely from the start, so that your VM can claim those cores for maximum performance. This changes is only temporary as your next boot will reset those changes.

If you want to make it permanent, refer the GRUB instruction here.

Static/Dynamic hugepage?

I've seen a lot of guides telling you to set up static/dynamic hugepage, however I also seen some people claim that it doesn't make much difference in terms of performance, even in my testing. So I don't use them anymore. If you think it will increase your VM performance or wanted to try it out first, check this guide here by ArchWiki.

FAQs & Troubleshooting

Blank screen output when I passthrough my GPU. How?

  • If there is no output, you might need to extract your vBIOS from your laptop (likely will happen if your laptop is configured MUXless)
  • I don't want to exclude people with this problem, but I don't know exactly how to do it because my laptop's configuration is MUXed. Please comment down below if you managed to get it working! Provide a step-by-step guide if you can.
  • Instead, check out this guide if you want to extract your vBIOS. You need to find your system's BIOS update from your manufacturer's website. If you dual boot Windows 10/11, you can follow this guide here (look at on board GPU section).

Go back to Before we proceed.

Can I use rebootless mode for supergfxctl?

If you're using terminal to switch mode, you might noticed this message:

Rebootless mode requires nvidia-drm.modeset=0 to be set on the kernel cmdline
  • This error was shown because nvidia-drm.modeset=1 is set by default in your kernel parameter.
  • This is entirely optional if you want to change nvidia-drm.modeset to 0 in your kernel parameter. That way, you won't need to reboot your system, either by logout then login, or instantly.
  • However, if you're using GNOME or KDE with Wayland, you are forced to use Xorg instead of defaulting to Wayland if you're passing that option.
  • If you don't care about this, you can refer to this section on how to edit kernel parameter via GRUB. (find nvidia-drm.modeset, change from 1 to 0)
  • If you still wanted to use rebootless mode but still using Wayland, see the next section. (I recommend doing GPU passthrough completely before doing this)

Go back to Switch to vfio mode

How to use rebootless mode while using Wayland?

VERY IMPORTANT NOTE!!!

(Works only for GNOME Display Manager) There is a reddit post talking about how to enable dGPU to enter D3cold state in hybrid mode to reduce power draw. The method seems to work with running Wayland while using nvidia-drm.modeset=0.

It is stated that this method supports D3cold if you're using Geforce 16 Series and above, however this doesn't matter as you can easily enter integrated mode from supergfxctl in case if your dGPU doesn't support it.

*NOTE: This method will make your external monitor to not display anything in hybrid mode (except for MUXless laptops). The display will still works if you're using "GPU passthrough"-ed VM connected to external display.

  • If you haven't done it yet, change nvidia-drm.modeset from 1 to 0 using GRUB configuration. Refer to this section on how to do it.
  • (For Geforce 16 Series+) Enable power dynamic management for power save if you plan to use it on hybrid mode. Copy & paste this code in the terminal:
echo 'options nvidia "NVreg_DynamicPowerManagement=0x02"' | sudo tee /etc/modprobe.d/nvidia.conf
  • In /usr/lib/udev/rules.d/61-gdm.rules, put # (comment out) at the left of ATTR{parameters/modeset}!="Y", GOTO="gdm_disable_wayland" in line 20. Use nano or vim for the editor.
  • Move 10_nvidia.json somewhere else in case if you need it again:
sudo mv /usr/share/glvnd/egl_vendor.d/10_nvidia.json ~/build # move to "build" or "VM" folder if you like
  • Reboot your machine. Check if you are using Wayland:
echo $XDG_SESSION_TYPE # should return wayland
  • Check if you can switch to integrated mode from hybrid. It should say you need to log out, instead of reboot. Do lspci | grep NVIDIA, should return nothing.
  • Check if you can switch to vfio, which be instant. Do lspci -ks 01:00., both should return Kernel driver in use: vfio-pci

How to switch supergfxctl mode using kernel parameter?

To edit your kernel parameter, reboot your laptop and press Right-Shift (or Esc if it didn't work) repeatedly to force GRUB menu to show up. Select latest linux kernel in the option, then press E.

Under the line linux /boot/..., insert/append this (use End button to move your typing cursor at the rightmost position in that line):

... supergfxd.mode=vfio

Of course, you can change vfio to integrated or dedicated if you wish.

Go back to Post VM setup with GPU passthrough

Uh, I follow your Envycontrol's instruction before, how to undo?

If you follow my instructions on using Envycontrol to switch VFIO before, you may need to do as follow to remove envycontrol entirely in terminal:

  • Switch back to hybrid mode: envycontrol -s hybrid then sudo reboot.
  • After reboot, make sure typing envycontrol -q returns hybrid.
  • Delete these files via terminal:
sudo rm /etc/modprobe.d/nvidia.conf /lib/udev/rules.d/50-remove-nvidia.rules /lib/udev/rules.d/80-nvidia-pm.rules /usr/local/bin/envycontrol
  • Do sudo reboot. Now envycontrol removed completely from your system. Make sure typing envycontrol returns command not found... .
  • If you have completed all the steps for my guide before I changed my instruction, you need to install supergfxctl. Click the second hyperlink below to get started. Or if you haven't, ignore this message.

Go back to Before we proceed or Using supergfxctl to bind vfio to GPU

I got error code 43! How to fix? (fake battery fix)

If you have problem installing NVIDIA and you got the error code 43, you might need to add fake battery to your VM to solve the error.

  • Copy this base64 code and decode it to file here:
U1NEVKEAAAAB9EJPQ0hTAEJYUENTU0RUAQAAAElOVEwYEBkgoA8AFVwuX1NCX1BDSTAGABBMBi5f
U0JfUENJMFuCTwVCQVQwCF9ISUQMQdAMCghfVUlEABQJX1NUQQCkCh8UK19CSUYApBIjDQELcBcL
cBcBC9A5C1gCCywBCjwKPA0ADQANTElPTgANABQSX0JTVACkEgoEAAALcBcL0Dk=
  • Rename it to SSDT1.dat and put it in your VM folder (make sure the folder is the same as your ISO file).

  • If you have your VM details window still open, head over to XML edit page, scroll down and between the line </devices> and </domain>, insert these codes and click Apply:

  <qemu:commandline>
    <qemu:arg value="-acpitable"/>
    <qemu:arg value="file=/home/bob/VM/SSDT1.dat"/>
  </qemu:commandline>
  • Try start your VM. If you have problem start your VM because of permission error, you can refer below to fix the problem.

Go back to Test your VM and install NVIDIA driver

I got permission denied or something when I start my VM.

  • This is due to SELinux uses "Enforce" mode by default in Fedora. You can set it to permissive, but you should give permission only for QEMU. In your terminal, enter these commands:
sudo -s # login as root
cd ~ # go to /root folder
ausearch -c 'qemu-system-x86' --raw | audit2allow -M my-qemusystemx86
semodule -X 300 -i my-qemusystemx86.pp
setsebool -P domain_can_mmap_files 1

Go back to Installing Looking Glass in Windows VM or PipeWire & JACK

I still got error 43 even after battery fix! How???

  • If you happened to have older graphics card, NVIDIA might forced you to use older version of driver which doesn't have error 43 fix provided by them. In this case, check out this blogpost to solve the problem.

I don't have dummy HDMI plug for Looking Glass...

If you don't have one, you might be interested in this solution by u/ImaginationLatter523. Just in case if the post got deleted or changed for some reason, I'll leave them here:

  • Start your VM if you haven't, and download custom IddSampleDriver here, and extract it to C:\ (your root C: drive).
  • Open your command line, either cmd or PowerShell if you like. Run these commands:
cd C:\IddSampleDriver
CertMgr.exe /add IddSampleDriver.cer /s /r localMachine root
  • Edit C:\IddSampleDriver\option.txt first before installing. You should edit the resolution and refresh rate according to your laptop's screen.
  • After that, go to Device Manager (tips: right click Windows icon), then click on any device. At the top panel, select Action > Add legacy hardware.
  • Then click Next > choose the link Install hardware that I manually select from a list (Advanced). Click Next while Show all devices is selected, select Have disk > Browse.
  • Find C:\IddSampleDriver\IddSampleDriver.inf, select it and click Ok. Then continue clicking Next. Wait for it to install, and then click Next again to exit the setup.

Go back to Installing Looking Glass in Windows VM

I can't get anti-cheat games to work!

Most anti-cheat games will not work with VM. However, there are solutions that you can use to get around it, albeit will not work on intrusive or ring 0 anti-cheats (eg: Valorant's Vanguard).

There is a great documentation on how to (atleast) bypass EAC anti-cheat provided by VRChat developers. See here for the instruction.

For other anti-cheat, you may need additional steps to hide your virtual machine. For instance, you need to enable "core isolation" under Settings -> Update & Security -> Windows Security -> Device Security. This will impact performance on Intel CPUs, and massive performance drop for AMD CPUs (except for Windows 11).

You can do further by passing through a real hard drive and install Windows there, and also use external monitor all the time and avoid using Looking Glass and remove SPICE server or any remote client software.

For advanced users who are familiar with QEMU/KVM, you can further spoof your VM by passing more smbios info, patch QEMU source code to change hardware names and also patch RDTSC timing. This is by far the hardest if you don't know how to compile a linux kernel. (Note that you have a high risk to get yourself banned from video games that you want to play)

Full stealth KVM resource can be found here. Only do this if you know what you're doing. For RDTSC timing patch, you can follow this instruction on how to do it.

Does this guide works for other distro (Ubuntu, Arch, etc.)?

  • It should work, but there are some steps need to be added or not needed depending on certain distros. Do it at your own risk, and you should do some research & troubleshoot by yourself.
  • For example, choosing OVMF_code.secboot.fd in virt-manager might not be present in other distro as this is exclusively for Red Hat distros repository (which Fedora also includes as well). Some distro doesn't even have OVMF in their repository.
  • Other example, such as SELinux fix might not be for other distro, but they have their own security tool such as AppArmor which requires different ways to fix permission problems.
  • I might add a guide for Arch Linux in the future, if I have more time of course.

How about other editions of Fedora?

  • It should work well, but I personally did not test them yet. Any fedora spins should work, if not the same as normal Workstation edition.
  • Fedora Silverblue/Kinoite might need a slightly different steps, as it is an immutable system. It might be (or maybe?) difficult to do depending on certain steps. They have RPM-ostree and toolbox for package installation, so that adds more difficulty to follow this guide. I might test Silverblue in the future.
  • Nobara Project should work as well, in fact it some steps aren't required. For instance, it has SELinux sets to "Permissive", applied ACS patch, and supergfxctl installed by default.

References


About Joyk


Aggregate valuable and interesting links.
Joyk means Joy of geeK