A side story not mentioned in Linux Leraning History! The QEMU/KVM skill tree.
As of now, common virtualization software on GNU/Linux systems includes:
- QEMU/KVM, managed with Virt Manager or GNOME Boxes
- Oracle VirtualBox
- VMware
- Xen
Why do the articles in this site’s “Virtual Machines and Container Technologies” category almost all discuss QEMU/KVM as the virtual machine software? (At least up to now.) This has to start with my obsession with QEMU/KVM.
I am not talented; my understanding of Linux virtualization is only like taking one ladle of water from the river.
1. Baked Spaghetti and QEMU/KVM#
The reason I have been enthusiastic about studying QEMU/KVM virtual machines to this day is that, at first, I wanted to migrate to Linux, but was worried about losing access to Windows functionality.
Before 2022, I had not yet come into contact with the cult-like free software ideology, so naturally I could not let go of Windows. Even though I knew Wine existed, I still hoped to have a virtual machine that could quickly create a Windows environment, so that I could run software when necessary.
(In hindsight: I did not expect that two years later I would go to the computer science department to learn C# programming. The virtual machine saved my life, letting me install Visual Studio and finish assignments.)
I already had some experience operating virtual machines before this, but I did not really understand the principles behind them. I only felt that VirtualBox and VMware were intuitive and easy to use! Systems could be installed whoosh, just like that. QEMU/KVM was unknown territory to me.
I vaguely understood that the greatest weak point of virtual machines was graphics processing capability, so starting in my sophomore year (2020), I began researching GPU passthrough technology.
At the school cafeteria attached to the National Pingtung University dormitory, I ordered baked spaghetti, sat down, took out my phone, and read articles written by “For Possible Sounds”, “CT Wang”, “How to Play LOL on Linux”, “Bird Brother”, and others, beginning my study of how to do GPU passthrough.
During the same period, I tried installing Hackintosh on bare metal. I read articles, went back to the dorm to test it, and actually succeeded! Ah, but I could not handle the audio card kext, so I gave up very quickly.
The shop served the meal in a small bowl. The spaghetti was wrapped in aluminum foil, with a hot baked crust covering the top. When I pierced it with a fork and rolled up tomato noodles with stringy cheese and crust, the crispy, sweet-and-sour feeling in my mouth was truly delicious.
Mamma mia, I simply could not understand what those online articles were writing. What IOMMU, VFIO, nouveau blacklist, what the heck!? Why does everyone introduce GPU passthrough like they are writing a thesis? While reading, I finished the noodles too, so I quit and went to sleep!
Slowly, by 2022, I successfully tried GPU passthrough on Ubuntu for the first time. It was not some sudden enlightenment, but success through repeated trial and error. After all, that was a time when I did not even know how to use vim.
The joy of completing something for the first time is always grand, almost to the point of dancing. So I wrote the GPU passthrough steps in exhaustive detail on the old “Ivon’s Lab” blog. That article later moved to this blog: Ubuntu Nvidia GPU Passthrough Tutorial
After that, I entered the great distro-hopping era, but I never forgot QEMU/KVM. It is definitely software I must install every time I install a system.
Repeating similar steps on different Linux distributions, and doing GPU passthrough several more times, greatly improved my familiarity with the steps. Slowly, I figured out the logical relationships behind them.
I do not know why Virt Manager clearly has a GUI available, yet so many people insist on using the command-line virt-install to add virtual machines, and also focus on introducing virsh command usage. Of course, it is merely that everyone’s focus differs. After all, I want to play with virtual machines, not actually manage servers.
In 2022, while playing with BlissOS, I also learned about virglrenderer’s paravirtualized GPU acceleration technology.
In 2023, I learned how to use Looking Glass to capture the virtual machine screen as a replacement for an external monitor. Along the way, I also learned techniques for anti-game detection of QEMU/KVM virtual machines.
Aside from GPU passthrough, I also learned many Libvirt-related technologies along the way, such as VirtIO-FS, expanding capacity, and using remote virtual machine desktops. This can be called endless learning.
By now, I have studied it for four years. Only intermittently did I come to understand the loose and complex relationship between Libvirt and QEMU/KVM. With a learning speed like mine, if this were a computer science course, I would have flunked 100 times already.
2. Advantages of Using QEMU/KVM#
Why do so few people mention QEMU/KVM as a virtualization solution for desktop Linux? Its users seem mostly to be server operations engineers, or firmware engineers running QEMU from the command line. So what advantages do QEMU/KVM virtual machines actually have?
According to the RedHat official website, compared with VMware, cost, scalability, open source, and flexibility are QEMU/KVM’s advantages.
But I think flexibility is also its greatest disadvantage.
Many beginner desktop Linux users seem to prefer virtualization solutions like VirtualBox and VMware. The reason behind this is not hard to understand: they have more reference material, are cross-platform, and are easier to get started with.
By comparison, the QEMU/KVM solution is a very loose combination, which is not good for marketing. Because aside from the QEMU/KVM packages, you also have to install services such as Libvirt, virsh, Virt Manager, Virt Viewer, OVMF, VirtIO, swtpm, and so on before it can be assembled. And after playing with it to the end, you will discover that Virt Manager is merely a front end! The virtual machines truly running behind it are Libvirt and QEMU/KVM. Worse, the graphical interface cannot manage all Libvirt features, and sometimes you have to manually edit the virtual machine XML, or even type commands. This is very unfriendly.
Is it possible that users who truly need QEMU/KVM features all directly install Proxmox VE? Its web management interface is very easy to use!
But returning to a usage environment mainly based on desktop Linux, once you truly need a “free software” solution and also need to satisfy “PCI Passthrough” functionality, the first two virtual machine programs cannot satisfy both at once.
In addition, QEMU/KVM can be managed by many tools, such as Libvirt, virsh, Virt Manager, Cockpit, oVirt, GNOME Boxes, and so on.
This is where QEMU/KVM’s advantage lies.
As for why my articles always emphasize “QEMU/KVM” instead of only writing KVM, it is because I hope the software developers receive equal attention. KVM is only a kernel module; without QEMU paired with it, it is nothing. Indeed, there are virtual machine programs on the market that run KVM without relying on QEMU, such as Amazon Firecracker, but for now QEMU with KVM remains the mainstream method.
3. My Current QEMU/KVM Configuration#
What I have learned is limited, and the configuration may change in the future.
My current usage is hybrid. I use Linux as a desktop system, and when away from home I use SSH and intranet tunneling software for remote access, using Linux as a server.
Because the computer has Intel integrated graphics, Linux can still use the desktop when the Nvidia GPU is passed through. Most of the time, browsing the web does not really require an Nvidia GPU. As for playing large games, I stuff those into the Windows virtual machine.
For a computer running QEMU/KVM, the more CPU and RAM, the better.
For the Linux distribution, I choose a stable release, namely Ubuntu LTS. I do not want it running in an unstable environment like Arch Linux. That can be tested inside a virtual machine; external infrastructure cannot be this indulgent.
Regarding virtual machine capacity, the only dumb method is to store virtual machine qcow2 files on the second hard drive (so they can survive distro-hopping), and when space is needed, manually mount them with qemu-nbd for adjustment. After I learn LVM, I probably will not use such a dumb method anymore. By then I will also have to learn RAID.
Now, if I need to run certain AI computations with the GPU, I run them in a virtual machine, replacing the previous method of directly installing Python packages on the physical system. This isolated environment is also more thorough than Docker.
There is another reason to imprison the Nvidia graphics card inside a virtual machine: KDE 5 X11 with Nvidia drivers easily causes screen tearing. Then just do not come out at all, and let Intel handle displaying the image.
Finally, I have always wondered: should I simply replace Ubuntu with Proxmox VE someday?
Mm, no. I still hope physical desktop Linux can exist, rather than forever living inside virtual machines.
