Virtio-nvgpu: Near-native Nvidia GPU access inside a KVM guest

Virtio-nvgpu: Near-native Nvidia GPU access inside a KVM guest github.com

[Experimental] A virtio device for near-native NVIDIA GPU access in KVM virtual machines. - nestrilabs/virtio-nvgpu

155 points•WanjohiRyan•7 days ago•65 comments•

#virtio #gpu #nvidia #near #native #nvgpu #access #kvm

65 comments

There are roughly 3 ways to give a KVM guest a GPU.

1) VFIO passthrough: host binds entire GPU to guest as PCI device, which only allows one VM to use the GPU, thus you sacrifice your host display too (unless you fallback to integrated graphics on cpu etc). Strongest isolation because host kernel module driver not involved.

2) virtio-gpu: guest sees paravirtual GPU and loads virgl/venus mesa driver which serializes graphics API calls and replays them on the host driver. This allows multiple VMs to use the GPU, but performance overhead can be significant, and guests can’t practically leverage lower level primitives eg NVENC without paying price of CPU readback.

3) virtio-nvgpu (this repo): guest loads standard NVIDIA user mode driver (closed source), a fake /dev/nvidia* kernel module copies ioctl bytes + handle onto queue for host kernel mode driver to execute. This also allows multiple VMs to use a GPU, but is near native speed due to low overhead. Unfortunately the tradeoff is this project has the weakest isolation, eg every guest ioctl is forwarded to the host by default, the VMM holds read/write FDs, no seccomp/caps/allowlist. With respect to There is basically no GPU related security measures here, the exposure is the same as running multiple processes using the GPU with no VM. Only caveat is these guests can’t drive a physical display, so there is some restriction of surface area but it feels incidental rather than intentional in this case.

Anyways this is a tough problem OP, I don’t want to discourage you.

Without hardware/driver support for isolation (MIG) on consumer grade NVIDIA GPUs, it won’t be possible to solve this properly for a long time.

Also a factor is that NVIDIA has no open Mesa driver to support a native context approach (guest owns GPU command buffers, host maps them) like we have for AMD/Intel.

deltoidmaximus7 days ago

A week or so ago when I stumbled on this he at least seemed to be clear about the potential security implications, which gave me pause. But it doesn't seem any worse than just running software on your main OS which is what most people do. Sure, it's DoA for a hypervisor in a data center but that isn't the only use case out there.

What kinds of things can a guest running undesirably applications (viruses, malware, LLM escaping a sandbox, etc) get up to with shared GPU access?

Sohcahtoa826 days ago

> Sure, it's DoA for a hypervisor in a data center but that isn't the only use case out there.

Indeed. For me, I find the ecosystem around AI/LLMs works better on Linux than Windows, but Windows is my main OS since I'm a gamer. Being able to run GPU-accelerated AI in a VM is huge for me.

In my case though, I just use WSL which does an amazing job.

> What kinds of things can a guest running undesirably applications (viruses, malware, LLM escaping a sandbox, etc) get up to with shared GPU access?

The most obvious answer is a DoS. If my malicious VM is sharing a GPU and has full access to it, I could simply tell the GPU not to run a victim VM's workload, or manipulate it in some way. I might not be able to pivot to having a shell on their VM, but I could at least read/write their data in VRAM. If it contained secret data (custom model, or secret data being processed by AI), I could easily steal it.

tarruda7 days ago

What is the state of virtio-gpu these days? I have a system76 "pang14" (Ryzen 7 7840U laptop), is that a valid option for me to have a VM with 3d acceleration? I'm fine with having some overhead if it is better than the current software GPU.

ranger_danger6 days ago

If you don't need Mac or Windows guests then the existing virtio-gpu works in my experience.

madushan10007 days ago

This is not a completely novel approach. This has been a thing in virtio-gpu for a while, it's called "DRM native context".

XorNot7 days ago

I'm more interested how this would effect providing VM graphics with migrations between hosts.

The dream would be put the user OS in a VM in a lab, and then be able to suspend and resume seamlessly if you need to push it to a new workstation, with locally accelerated graphics available.

JackSlateur7 days ago

4th solution: nvidia vgpu

If I remember correctly, the idea is that you have a physical GPU and you split its memory (with, eventually, time-budget) to create multiple virtual GPUs, which can then be associated with a KVM guest and use by it

(not available legally on consumer-grade GPU)

KetoManx646 days ago

I don't believe it's illegal to run your own firmware. It's just not supported by the manufacturer.

mcmatterson7 days ago

Amazing project! But man, that README is just a textbook example of LLM word salad. It's wild how these tools are so incredibly capable at many things, but their writing sticks out like a sore thumb

LiamPowell7 days ago

When the lines in the ASCII charts don't even line up my immediate assumption is that the author didn't even bother to glance at it.

Borealid7 days ago

The percentage-overhead comparison is pretty choice nonsense. It has only percentages to try and "explain" that overheads don't matter if the system is slow anyhow.

A fair comparison would be this project vs virtio.

WanjohiRyan7 days ago

Virtio? what virtio? virtio native drm native context, is that what you mean? We actually use it in nesbox[1] for AMD/Intel cards.

Venus is the only one we could directly compare to, as it is the only one that supports Nvidia GPUs. vDRM works only on AMD/Intel GPUs and has a similar performance (~98% baremetal performance) to virtio-nvgpu.

[1] https://github.com/nestrilabs/nesbox

vlovich1237 days ago

Claude is much worse for having a distinctive style you can spot from a mile away. I’ve found GPT-6 to not suffer from this or it’s insanely verbose markdown salad.

verall7 days ago

I see everyone saying this but I've been using 6-astra lately and afaict it's not much better

serf7 days ago

it's not just literary authorship, it's pretty easy to spot LLM driven programming paradigms too, especially if you look at the test suites of a given package.

WanjohiRyan7 days ago

My bad, i am not a native English speaker... I did my best to try and brush it up. Terribly sorry if it did not match your flow.

jchw7 days ago

LLMs are pretty good at translation, they're just pretty awful at generating natural-sounding English prose from scratch. In my opinion, probably the best solution is to simply write the README in your native tongue and use an LLM to translate it.

mcmatterson7 days ago

All good friend! I'm as guilty as anyone. It's a super cool project though.

Now that I have you on the hook, is there any benefit to this over virtio for a single KVM passthrough situation? I previously ran a proxmox based gaming PC setup (docs here: https://github.com/mtrudel/rabble/tree/4d9329f3dd0fb09123a8f...), and was lucky enough that the GPU passthrough part of that build 'just worked'. I'd started down a path of trying to share the GPU between VMs based on a naive 'one VM owns it at a time' setup, but never really got it off the ground.

lisp22407 days ago

Jdkhdkhdoudludlhxljdluelxfgncbt vkcljxlj. Fulxkhxjcouxyo. Choxoufoudljcljcljc gufkhxpjclixluxouc vfuoxoudydoudljc ycoucupck mccddhxhoxohxoj ljxohhlxljcluxkh ckhxoydoudljcljcljn lhxkhdhx.

Sorry. Qwerty isn't my native keyboard.

Fucukdyshd iohhfghfhgh ggbvfg gdh ghcfjh hdhvbbghb hjjgigggfv.q gfghbdbdbd cjcjcjcnn. Dr rhrjfnnf cjcucjcjjcnr rbjjdisnxbbfb hehrbrbrb. Xjcjcjcjbd rhrjrnrbrb xhfjcjcjbde hdhdbcncbfn dhdjjfjchcnbdbdbffnndnd

athrowaway3z7 days ago

I'm a guy with vague knowledge on KVM - having only tinkered with it and briefly had a GPU pass-through setup 2 years ago.

I suggest putting the 'multiple guests at near-native speed' use-case in the opening paragraphs of the README.

WanjohiRyan7 days ago

Thank you for the feedback... i will do that

mjg597 days ago

Hrm, something of a lack of discussion about what level of access the card has to the host in the absence of IOMMU-restricted passthrough.

orphereus7 days ago

I am very sceptical about this. I have some experience in GPU virtualization and passthrough with Nvidia GPUs and they are really not designed to be able to share them with multiple guests/host without Nvidia's blessing (licenced drivers).

WanjohiRyan7 days ago

Yes they are not... we run the Nvidia drivers unmodified. Only thing we have done is make the guest driver think it's running on the host, talking to the host's GPU kernel.

It works really well with a ~2% performance penalty. Nvproxy by google/gvisor has been doing this for years.

You should try running it yourself and see how it goes :D

az2267 days ago

If memory serves, there is a way to hack the drivers to unlock MIG for GeForce GPUs.

markasoftware7 days ago

How is this different than gVisor's nvproxy? https://gvisor.dev/docs/user_guide/gpu/#compatibility

Edit: nvproxy is mentioned as the "direct inspiration" in the readme without mention of how this is different or why it doesn't use nvproxy as a backend.

WanjohiRyan7 days ago

We borrowed a lot of the architectural design from nvproxy, then built it to support graphical workloads. Plus it is reusable in such a way you can hot plug it into any microVM, cloud-hypervisor, maybe even Firecracker

Read the full thread on Hacker News →

65 comments

Related stories