Show HN: Janus – Go binary that runs GGUF models via Vulkan on AMD/Intel/Nvidia

Janus is a API router for AI models written in Go and has a Vulkan Model runner - Vibra-Ingenn/Janus

44 points•Maverick617•about 4 hours ago•5 comments•

5 comments

PcChipabout 4 hours ago

I didn't see any benchmarks against vllm, sglang, exllama, etc

rancorabout 3 hours ago

Since this is basically a wrapper around libllama.so, I would assume that the performance is roughly the same as llama.cpp upstream.

dlcarrierabout 2 hours ago

From what I've seen, Vulkan adds a lot of overhead on Intel hardware.

peddling-brinkabout 2 hours ago

> llama.cpp via Vulkan (AMD / Intel / NVIDIA) or CPU fallback

I got excited about someone paying attention to intel. Oh well.

kamranjonabout 1 hour ago

llama.cpp sycl and vllm xmx work is pretty incredible right now - you just gotta build it with some extra flags