Vulkan
3 storiesVulkan is a cross-platform graphics and compute API often used to run local AI models across AMD, Intel and NVIDIA GPUs. Recent coverage includes Janus, a single Go binary from a developer that runs GGUF models via Vulkan or CPU with no Python, Docker or Ollama, and an enthusiast's six-board BC-250 cluster running Qwen Next Flash on four boards at about 28 tok/s.
Related topics
An enthusiast built a local AI cluster from six BC-250 mining boards, running Qwen Next Flash on four at about 28 tok/s and a 3.6 35B Q4 model on two at 60 tok/s.
A developer released Janus, a single Go binary that runs GGUF models on AMD, Intel, or NVIDIA GPUs via Vulkan or CPU, exposing an OpenAI-compatible API. It requires no Python, Docker, or Ollama, and supports hot-swapping models and thinking models.
virtio-nvgpu forwards NVIDIA kernel driver ioctls at the driver ABI level, letting a Linux guest run the vendor's own unmodified user-mode drivers. On an RTX 3060, guest performance is within 2% of bare metal above about 2 ms per frame and costs the same CPU; four guests together hit about 103.7 fps, targeting headless streaming.
That is everything