jitllm Moves Model Layers Between GPUs Mid-Reply in 8 ms

Open-source inference engine jitllm moved all layers of Llama-3.2-3B between two T4s mid-reply with an 8.1 ms pause, paging the KV cache alongside the layers so no tokens were recomputed and output matched an unmoved run; on V100s the pause was about 20 ms and on a laptop crossing CUDA and Vulkan about 91 ms.

• Post: owning a GPU cluster in 2030 like a gun for home defense2026-10-12
• Forum asks whether any hidden-gem GPUs remain2026-09-26
• User builds server with six V100 GPUs in standard case2026-09-18