samir-vllm

1 stories

Every samir-vllm story collected by The AI Daily, 1 so far, newest first, refreshed hourly.

Related topics

Open-source Project Teaches LLM Inference from Scratch in C++/CUDA

A teaching project, samir-vllm, appeared on Hacker News, explaining LLM inference from first principles by building a small engine in C++ and CUDA. It offers over 6 hours of video and 80+ pages of handwritten notes covering weight loading, RMSNorm, RoPE, self-attention, KV cache, batching and paged attention. It already supports loading Llama 3.2 1B Instruct safetensors weights.

Hacker News ·
That is everything