The A.I. Daily
ds4
Local inference engine for DeepSeek 4 on Metal, CUDA, and ROCm.
ds4
4 storiesds4 is a local inference engine for running DeepSeek 4 open-weight models on Metal, CUDA and ROCm. Recent coverage centers on its appearance on GitHub Trending and its release by the creator of Redis: a narrow C engine, MIT-licensed, bundling a CLI, HTTP APIs and a native agent, and using asymmetric quantization to make MoE models practical locally. Related discussion covers tuning GLM-5.3-Flash on Apple's M3 Ultra.
Related topics
ds4
Local inference engine for DeepSeek 4 Flash and PRO on Metal, CUDA and ROCm.
Redis creator releases ds4 local LLM inference engine
The creator of Redis released ds4, a narrow C inference engine that runs frontier open-weight models such as DeepSeek V4, GLM 5.x and Qwen3.8 locally on high-memory Macs and CUDA/ROCm machines. MIT-licensed, it bundles a CLI, HTTP APIs and a native agent, using asymmetric quantization to make MoE models practical locally, with roughly 34.4 T/s generation and 557 T/s prefill on a 128GB Mac.
GLM-5.3-Flash optimized on M3 Ultra for 60 tps / 550 tps
An engineer optimized GLM-5.3-Flash on Apple’s M3 Ultra by fusing kernels and using parallel scans, boosting throughput from 21.6 to 37.4 t/s at 300k context and achieving up to ~60 t/s for SQL workloads at short context.
That is everything