gpt-oss-120b
5 storiesgpt-oss-120b is an open-weight mixture-of-experts model from OpenAI that developers commonly run locally. Recent coverage focuses on deployment: one developer kept expert weights on an SSD and loaded them on demand, averaging 7.84 tokens/s within a 16 GiB memory budget on a 32GB M1 Max; Nvidia's DGX Spark drops from 65.4 to 13.9 tokens/s on Linux due to encrypted firmware; and a paper puts self-hosted cost at $0.12–$0.35 per million output tokens, with the A100 cheaper than the H100.
Related topics
Benj Edwards designed and AI agents coded an unofficial Super Nintendo experiment running a roughly 2.2M-parameter model on the Super FX 2 chip, writing answers word by word; an optional build adds 150,000 Wikipedia article openings as a fact book. Its creator stresses it is not a useful assistant and often errs.
A developer kept MoE expert weights on SSD and loaded them on demand, running gpt-oss-120b and Qwen3.5-122B-A10B within a 16 GiB process budget on a 32GB M1 Max. On gpt-oss-120b it averaged 7.84 tokens/s, about 36% faster than TUFF, and degraded less under competing memory load.
On Linux, Nvidia's DGX Spark laptop locks the GPU to about 780 MHz versus 2.2 GHz on Windows, because its encrypted power firmware is only readable by the Windows driver. GPT-OSS-120B inference drops from 65.4 to 13.9 tokens/s. WSL keeps full performance; a real fix needs Nvidia.
A user on r/LocalLLaMA found that the same IQ3_S model produces coherent output in llama.cpp but degrades into repetitive loops after roughly 10k tokens in another engine. This highlights how much the inference framework affects long-form generation stability for local LLMs.
A paper finds open-weight inference costs about one-fifth of comparable closed models, with self-hosting at $0.12-$0.35 per million output tokens. On gpt-oss-120b, the A100 is cheaper than the H100, and A100 five-year rental prices retain 80% of one-month prices, showing older GPU families keep multi-year earning life.
That is everything