gpt-oss-120b

5 stories

gpt-oss-120b is an open-weight mixture-of-experts model from OpenAI that developers commonly run locally. Recent coverage focuses on deployment: one developer kept expert weights on an SSD and loaded them on demand, averaging 7.84 tokens/s within a 16 GiB memory budget on a 32GB M1 Max; Nvidia's DGX Spark drops from 65.4 to 13.9 tokens/s on Linux due to encrypted firmware; and a paper puts self-hosted cost at $0.12–$0.35 per million output tokens, with the A100 cheaper than the H100.

Related topics

Economics of Open-Weight Inference: Older GPUs Retain Value

A paper finds open-weight inference costs about one-fifth of comparable closed models, with self-hosting at $0.12-$0.35 per million output tokens. On gpt-oss-120b, the A100 is cheaper than the H100, and A100 five-year rental prices retain 80% of one-month prices, showing older GPU families keep multi-year earning life.

Hacker News ·
That is everything