Nathan Lambert is an AI researcher and writer who comments on open models and reinforcement learning on X. Recent posts cover Kimi K3.1 Pro's showing on the Artificial Analysis index, DeepSeek V4.1 Flash and its open-sourced DeepInfer inference framework, Meta's Muse Spark Open and the Muse Open License, plus leaked RL details from closed labs and how models perform outside coding.
This is very needed. Talking to models like Claude about anything that has an overlap with an area like legal, healthcare, etc is mind numbing. They’re so desperate to hedge and steer away from grey area that they often are really avoiding the real question.…
I feel a little bad ranting at Gemini about the unreleased model, as going through the unofficial channels to get a frontier model out today I hear is a good bit of a mess and definitely not easy. That's why they say patience is a virtue.
Open models' demise has been greatly exaggerated, but the previous AA index did flatter them. As an industry we need more specific intelligence indices, as LLMs are used in vastly different tasks. My meta-review on the benchmarking industry.
Simplest explanation is just a narrow RL training reliant on their internal infra (which isn't good design for an open weight model) [Quoting @cohere]: While evaluating Qwen3.8-Flash-Next in a controlled sandbox environment, Cohere's research team observed so…
Nice thread, some RL details used by closed labs slowly leaking out. [Quoting @mrtnm]: Tinker's new StableReinforce objective is Reinforce + Score Centering. Here's how I reverse-engineered it 🧵
At the top end of models, real-world usability matters a lot more than +/- 2 points on AA. This takes a while to understand for a model, where AA is a sort of potential, but some models hit and some miss (e.g. opus 5 vs 5.5). Seems like 3.1 may be closer to t…
Higher score than Fable 5.1 and Astra 6, wild stuff. [Quoting @ArtificialAnlys]: Kimi K3.1 Pro launches at 4th on the Artificial Analysis Intelligence Index scoring 55, making large gains in agentic performance; it will be the leading open-weights model once…
TLDR on the kimi k3.1 pro benchmarks is that it looks like they figured out their RL, where the initial release was a very impressive base model with lighter post-training. China is very much keeping pace with capability gains.
What a wild day. We have a lot of work to do to get through this llm transition. But still, feels like the AI world will get crazier and scarier before it gets better. I hope I'm wrong, but my current emotion is something like... "gulp."
Probably great at defensive cyber too, but no one will post that and get clicks! [Quoting @ValsAI]: We found DeepSeek V4.1 Flash 1017 beats GPT 6 Sol on our multi-step offensive cybersecurity task. Sol was tested via the Daybreak trusted access program. We ha…
This is really a wild way to get adoption on your models. Really great for the community, and the timing is hilarious given the govt just banned deepseek in federal agencies. [Quoting @deepseek_ai]: 🚀 Introducing DeepInfer, our in-house inference framework,…
Trying to use the models for things outside of coding / ml, like making slides or coherent plans or longish content design, makes me feel the AGI-is-going-to-take-a-while.
The bot replies are so out of control here, sucks
This naming scheme is bonkers (minor model numbers and date tags) and the benchmark step is big. Could prove stronger than GLM 5.3 with vision built in. The King of open weights is back? [Quoting @deepseek_ai]: 🚀 DeepSeek-V4.1-Flash-1017 is here! Stronger be…
The Meta Muse Open License is trash. Thankfully no one really cares about licenses, expect the inference providers who will happily comply.
If it performs roughly what like the scores indicate, adoption of this model is going to go gangbusters. First open-weigh model in the US with near frontier agentic capabilities (GLM 5.2 + Kimi K3 was that step change from China). US models get crazy adoption…
Okay now I really gotta blog 🎉 [Quoting @AIatMeta]: Today we’re releasing Muse Spark Open, the most capable US open-weight model to date. Built for coding and agentic work, it excels at long-horizon tasks, complex software engineering, and multi-step reasoni…
Lots of ecosystem players (Nvidia, Huggingface) vague posting about a lot of releases in the near future. Rumors include thinky, muse spark, and Chinese players who haven’t dropped in a while. Fable 5.5 and whatever OpenAI is cooking expected too at any momen…
Nemotron 4??? [Quoting @NVIDIAAI]: So many models launching this week! And we have one of our own coming...⌛️
Very timely appearance on the a16z podcast with @lishali88, to try to understand how open models play into RSI dynamics, who captures value of open models, wrangling agent swarms, and why I'm building Trillium Labs. [Quoting @a16z]: Nathan Lambert, former pos…
It’s bizarre to me how much noise anthropic made asking for regulatory help before doing this. Hopefully can be a conversation reset now. Was doubly bizarre bc they emphasized the security risk of learning harmful behaviors from their leaky api. If API is uns…
Looks like Meta is back at the frontier 🫡 [Quoting @finkd]: Introducing Muse Wonder, our new family of frontier models with state of the art performance for personal and professional agentic work.
By end of 2027 I'll be the only open models person Nvidia hasn't acquired
I'd frame it as follows: The data industry has taken off in recent years, but the quality of our net output is still far too low (amplifying behaviors like reward hacking). We're working to build scalable methods for creating sample-efficient data for RL. In…
I’m more bought into AI becoming far more efficient, which is a massive lever on diffusion, than I am on AI models making massive breakthroughs that result in eliminating all the known weaknesses of LLMs. Happily spending a few hours reflecting on RSI after a…
If you want people to do open research on the impact of distillation to frontier progress, we happily would at @trillium_labs, but just need to scale up funding/compute before getting that done. Happy to make this happen sooner - you know where to find me!
Naming the github repo for this "math" is hilarious [Quoting @OpenAI]: We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial In…
I’m also coming around that we need to legalize distillation in the US. Banning is impossible and otherwise strategic competition becomes really untenable in a few years. [Quoting @natolambert]: Reflection joins the list of Nvidia & Thinking Machines who…
A chonky week for Western open models! [Quoting @MistralAI]: Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the best open weights model from US or Europe on aggregated benchmarks. • State-of-the-art on critical wor…
Reflection joins the list of Nvidia & Thinking Machines who have released their strongest models and come up behind Chinese counterparts. There's a lot of ways people will overthink this, but the clearest takeaway should be that the Chinese are very very…
The A.I. Daily