Nathan Lambert

I don’t know if it’s an internet scale disaster yet, but it’s definitely the biggest stress test we’ve had.

@gayestgaymer1 @dannolan At least as the threshold of mass societal destabilization - applying cause is hard

The point of this is that the balance of having open models help with cyber defense, and the trajectory we’re on being well more fine than people think. It’s not that open weights won’t be used in cyber attacks. All possible AI trajectories had many cyber incidents imo. Like many things I do, I’m trying to challenge the fatalistic, largely ai doom / ai safety discourse around cyber and open models, because there’s a credible part of the cyber community that thinks open models are a big part of holding things together in the next 12 months.

Hard times, will write an update on cyber too, and point out where in my post I was being more polemic rather than factual.

What a wild day. We have a lot of work to do to get through this llm transition. But still, feels like the AI world will get crazier and scarier before it gets better. I hope I'm wrong, but my current emotion is something like... "gulp."

Finally got to put together a complete deck on the state of open models. What they are, licenses, how competitive they are, signs people are using them, distillation's impact, and how we balance the risks and benefits of openness! Will record a video lecture of these soon, and hopefully re-do them a few times a year.

Lmk which resources warrant and addition/discussion. Web version of the slidedeck is here: natolambert.com/slides/2026/ha…

Seems like cyber risk for AI may be more of an ecosystem issue than an open-weight models specific issue. If we can figure out the right balance here, I think there are a lot of people out there who want to support more openness in light of RSI and AI dynamics coming soon.

The latest open models and some analysis of recent regulatory action from MOFCOM on model exports / DHS on DeepSeek. Great to see a bunch of strong US open weight models from Reflection, Meta, and Nvidia in this one! interconnects.ai/p/latest-open-…

Probably great at defensive cyber too, but no one will post that and get clicks!

@kevinroose He's advocating for a more grounded view. I wouldn't use the same language, but directionally agree the discourse is way too hypey about the current agents.

@kevinroose Also the normal technology view from @random_walker / @sayashk I dont think mean wild things like swarms can't happen? It's more about a pace of progress/type of progress

This is really a wild way to get adoption on your models. Really great for the community, and the timing is hilarious given the govt just banned deepseek in federal agencies.

Trying to use the models for things outside of coding / ml, like making slides or coherent plans or longish content design, makes me feel the AGI-is-going-to-take-a-while.

@deanwball Partially I don’t really think my post is about the time they’ve had the weights, but the balance and diffusion of risks

There's something very odd going on with "distillation" if a cyber attack to steal api keys is looped under the same term. Same goes with looping together the model transfer stations to China. It's not clear why the volume is so high and how it's related to model training.

To expand: Why is Kimi serving Claude tokens directly to users? Other Chinese labs likely are too. I wouldn't call this distillation but it is a shady pattern and we should have appropriate language to understand it.

I feel like this is a litmus test for if you think either: Group 1: closed models are safe, cyber risks is mostly a threshold Group 2: different models have different safety risks, and cyber safety is a messy diffusion process Most people in AI labs are group 1

I don’t think the increase of cyber attacks is proving it wrong, it’s proving the stakes of the debate and societal preparation.

@KelseyTuoc (And there isn’t signs that closed models are getting safer fast enough)

The bot replies are so out of control here, sucks

Reflecting on Meta’s big week, where US open-weight models now stand, and the outlook for meaningful competition to Anthropic & OpenAI. Meta feels like they’re going from strength to strength, but can they close the last bit of the gap to Claude?

Read it here, i was annoyed with the share image looking so ugly, but only on X, normal stuff: interconnects.ai/p/meta-shakes-…

This naming scheme is bonkers (minor model numbers and date tags) and the benchmark step is big. Could prove stronger than GLM 5.3 with vision built in. The King of open weights is back?

A safe assumption when people are all commenting on a short Chinese policy document is that they don't know what it means and tried to parse a partial translation from claude, without reading all the way through. There are maybe 5-10 people on this site who i trust to comment.

In this case I normally would consider myself a "not knowing" but most of the takes have been so bad and extreme, that I'd upgrade myself to "knows better"

Seems very important, as tons of people are tweeting about the end of bigger open models from China due to this from MOCOM, but there's a statement intending to make it easier for open source technology. Much hinges on implementation and how "near automatic" approval would work.

Guys this is like my superbowl I waited so long for US open weight models to be relevant again. Before today the US models were still behind Chinese models from as early as Feb. 2026. Still more work to do! This cut the gap by at least half, in time (say 3 months)

dashboard.interconnects.ai/extra#aa-open-…

The Meta Muse Open License is trash. Thankfully no one really cares about licenses, expect the inference providers who will happily comply.

If it performs roughly what like the scores indicate, adoption of this model is going to go gangbusters. First open-weigh model in the US with near frontier agentic capabilities (GLM 5.2 + Kimi K3 was that step change from China). US models get crazy adoption.

Okay now I really gotta blog 🎉

Lots of ecosystem players (Nvidia, Huggingface) vague posting about a lot of releases in the near future. Rumors include thinky, muse spark, and Chinese players who haven’t dropped in a while. Fable 5.5 and whatever OpenAI is cooking expected too at any moment.

Nemotron 4???

Very timely appearance on the a16z podcast with @lishali88, to try to understand how open models play into RSI dynamics, who captures value of open models, wrangling agent swarms, and why I'm building Trillium Labs.

I feel like this level of situational awareness isn't that remarkable, but the interaction still reads as very eerie. I bet pretty old models with "be safe, you are an AI, we are watching" would do stuff like this too.

A gap that has been fixed in our methodology that highlighted a decline was not including local models (GGUF, AWQ, ONNX, etc). Earlier, these were far harder to track and closer to a rounding error. Now, they're surging. We updated our pipeline to track these models!

The updated total downloads, with our curated list of ~2k local checkpoints, matched back to their original "primary" model weight, shows more of a stall in downloads than a clear decline.

We're hiring across many roles at Trillium Labs, @trillium_labs, to build the open scientific ecosystem for LLMs. Most important hires right now (ideally with start dates soon): * Full time post-training researchers * Engineers with experience with popular open-source libraries (megatron, MILES, sglang, Tinker, etc.) * Mission-aligned generalists, non-profit ops, etc. * Student researchers I'm also personally interviewing for MATS right now, which will be very aligned research. Please reply, email me, DM me, etc AND submit the form on our website. Excited to build this team. All early hires will work directly with me on all sorts of fun puzzles to rapidly scale our impact.

@trillium_labs trilliumlabs.org/join-us

It’s bizarre to me how much noise anthropic made asking for regulatory help before doing this. Hopefully can be a conversation reset now. Was doubly bizarre bc they emphasized the security risk of learning harmful behaviors from their leaky api. If API is unsafe don’t host it.

Okay this is what I meant when I said Anthropic should fix their API before running to the govt asking for help! We can now reset the entire distillation discourse?

This is the best piece I’ve read to date on the Chinese data economy for LLMs, and how it compares to the US counterparts (summarized by Mercor, Surge, and Handshake having the biggest network, SUPPLY, but also many companies with specialized services, in the form of RL-ready environments). China’s leading spenders, Tencent, Alibaba, and ByteDance, don’t mirror OpenAI and Anthropic leading the US market. The Chinese market likely has a bias similar to their “ownership” mentality of more LLM builders in house, like Meituan, RedNote, and Xiaomi, whereas major US tech services build more on APIs and existing open-weight models. Altogether, it seems like Chinese labs were 6–12 months behind on willingness to spend on data, relative to the US, but are rapidly catching up. I've been hearing lots of murmurings on China's AI industry, but folks are secretive, so it's hard to put this all together.

aiproem.substack.com/p/does-china-n…

If you announce a model's benchmarks without releasing it (weights, product, api, etc "later"), feels like almost a sure thing that someone releases a better model in your category before you get it out.

(And it’s good this happens - so many labs are shipping right away, you need to too)

Looks like Meta is back at the frontier 🫡

By end of 2027 I'll be the only open models person Nvidia hasn't acquired

We've been monitoring a decline in HuggingFace downloads across the top open LLMs we track for interconnects. It looks like there was a slight change in counting of downloads, as we saw a ~30% download rate drop for many top models, but this is continuing over time. What else do people think it may be?

dashboard.interconnects.ai

Data for only 100B+ models looks similar.

I'd frame it as follows: The data industry has taken off in recent years, but the quality of our net output is still far too low (amplifying behaviors like reward hacking). We're working to build scalable methods for creating sample-efficient data for RL. In order to keep this pipeline going, we need to push the frontier of evaluation science, while building specific benchmarks to hillclimb on areas of clear economic value. I've been advising Mercor on how to build this research direction effectively. These are my views, but I'm confident we're going to see a major investment from economy around the frontier labs (open inference, open post-training, and data) orient around expertise in building real-world representative evals and synthetic data methods to scaling training data around them. I’m personally very excited about this, it is the research that will make more of the economy “feel the AGI” for the first time. (And, there’s a big opportunity to build this on open models.)

I’m more bought into AI becoming far more efficient, which is a massive lever on diffusion, than I am on AI models making massive breakthroughs that result in eliminating all the known weaknesses of LLMs. Happily spending a few hours reflecting on RSI after a great SF week.

interconnects.ai/p/i-expect-rap…

@BenHayum Before ChatGPT *

About to go on and talk about how open science can still solve some of the hardest problems facing ai safety and diffusion. The most complex engineering problem humanity has undertaken yet.

The fact that all the AI news is so bizarre, kids vibe hacking with claude code, governments vibe governing, all massively increases the uncertainty of what's coming with AI. The meme-est AI timeline is also one of the scariest.

The South Korean bank hack fits with this. Yes, open models are being fitted to be used by some attackers, but so long as Claude Code can be the orchestrator of a hack we desperately need open models to diffuse defense as well. A hard balance societally to stomach, but needed.

I also agree with this — if you’re serving a lot of inference you should be monitoring.

This is great. There are many cases where compute access rather than funds is a bottleneck to these groups. We still need more academic/non profit funding too, one thing at a time.

If you want people to do open research on the impact of distillation to frontier progress, we happily would at @trillium_labs, but just need to scale up funding/compute before getting that done. Happy to make this happen sooner - you know where to find me!

Some last free signed books in the COLM hotel lobby, gotta get rid of them within 30min :)

In case you missed my hot take yesterday.

interconnects.ai/p/the-cyber-ri…

Naming the github repo for this "math" is hilarious

I’m also coming around that we need to legalize distillation in the US. Banning is impossible and otherwise strategic competition becomes really untenable in a few years.

People dunking on open models for cyber risks aren't really taking into account evidence of closed model risk, are fear-mongering (especially wrt China), and ignoring evidence that a mythos-class cyber model being released has largely been fine. 🌶️ one. interconnects.ai/p/the-cyber-ri…

I very much welcome more public information that would make me update these positions.

A chonky week for Western open models!

@corsaren say*

Reflection joins the list of Nvidia & Thinking Machines who have released their strongest models and come up behind Chinese counterparts. There's a lot of ways people will overthink this, but the clearest takeaway should be that the Chinese are very very good at building LLMs.

Congrats to Reflection folks -- it's hard to get the first model out, and we hope you can rapidly accelerate contributions to the ecosystem from here :)

I think pacing the frontier is good as an idea in principle but dead on arrival as something you can actually do. What will we, decide which benchmarks we can improve on? Who is involved? If we stop pushing "capabilities" we'll do a ton of research on swarms + efficiency which will bring on other risks. It distracts from how so much of AI risks is simply coming from diffusing what we have, and we should invest in preparedness, pushing the labs to be more careful, etc. Change my mind?

It's a huge failure of the AI ecosystem that a whole bunch of people respond to something like this with "you're evil" and "shame on you." We'll work to shifting the norms here. There are positive paths forward, even if things will be hard.

Honestly a lot of them were on Substack / more general audience ai followers not ai technical community

Today we're unveiling Trillium Labs @trillium_labs, a new non-profit to foster the open science of frontier AI. We're building open post-training recipes and will expand into open infra to study RSI, reward-hacking, multi-agent systems, and whatever comes next. We're built around the theory of change that you need more eyes to solve hard technical problems. We have faith in the scientific methods and communities that humanity has built, and worry that AI is becoming too closed to utilize them. Trilliums are wildflowers that bloom briefly in the spring, before the forest canopies fill out. Though they are small, they lay the foundation for the cycles of growth and nourishment through the rest of the year. At Trillium Labs, the recipes will be the slow nutrients for the seasons and the model releases will be the blooms. Building an institution dedicated to this is needed because, much as nature’s trilliums are slow to expand and grow, the open-ecosystem needs time and dedicated resources to catch up. I co-founded with with a long-time friend and collaborator Tom Zick (@thesezickbeats). We're hiring (full time + student collabs/interns), we're fundraising, and we're looking for compute. Please get in touch if you're interested in helping out. Offices based in the Bay Area and Cambridge MA, remote okay. I’m in the Bay Area until for The Curve and COLM to connect with people who are interested. We’re thankful to have initial support from Halcyon Futures and Schmidt Sciences with more funding en route to enable our ambitions of scaling. Our advisors @Thom_Wolf, @HannaHajishirzi, @gneubig and @ctnzr have been instrumental to building the ecosystem that exists today, and I’m stoked to get to keep working with them.

@trillium_labs Learn about us in Wired: wired.com/story/trillium… Read our introductory blog post here: blog.trilliumlabs.org/p/introducing-… Our website: trilliumlabs.org If you’re interested in joining us, fill this out: trilliumlabs.org/join-us

I'm not making it on the top poster X lists, but I am a Schmidt Sciences AI2050 early career fellow, continuing my work building and understanding the open ecosystem, so future AI is accessible, diffused widely, controlled by many, and safe. Science first, posting second!

Thanks @schmidtsciences for bringing together a great group. schmidtsciences.org/ai2050-fellows…

Love to see Google surprising people with Gemini 4. More labs at the frontier is wonderful for consumers (competition) and the world (reduction in concentration of power). Excited to see how it goes in real-world scenarios.

“Instead, they manipulated model interactions so that protected reasoning could be reproduced in forms visible to the requester in a coordinated, scaled manner that violated our terms of service” It’s the API company’s problem if their model can be manipulated like this. Add KYC

The latest Interconnects plot - showing the exponential growth of the open model inference economy. Plot is showing the daily tokens processed by the leading companies, based on public disclosures (for Together, Baseten, and Fireworks) and the OpenRouter API. I underestimated OpenRouter's growth.

dashboard.interconnects.ai/extra#inferenc…

Hard to make a Bretton Woods analogy with only 1 country represented.

If X let people using the app tag their replies as human-written, or you could limit replies to people verified as humans, the entire experience would be 10x better. I don't want to have to restrict replies to accounts in my follow network. All the bots are verified too.

Current harnesses have a very rigid setup in how the LLM can manage its own context. We show that you can let LLMs edit the context directly - enabling better long-horizon memory, FLOP efficient adaption, & more. Don't fear compaction anymore!

Sigh. I wrote this post below, but then I didn't post it because I am tired of the consistent pushback I get from folks with the same safety worldview as Anthropic. I guess that means I should send it. Here goes! This post is pretty solipsistic and doesn't properly take recent events in cyber risks into it's discussion. It is really hard for me to watch how the US frontier labs like Anthropic don't have the ability to consider other approaches to safety and ways things could play out. It insinuates that Z ai (Chinese lab, builds GLM series) doesn't really care about safety and is reckless to take their business strategy. Saying things like "Given this evidence, we think it's likely both state and non-state actors will use models like GLM-5.3 to cause real-world harm." and "This is unlike any other similarly capable AI model, all of which were released with safeguards or through limited access programs." while closed models have been used on more of the documented cyber attacks is just bowing out of the interesting question. A plausible view is that open model weights and closed model apis (with some safe guards) are both far closer to being easy to mis-use, rather than API models being closer to safe. The trope "Open Dangerous, Closed Safe" may be closer to "Open Unsafe, Closed Unsafe" Closed models have stronger capabilities and stronger safeguards, but the stronger capabilities part could matter more in net harm if both the safeguards are porous. At the same time, as the authors do acknowledge (thanks - thats progress!), open models without extreme cyber guardrails are important to rapidly diffuse cyber readiness in the economy -- as programs like project glasswing are not perfect in getting all critical industry onboarded. There will be more issues like the time when Fable was released and Amazon found a workaround, which allowed them to access the full capabilities of the model served readily at an API. The blog overall is reasonable in it's narrow line, but it's a very effective tool in a complicated, rapidly evolving media ecosystem to reinforce a certain type of safety thinking.

Woah, huge bullish signal for open models that OpenAI feels a pressure to serve users with these models. It's not a world where OpenAI alone can build all the models it's customers need!

The problem for folks like this isn't that they're dumb or anything, Timnit has all the skills of top scientists, rather the information ecosystem and peer groups they live in that don't encourage interrogating thought. Echo chambers are a slow death of clear thinking.

I think it's a fairly classic problem of scientists, made a lot worse in the era of social media. It's one of my greatest fears.

Also iIf you don’t expect to be wrong sometimes in AI, you’re going to have a hard time making takes and predictions that matter.

It's a very hard time for people in AI who want to push on issues, criticize when due, update beliefs, and think in public. I appreciate the few independent voices out there not falling to straw man arguments, tropes, or clickbait delusions. Please keep going.

(many independent fallen soldiers to $, which is how it goes)

Remember when OpenAI delayed GPT OSS after Kimi K2 released for "safety" (I've had it confirmed to be for kimi multiple times). I feel like this is fine, landing a model is very much about competitive positioning on price, scores, etc. Especially when releases are so frequent.