
@NandoDF
I seek to understand intelligence & agency and build AI aligned with compassion, freedom & universal human empowerment through progress in science & engineering
For the last 10 years we've been raising funds to bring AI to Africa (@DeepIndaba) and South America (@Khipu_AI). The efforts have been very successful, but this year we are finding it hard to raise money. If you work for an AI company, please share the following with the right team to support us. Thank you! About KHIPU 2027 What it is: KHIPU AI (t.co/psxMsgVsGS) is a non-profit organization dedicated to advancing AI talent, research, and community across Latin America. Alongside year-round regional initiatives, we host a biennial in-person gathering rotating across the continent, previously in Uruguay (2019, 2023) and Chile (2025). Next Edition: April 5–9, 2027, in São Paulo, Brazil, hosted at the Universidade de São Paulo (USP), one of Latin America's leading research universities. Watch our 2-minute video: t.co/FgoNN4JD10 (hear directly from past speakers, including Nando, and Latin American students on how KHIPU grew from a single gathering into a regional AI movement). Who Attends & Why Support - Engaging with the region's top talent and lab leaders: We bring together around 500 participants, including leading faculty and lab directors from across Latin America alongside a curated cohort of ~300 graduate students and industry researchers selected from over 1,000 applicants. As the local AI ecosystem grows rapidly, KHIPU offers a rare opportunity to engage directly with these emerging researchers and regional leaders in one place, helping shape research priorities and the safety mindset across Latin America while connecting with talent and perspectives outside Silicon Valley. 100% free attendance & student grants: Attendance is completely free for all selected students, professors, and researchers. Sponsorship funding goes directly toward travel and accommodation grants for students and faculty across Latin America who otherwise could not afford to attend. Community & Track Record: Past editions have featured speakers such as @Yoshua_Bengio , @JeffDean , @jpineau1 , and @NandoDF (2025 lineup: t.co/9KU26hLXZE), and have been supported by the Hopper-Dean Foundation, @GoogleDeepMind , @Apple , @Microsoft , @Meta, @cohere , and @MistralAI . For 2027, @OriolVinyalsML and @sarahookr are already confirmed. 2027 Sponsorship Prospectus: Attached (also available at t.co/EwXyCrGXWl) Contact: @FortunatoMeire (meirefortunato@khipu.ai) & KHIPU Sponsorship Committee (sponsors@khipu.ai)
The email for sponsorship should be sponsorship@khipu.ai Thanks again for helping out.
It is a wonderful thing for AI to become superhuman at mathematical proofs. It will be even more wonderful to see AI taking advantage of such knowledge to advance our understanding of the universe and to solve some of our biggest problems, including healthcare and energy. It was hard for Kasparov with chess, for Lee Sedol with Go, and now for several mathematicians to accept AI’s superiority. I feel sad for many mathematicians and worry about the students. It is hard when the thing you dedicated your life to is taken over by a machine. Not easy to accept in the short term, even if it is a good thing in the end. I spent my life writing AI papers, and that too will be done far more effectively by machines soon. I feel awe, some emptiness, and I rejoice at the fact that it was all meaningful because someone had to create the initial training data, and because we now have more powerful tools to start solving problems that impact everyone. These developments are amazing, energising and exciting, but it is also important to acknowledge that they create anxiety (meaning, finances, social impact) and a lot of uncertainty.
Curt Jaimungal@TOEwithCurtJacob Tsimerman, who won the Fields Medal this year, thinks mathematics will soon rest on proofs no one understands. He says it’s already true of many of his own theorems, which build on accepted results whose proofs he doesn’t know. Full episode: youtu.be/6uIJdXmB4vE
I attended my first talk by Geoffrey Hinton in 1997 at the Royal Society — It was utterly inspiring. He is as inspiring today as he was back then. I highly recommend this thoughtful and funny interview with him. youtu.be/l6ZcFa8pybE?is… via @YouTube
When a movie heartthrob becomes an AI nerd! — The evolution of who does or talks about AI has been far bigger than any of the technical changes. Ben Affleck on AI ... youtu.be/MzXEWrT3Ld8?is… via @YouTube
A wonderful article on AI. ft.com/content/462c30…
ALL AI startups in London united and fought for this, and together with government, we achieved it. It’s only the beginning, we need to go after unfair use of garden-leaves and non-solicits next. Nothing is impossible. We’ll keep fighting for a sovereign UK 🇬🇧 with AI at the service of its citizens. My recommendation to the one and only big AI tech company that is currently enforcing 6 to 12 month garden-leaves by sending baseless threatening letters to British scientists: Change your policy and educate your VPs in London. There’s time to do this in a nice way. You have bigger and nobler goals to achieve than to cause pain to young families in our country by abusing laws that were meant to protect labour. It’s time to de-escalate and to cooperate to build a better future.
Dom Hallas@Dom_HallasNON COMPETES BANNED Today, the PM has announced that his Government will legislate to functionally ban non-compete agreements for startups and scaleups in the UK or as he called it - the Bosman ruling for ruling for the innovation sector' 😂 At the @StartupCltn been working on this for months and we think it’s a huge potential unlock for our tech ecosystem. Here’s what you need to know… - Measures to functionally ban non-competes for startups and scaleups in the UK are going to be set out. - There will also be consultation on a wider set of restrictive practices including gardening leave. - More detail on these plans at the budget in a few weeks (Wednesday 28th October) including a timetable for consultation with a view to legislation in the coming months. We think this is a huge move and so do lots of friends including so many of the great founders that signed the letter with Inherent just the other week. Why? Because talent matters and mobile talent means startups can build. The Valley was built on free-flowing talent. In the UK, we’re now seeing our own versions of this story with companies being built from alumni of both homegrown firms like Revolut and beyond. But we need that to move faster. Hiring and building faster means British startups and scaleups can grow faster. Barriers to that slow our ecosystem down. We won’t pretend it will be perfect, and people (including some of our friends but also lots of fairly vested interests too) will have a right to disagree. But we think that startups will thrive and win in a faster paced Britain. So we are delighted with the progress today - as part of a strong speech by the PM outlining his ambition for building the world’s leading innovation nation including more on tax, growing companies outside London and more. But this is the start, not the end of the process. The announcement launches a consultation. It doesn't mean the issue is settled. How far reform goes, and what it looks like on the ground, is still to be decided. The Prime Minister today has done something that can materially impact the trajectory of the UK tech ecosystem and with it our economy - but only if we make sure it’s delivered. So, the real work starts now.
In red 🟥: Several companies including Apple, Microsoft, Citadel and some smaller companies that are friendly to UK AI startups. In blue 🟦: The one and only AI company using garden leaves as a coercive tool. Garden leaves were intended to protect labour, not to control it.
ALL AI startups in London united and fought for this, and together with government, we achieved it. It’s only the beginning, we need to go after unfair use of garden-leaves and non-solicits next. Nothing is impossible. We’ll keep fighting for a sovereign UK 🇬🇧 with AI at the service of its citizens. My recommendation to the one and only big AI tech company that is currently enforcing 6 to 12 month garden-leaves by sending baseless threatening letters to British scientists: Change your policy and educate your VPs in London. There’s time to do this in a nice way. You have bigger and nobler goals to achieve than to cause pain to young families in our country by abusing laws that were meant to protect labour. It’s time to de-escalate and to cooperate to build a better future. Thank you @sundarpichai
Dom Hallas@Dom_HallasNON COMPETES BANNED Today, the PM has announced that his Government will legislate to functionally ban non-compete agreements for startups and scaleups in the UK or as he called it - the Bosman ruling for ruling for the innovation sector' 😂 At the @StartupCltn been working on this for months and we think it’s a huge potential unlock for our tech ecosystem. Here’s what you need to know… - Measures to functionally ban non-competes for startups and scaleups in the UK are going to be set out. - There will also be consultation on a wider set of restrictive practices including gardening leave. - More detail on these plans at the budget in a few weeks (Wednesday 28th October) including a timetable for consultation with a view to legislation in the coming months. We think this is a huge move and so do lots of friends including so many of the great founders that signed the letter with Inherent just the other week. Why? Because talent matters and mobile talent means startups can build. The Valley was built on free-flowing talent. In the UK, we’re now seeing our own versions of this story with companies being built from alumni of both homegrown firms like Revolut and beyond. But we need that to move faster. Hiring and building faster means British startups and scaleups can grow faster. Barriers to that slow our ecosystem down. We won’t pretend it will be perfect, and people (including some of our friends but also lots of fairly vested interests too) will have a right to disagree. But we think that startups will thrive and win in a faster paced Britain. So we are delighted with the progress today - as part of a strong speech by the PM outlining his ambition for building the world’s leading innovation nation including more on tax, growing companies outside London and more. But this is the start, not the end of the process. The announcement launches a consultation. It doesn't mean the issue is settled. How far reform goes, and what it looks like on the ground, is still to be decided. The Prime Minister today has done something that can materially impact the trajectory of the UK tech ecosystem and with it our economy - but only if we make sure it’s delivered. So, the real work starts now.
Congratulations @im_td @AnjaSurina and @caglarml. Your grandfather would have been very proud Tim 💙
Kaiyue Wen@wen_kaiyue3/ 🏆 Best Paper The Future of Facts: Tracing the Factual Generation-Verification Gap @im_td @AnjaSurina @caglarml openreview.net/forum?id=181wn…
This is the most important thing a UK government has done for AI entrepreneurship in history. Non-competes are gone, and next we'll be tackling irresponsible use of our garden-leave laws and non-solicits. Dear @KanishkaNarayan and @andyburnham: Thank you for standing up for labour in our country. I'm very proud to be 🇬🇧 today!
Kanishka Narayan MP@KanishkaNarayanWe want people to have the freedom to move to a better job and help our most promising businesses grow. Today, the PM announced we will legislate against non-compete clauses, so they no longer hold back workers or stand in the way of startups hiring the talent they need. We’ll also bring forward options to tackle wider restrictive practices, like excessive notice periods. We have to empower the people taking risks to build something, giving them the freedom to start and scale.
It is morally wrong for some of the billion and trillion dollar AI companies that we enabled and helped build over the last two decades to (1) enforce 6 and 12 month garden leaves on the scientists we trained for them, and (2) choose not to fund education efforts like @DeepIndaba and @Khipu_AI
Nando de Freitas@NandoDFThere was a time when @nvidia and @Google weren't into AI. There was a time when the big cloud providers didn't have GPUs. There was a time before GPUs and deep learning, and AI, ... During that time professors and students created the foundation of the AI we have today.
There was a time when @nvidia and @Google weren't into AI. There was a time when the big cloud providers didn't have GPUs. There was a time before GPUs and deep learning, and AI, ... During that time professors and students created the foundation of the AI we have today.
A time before flow matching, LLMs and GANs but when those principles were already being researched, incubated or taught.
This is truly inspirational! We called it neural nets, machine learning, deep learning, AI, SI over the decades, but it was always the same idea: artificial neural nets are remarkable at learning to solve problems from data
OpenAI@OpenAIWe’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results. t.co/7N6TPlft1P
Really proud of the achievements of the MAI speech team I hired when there. Microsoft is once again the leader in speech. I have no doubt they will go far beyond this. I’ll continue rooting for them and MAI. They’re exceptional.
Zalán Borsos@zalanborsosFollowing Transcribe-2, Transcribe-2-Streaming is another great model shipped by our team. x.com/artificialanly…
💯 text diffusion is the future 🚀
varun@varunnealnew blog post! I'm convinced text diffusion is the future, and we'll start to see it everywhere very soon. varunneal.github.io/essays/diffusi…
“The BBC acts in the public interest, serving all audiences through impartial, high-quality, and distinctive output that informs, educates, and entertains”. So dear @BBCPolitics how about you give it a break to reporting AI doom stories, and for once start living by your mission. We need sovereign AI in the UK for many good, noble and existential reasons. The issue is not that the BBC reports on the potential dangers of AI. The issue is it doesn’t talk about its many benefits: better healthcare, improved security, new tools for artists, new job creation, empowerment - everyone can now code and has access to all the general knowledge in the world, advances in STEM, and so much more the public doesn’t know about, precisely because it’s not reported by the BBC. For example, this was huge news yesterday, but the BBC stayed silent. t.co/jP0TcOtGUD t.co/68h2I9Yl0l t.co/sHH4iObUmN
BBC Politics@BBCPoliticsUK needs plan in case of 'unprecedented' AI job losses, says minister bbc.in/4iN13T4
“The BBC acts in the public interest, serving all audiences through impartial, high-quality, and distinctive output that informs, educates, and entertains”. So dear @BBCPolitics how about you give it a break to reporting AI doom stories, and for once start living by your mission. We need sovereign AI in the UK for many good, noble and existential reasons. t.co/jP0TcOtGUD
BBC Politics@BBCPoliticsUK needs plan in case of 'unprecedented' AI job losses, says minister bbc.in/4iN13T4
Frankly 1 year noncompetes and garden leaves in AI are the dumbest affront on British science and engineering EVER since Isaac Newton. 🇬🇧 freetostartandscale.uk
Freedom is the greatest act of love. Then, why are 🇺🇸 Big Tech bosses in London preventing people from being free to start new companies in the UK 🇬🇧 by enforcing long noncompetes, sending threatening letters that bring young families to tears, and coercive 6-12 month garden leaves? We need our freedom back. Our sovereignty depends on this. t.co/jP0TcOtGUD
At our new startup, there are ZERO non-competes and we give each employee the choice of either 1 or 3 months notice periods, upon their request. Competition is for winners. BTW, today is the last day to join the founding team.
Inherent@inherent_labs1/ Today we’re launching an open letter calling on the government to end unfair restrictions that stop Britain’s best switching jobs or starting companies. UK AI companies that have raised $5bn agree: our best should be free to start and scale. 🧵 @downingstreet @biztradegovuk
Freedom is the greatest gift of love. So why are the bosses of California tech companies in London so intent in preventing people from being free by enforcing noncompetes and coercive 6 to 12 month garden leaves? freetostartandscale.uk
These 2012 lectures continue to be relevant today. They cover the principles for learning — for LLMs, image generation, protein folding, materials, etc. Highly recommend them for people wanting to understand what AI is all about. youtu.be/XYEs7kK2efY?is…
This is priceless wisdom. Coding not only brought joy, and created the wealth of Nasdaq. Coding was needed to train the AI coding machines. We all contributed.
Scott@scottsttsMy god this is such a good speech that every SWE needs to hear. You know what? Every person should hear it Keep the happy memories, eyes on the reality, be excited about the future. That’s the best that anyone can do
I always believed in free education. I made a decision to never monetise my popular UBC and Oxford lectures. I’ve lived my life by working to bring knowledge to everyone on Earth: the pioneering lectures, the startups past and new, and my AI research. This desire came as the result of witnessing and experiencing a lot of suffering in my childhood. This course is part of my legacy and I’m very proud of it, and thankful to Oxford, the TAs and my students then.
mentalium@mentalium_My friend works at Anthropic. He said they pay $650,000 a year for people who truly understand deep learning. This exact course by Nando de Freitas is free forever. Most AI courses charge $2,000+ for this foundation. This one is completely free. You get the real technical foundation ➜ taught at Oxford by a professor who later joined DeepMind. Save this ⭣
Highly recommend these opportunities!
Jakob Foerster@j_foerstWe are again recruiting @bold_lab_ai - please share the love 🙏. We are looking for: 1) Postdocs (t.co/jqmfzk0W7H) -- deadline 9th of Oct at noon 2) research assistants (t.co/ittyN0e9NT) -- deadline 9th of Oct at noon 3) Strategic Partnership Project Manager (t.co/oYE6j8D7n5) -- deadline 14th of Oct at noon
This is a fantastic presentation by @CompleteSkeptic — highly recommend it. He is spot on.
Diogo Almeida@CompleteSkepticWe should be developing AI as a tool (1) to empower engineers and scientists to solve the big engineering challenges we face such as energy and the health of our planet (2) to empower more people to code in natural language and ensure the formal code behind the scenes is verifiable, robust, safe and secure, thus creating more jobs (3) to empower people throughout the world with access to general knowledge (4) to empower people with disabilities (5) to empower people suffering with health problems (6) to empower musicians and other artists with tools that enable them to have control and ownership of their data while allowing for them to express themselves in new creative ways (7) to empower historians and other social scientists with their research (8) to empower farmers, and others responsible for our precious food (9) to empower governments and companies with better decision making and planning to deliver better services and welfare … However, agents capable of dangerous tool use, eg autonomous weapons or hacking websites require legal scrutiny. The creators of such agents must be answerable to sovereign laws. Weaponisation of AI is a serious risk for humanity. We need diplomacy. None of the AI goals should ever require that we compromise on safety and human dignity.
I agree with this sentiment. We should be developing AI as a tool (1) to empower engineers and scientists to solve the big engineering challenges we face such as energy and the health of our planet, (2) to empower more people to code in natural language and ensure the formal code behind the scenes is verifiable, robust, safe and secure, thus creating more jobs, (3) to empower people throughout the world with access to general knowledge, (4) to empower people with disabilities, (5) to empower people suffering with health problems, (6) to empower musicians and other artists with tools that enable them to have control and ownership of their data while allowing for them to express themselves in new creative ways, (7) to empower historians and other social scientists with their research, (8) to empower farmers, and others responsible for our precious food, (9) to empower governments and companies with better decision making and planning to deliver better services and welfare, and more. Agents on the web require strong scrutiny. Weaponisation of AI is a huge risk for all humanity. It’s frightening. None of the noble goals listed above should require that we compromise on safety and human dignity and respect. Sovereign responsibility is imperative.
MeidasTouch@MeidasTouch🚨 OBAMA ON AI: "If we are thinking about AI just in terms of how do we cure cancer or get better energy, you can do that without having agentic AI and having it just roaming free in the internet. The reason you are doing that is because you have to market a product that people will pay money for. That’s a misalignment between what our society needs and the commercial imperatives that these companies are facing, not because necessarily they’re trying to do bad things, but because they’ve got to justify these valuations. So, that’s one more reason why it is really important for us to have a competent government and a serious bipartisan conversation around this issue, and we have to do it fast. And I would encourage voters to pay attention to this. If somebody does not have a serious plan for how to deal with this, then they’re not meeting the moment, and you should probably look for somebody else."
Another amazing use of AI to empower people, and not leave anyone behind. deepmind.google/blog/putting-s…
Access to artificial intelligence should be a universal human right.
Concentration of AI in the hands of a few corporations, with a record of reckless agent experiments and militarisation, is our biggest safety risk. Sovereign AI, decentralisation, inclusion, equity, is not only the most compassionate route, it is also the safest for our children. AI empowers people. It empowers the farmers throughout the world who produce our food. It empowers girls throughout the world with less access to education because of gender bias and security. It empowers people to communicate better and deal with conflict in more productive ways. It empowers scientists, lawyers, architects, developers, doctors, nurses, ….. intelligence empowers everyone. With power there is risk, but decentralisation is the best approach we know for managing that risk.
@desirivanova Plus a long overdue consultation on non-competes and garden leaves, which the previous government refused to embrace.
I don’t like to get into politics, but I really like what the UK labour party is doing for responsible AI development in the UK. The conservatives failed. The focus is on improving security, generating jobs, and generating taxable revenue to improve our social systems. The UK has always been a leading power in science and engineering and it’s wonderful if it rises to the moment
BEING RESPONSIBLE 1. Design AI seriously for global safety and security. 2. Make safe AI accessible so everyone can benefit. Design for decentralised AI because it is more robust than the current concentration of power on the hands of a few individuals. 3. Seek international agreements to control AI for defence. I personally am against the current weaponisation of AI. It is dangerous. 4. Incentivise AI efforts that advance science, engineering, clean energy, decarbonisation, healthcare, medicine, safety and security. @periodiclabs @cusp @IsomorphicLabs etc are great examples. There is a huge cost associated with preventing AI tools to help solve some of the greatest problems we face. 5. Let’s not use AI risk hype as pre IPO marketing. It is shameful, hypocritical and irresponsible. We scientists see it. 6. Read Terence Tao’s comments on AI for mathematics - focus on building tools to enhance our understanding.
Others have expressed it well: “If you don’t have a seat at the AI table, you’re on the menu” “If you don’t own the AI model, someone else’s AI model owns you” “There are a lot of things wrong with this world… but too much intelligence is not one of them” Let’s act responsibly: design openly and seriously for global safety and security, focus AI efforts on advancing science, engineering, clean energy, decarbonisation, healthcare and security, and lastly let’s not use AI risk hype as pre IPO marketing.
James Wise@JameswiseIt's good that MPs want to discuss long tail risks, and engage with what is certainly a technology that presents both a huge opportunity and challenge for Britain. But it is wholly irresponsible to say on the one hand 'we think this is really important' and on the other block the UK's ability to build AI capabilities here and support British AI companies. In the event that neither our allies nor foes listen to the proposed international agreement and that in fact a 'kill switch' is impossible, every single one of these MPs should be fighting to strengthen Britain's hand in this critical technology by supporting local datacenters, so we're not exporting our data abroad, reforming regulations so that AI models like cyber defenses can be trained here, so we're not entirely reliant on foreign technology and supporting British AI companies, so we see the economic benefits here too. We're doing our small bit at SovAI, but this should be a national level debate, and I hope this MP champions the UK scaling those ambitions just as passionately.
If you don’t have a seat at the table, you’re on the menu. If you don’t own the model, someone else’s model owns you. I think @MarkJCarney and @j_foerst have said it well. I agree with @Jameswise on this.
James Wise@JameswiseIt's good that MPs want to discuss long tail risks, and engage with what is certainly a technology that presents both a huge opportunity and challenge for Britain. But it is wholly irresponsible to say on the one hand 'we think this is really important' and on the other block the UK's ability to build AI capabilities here and support British AI companies. In the event that neither our allies nor foes listen to the proposed international agreement and that in fact a 'kill switch' is impossible, every single one of these MPs should be fighting to strengthen Britain's hand in this critical technology by supporting local datacenters, so we're not exporting our data abroad, reforming regulations so that AI models like cyber defenses can be trained here, so we're not entirely reliant on foreign technology and supporting British AI companies, so we see the economic benefits here too. We're doing our small bit at SovAI, but this should be a national level debate, and I hope this MP champions the UK scaling those ambitions just as passionately.
Amen
Richard Sutton@RichardSSuttonThere are a lot of things wrong with this world… but too much intelligence is not one of them.
This is a very important message for all enterprises in the world
Jakob Foerster@j_foerstif you don't own your model, the model owns you. x.com/suchenzang/sta…
ENOUGH PESSIMISM IN AI PLEASE I feel that we have become unreasonably pessimistic in our field. 1. I keep hearing AI engineers saying we have to make money quickly because there’s only like 2 years left before we’re automated. Depressing. 2. I see a constant obsession with “having a moat”. This is an incredibly sad mental frame. 3. I keep hearing “we have to catch up”. Soulless. And so on. People: Every solution creates the possibility to attack new real problems. We face gargantuan engineering challenges in our world. How to capture carbon? How to get rid of teflon and plastics in water? How to invent batteries that are at least 30 times more efficient? How to solve clean energy? Better solar cells? Better ways of producing clean energy so we stop wars and famine? How to eradicate hundreds of diseases? Cures for addiction? And so on. Real engineering is about being brave and truly attacking the many problems we face, to engage with a true desire to improve the lives of others and our environment. Good engineering is not about protecting your product to make money at the expense of progress (moat thinking). Good engineering is about ensuring your children and grandchildren will be proud of the choices you made in 30 or 50 years. It is about empowering others. It is about advancing science. It is about being one step ahead. Always, one step ahead, meaningfully, proudly. These are great times. Let’s start thinking positively about all the wonderful things we could achieve together.
This is the wisest and most accurate take on scaling laws — their promise and their costly failures — that you could read today. This is what building AI systems is currently all about. Well said @jietang
jietang@jietangThoughts About Scaling Law Scaling, but not only of parameters. Every model release now ends with the same question: how many parameters? It isn't a question that can be answered on its own. Parameter count is only meaningful alongside three others — how much data you have, where you intend to spend your compute, and who will run the model, under what conditions. The field learned this the hard way. Kaplan et al. (2020) fit an exponent that told everyone to grow parameters faster than data — roughly 2.7:1 — and the industry complied: GPT-3, Gopher, MT-NLG. Hoffmann et al. (2022) redid the experiment across four hundred models and found the compute-optimal split is closer to 20 tokens per parameter, and that with sufficient compute the two should grow at the same rate rather than drifting apart. The error in the earlier fit compounded with every order of magnitude of compute, which is why the largest models of that generation were the most misallocated. The trillion-parameter round was, in retrospect, a detour the whole field took together and then reversed. Chinchilla wasn't the end either. It optimized training compute for models that would be trained once and evaluated. Today a model is called billions of times a day and inference dominates lifetime cost. Put inference into the objective and the optimum moves toward smaller models trained far longer — deliberate over-training, which is what Llama-2-7B and Gemma-2-9B were doing at roughly 290 and 889 tokens per parameter. Sparsity moved the target again. In a MoE model two quantities have to be kept apart: total parameters govern roughly how much the model can hold — knowledge, facts, the long tail — while activated parameters and effective depth govern roughly how far it can think, how many steps of a causal chain it can carry before it comes apart. A dense 20:1 ratio does not transfer. And the ratio isn't a single number at all: Roberts et al. (2025) find the optimal tokens-per-parameter is task-dependent, with memorization favoring more parameters and reasoning favoring more data. Follow-up work on MoE observes that at fixed TPP, pushing total parameters higher actually degrades reasoning, while activating more experts reliably helps it. This matters for what we are building toward. Finding a vulnerability is not a retrieval problem. It doesn't come from having memorized more CVEs; it comes from carrying a twenty-step chain of inference to the end without losing the thread. That capability does not live in total parameter count. Which brings us to this release. Total parameters appear to matter up to a threshold — enough to hold the world — after which additional capability comes from scaling elsewhere: effective depth per forward pass, and above all post-training. GLM-5.3 is our controlled experiment on that claim. Same base, same architecture, same total and activated parameters as GLM-5.2. One month of scaling long-horizon environments and RL. The gains are not marginal. Well, scaling has more than one dial. We turned the post-training one this time because it had the most slack left in it — not because the others are finished. Base model size, pretraining data, compute spent per forward pass: all of them are still on the table, and we will come back to each. What this experiment taught us is that the dials do not have to be turned together, and that the one worth turning next is rarely the one that was worth turning last. We are not done scaling. Next time, maybe mid-training, pre-training, and even more.
I’m super impressed with the fast progress of @inherent_labs. Worth checking out @Orbital_Ind @cusp_ai @wellingmax @ac_edwards_1
Inherent@inherent_labs1/ Today, we introduce Faraday, a 27B-parameter AI Scientist that extends the capabilities of coding agents with a layer of scientific intuition. Trained via long-horizon RL, Faraday outperforms Claude Opus 4.8 and GPT-5.5 on the task of replicating research papers. 🧵
@jamesrichards2 @KanishkaNarayan Instead of arguing here however, how about we meet in person with other founders and VCs and share our views.
What UK AI startups need: @KanishkaNarayan In California, one can start a company the day after leaving Google. My colleagues Jeff Dean, Oriol Vinyals et al made this very clear recently with their impressive speed. In the UK, the same American companies impose 1 year garden leaves on senior AI researchers and 6 months on junior researchers. @GoogleDeepMind for example forced people to sign these contracts at the time of promotion, not the time of hiring. What this means is that researchers cannot start new companies for up to 1 year, cannot easily hire, and in short: they cannot compete. American VCs cannot understand why we move so slow. It is time for UK Gov to do the right thing: Make garden leaves optional for employees. If you’re an AI entrepreneur or VC in the UK or Europe, I would appreciate your comments here. You’ve all confided in me. It is time you let government know the urgency of this — everyone should speak up.
BTW @KanishkaNarayan — you have all our support in bringing this important change into effect. It’s an easy step, but it will transform entrepreneurship in the UK.
To be clear, @GoogleDeepMind enforces these because UK gov currently allows it. However, I think this is terrible for them too. @GoogleDeepMind has been one of the greatest places to do AI research in the UK, full of exceptional people, and a great work life balance - I highly recommend it. However, embracing these noncompete measures creates bureaucracy within the organisation, inefficiency, power trips, and altogether I don’t think this is great for anyone at @GoogleDeepMind. They should focus on doing what they can do well and best: compete.
Excessive weight decay #patagonia
Did anything happen in the world of AI while I was away with the family?
It’s my last day at Microsoft AI. It was a privilege to be part of such a talented team that achieved so much in such a short time. I’m super grateful to the @bing and @Azure teams too. @satyanadella impressed me a lot: A leader that taught us to care not only about Sota, but about delivering value to communities. I believe Microsoft AI is now in a strong place and have the talent and resources to deliver value for their users in many years to come. Thank you, so long, and looking forward to my next AI adventure too, after a precious break with my family in the Argentinian Andes.
Enhorabuena 🇪🇸 El mejor equipo ganó la copa ⚽️ 👏👏👏👏
I know everyone knows this, but what a beautiful visualisation x.com/themathflow/st…
Great point 3 by @RichardSocher —likely many bullshit jobs en.wikipedia.org/wiki/Bullshit_… won’t be impacted by AI either 🤔 x.com/richardsocher/…
What a game 🇦🇷🇦🇷🇦🇷🇦🇷🇦🇷🇦🇷🇦🇷
I loved the analysis in the CANDI (text diffusion) paper. Wonderful to see new works in this direction. However, beyond perplexity, we urgently need to start seeing benchmark results after post-training, and comparisons to autoregressive LLMs. We must measure the performance gap to know whether we’re making progress.
Any predictions for when open source GLM or DeepSeek will match Claude Fable 5? x.com/anthropicai/st…
A model trained for next-token prediction is forced to build compressed representations of latent structure in text. Ilya Sutskever correctly refers to this phenomenon as understanding. Here, a model trained for next-step sensor prediction, with a robot that has proprioception and touch sensors but no vision, is forced to build compressed representations of latent structure in the physical world. The robot becomes aware of the shape of external objects. That is, it understands the physical properties of the external world that enable it to make better next-step sensor predictions. This research was previously done by a diverse team of expert engineers at DeepMind over a month - including stars like @notmisha and @yuvaltassa. Remarkably, this reproduction with a completely different robot took only a few hours to implement using Codex. The automatic creation of physical environments by AI will likely lead to huge advances in areas of science and engineering that use physical simulators or twin models. The paper and notebook are available at ❤️∀ t.co/z2N29JF1FY
The original 2017/18 experiments by @notmisha @brandondamos @serkancabi @sergomezcol @laurent_dinh @yuvaltassa and a few other colleagues. x.com/nandodf/status…