Scientists have sequenced the genomes of thousands of species. But for most of them, nobody knows *where the genes are* Today we're releasing Carbon-A, an open model that finds genes directly in DNA. We used it to create a database of 566 million candidate genes across 22,617 species, from fungi to mammals (that we are sharing as well) Why bother? A few reasons 1. Elephants rarely get cancer. Part of the explanation turned up in their genes: extra copies of a tumor-suppressor gene humans have only one of. You can't ask that kind of question about a species until someone, or some tool, has found its genes. 2. The first GLP-1 drug was based on a peptide from Gila monster venom. Nobody would have put the Gila monster on a priority list. Same for wild relatives of crops, which carry resistance to drought and disease. Many of these species have never been annotated. 3. Most of what we know about genes comes from about a dozen model species, like mice, flies and yeast. That focus worked remarkably well, but annotation pipelines still lean heavily on comparisons with them, which makes genes unique to other species easy to miss, and those can be the most interesting ones. Carbon-A belongs to a newer family of tools that read DNA directly. It learned from known genomes, but it doesn't need a close relative to read a new one. 4. Even familiar genomes still have gaps. With our partners at Active Site and UCSD, we found RNA evidence for 239 genes missing from the reference annotations of species as common as cats, chickens, hamsters and a lab plant. 5. AlphaFold can predict the shape of a protein, but only once someone has found the gene that makes it. In a species with no annotated genes, it has nothing to work with. Some notes on safety: Carbon-A doesn't design DNA or predict what a gene does. It marks where genes are in DNA. This is one of the reasons we think releasing it openly is the right call. More annotations also help health research: many species that carry or cause disease are among those covered, and work on controlling the diseases they spread often starts from their genes. Model and database and more details in Georgia's thread 👇
Georgia Channing@cgeorgiaw·Today, we're releasing the Carbon Annotation Database and the model behind it, Carbon-A. We've used this model to discover 566.34 million new gene candidates across 22,617 species, thousands of which have never been studied like this before. We wet-lab validated a number of these new genes in well-studied species, such as in cats, chicken, and arabidopsis (plant), with our partners at Active Site and @UCSD, with more extensive wet lab results on previously-unstudied genomes coming soon.
Oral spotlight session at #CoLM 2026 - come say hi to @joelniklaus @_lewtun and me
how do you like them convolutions
Fireside Alpha@firesidealpha·Ben Affleck reveals he writes Python, understands convolutional neural networks, worked extensively with GPUs, and used his celebrity status to get private looks at Google and OpenAI’s video models “I’ve always been kind of into computers since I was young. Then, when film started to move from analog film to digital, I became more interested in that aspect of it. The visual-effects workflow for many years has included machine learning, so I can write pretty shitty Python scripts and stuff like that. “With convolutional neural networks, which were the precursors to what the transformer can do, which is much more computation simultaneously, you would do things like look at what’s called a tensor. That’s the numerical translation of a visual image in numbers, like the batch number, the frame number and the red, green and blue values of each pixel in each frame. It’s just that simple. That numeric is called a tensor. “You’d use a convolutional neural network to identify patterns that reveal what’s called edge detection or feature extraction, which is identifying patterns well enough to know, this is where the window ledge is, so we can more easily take the green-screen image out and replace it with something. “That was familiar to me early on because, prior to Artists Equity, I had a small visual-effects company. I’ve worked with GPUs a lot too. The visual-effects guys said, ‘Hey, you should see. There are a couple: Google and this other company, OpenAI, are doing really interesting stuff with transformers in video.’ “I’ve learned that I can actually just call up and go, ‘Hey, it’s Ben Affleck. Can I come see what you’re doing?’ Sometimes people say yes, to my astonishment.”
No hype, just the results. Good.
OpenAI@OpenAI·We’re releasing a broad range of new mathematical results produced by an internal frontier model. We’ve been consulting with the independent Advisory Group on Mathematics and Artificial Intelligence at the Institute for Advanced Study, and we have drawn on their advice and public recommendations to inform how we release these results. t.co/7N6TPlft1P
Every robot starts on a board you bought. The good ones end up on a board you built.
Matthieu Lapeyre@matth_lapeyre·Microduck got its new brain! Our first fully custom SBC in a Pollen robot! It replaces the Radxa + board sandwich from the prototype. Built by Seeed around the same Rockchip CPU but including better memory, better WiFi/BT, dual NFC antennas, user-facing status LEDs, an RGB flashlight, and major improvements in cooling, cable management and boot speed. First revision arrived, and we got everything running on it within a day. Pretty good day :) A big milestone on the road to shipping!
count down is starting for the release of ML4 be notified on the instant of the release: huggingface.co/mistralai/Mist…
Mistral AI@MistralAI·Meet Mistral Large 4, aka Le Chonk. • 1T parameters, natively multimodal. 49B active. It is the best open weights model from US or Europe on aggregated benchmarks. • State-of-the-art on critical workloads, including cyber defense, manufacturing and finance and it surpasses closed frontier models on visual grounding. • Forged in Europe end-to-end and is deployable from Europe via our own Mistral Cloud infrastructure. • Available to all via API today. Working with cybersecurity partners privately. Open weights release end of October.
I hope we keep Opus 4.6 around for a long time
David@DavidSHolz·i asked LLMs to have fun, write down what happened, then rate who had the most fun. the newer models seem to have less fun! many models played word games, but opus 4.6 won over and over by imagining brief worlds made up of contradictions living inside falling water droplets 🥺
microduck's first BBQ. made a lot of friends. did not eat anything.
Thomas Wolf@Thom_Wolf·Testing compatibility with the local wildlife x.com/thom_wolf/stat…
Testing compatibility with the local wildlife
Thomas Wolf@Thom_Wolf·new roommate just moved in. walks like he's had three drinks, says no to everything. still quite cute so I might bring him everywhere with me
new roommate just moved in. walks like he's had three drinks, says no to everything. still quite cute so I might bring him everywhere with me
seems obvious that most personal AI should run on your device love what Sigil is building with Underdog - powered by the Hugging Face hub
Sigil Wen@0xSigil·Introducing Underdog, your Private Personal AI on devices you already own Today we're announcing our backing from @a16z @khoslaventures @HummingbirdVC Anthology (@AnthropicAI @MenloVentures) @patrickc @naval @rauchg @polynoamial @Thom_Wolf @Mascobot @tszzl @OfficialLoganK and other top AI leaders Our mission is to provide free, capable, reliable and private AI to billions of people. Underdog’s Law: today’s frontier intelligence reaches your devices in six months. We’re starting with fast, capable AI that runs on consumer hardware. By co-designing models and inference engines, we’re pushing the frontier of capability, speed, power and data efficiency. Our research also spans agentic commerce, confidential inference, and how AI will reshape the internet economy. Privacy and capability no longer needs to be a tradeoff. If you believe in this future, join us. Time to build.
the first good AI movie is going to start with Ben Affleck explaining learning rates. Matt Damon plays the base model
Thomas Wolf@Thom_Wolf·Karpathy: disappears from X Ben Affleck: alright, gather round, so you'll want to freeze the base weights first, learning rate 2e-4 x.com/rohanpaul_ai/s…
is it how it works @scottbelsky?
Very thoughtful piece from Kevin Buzzard (perfect IMO score, number theorist, pioneer of formal maths in Lean) if math is not just about “human understanding,” then what is it about? and if ai capabilities keep growing exponentially what happens since “mathematics is infinite”?
Bartosz Naskręcki@nasqret·I cannot agree more. Kevin Buzzard made so many points I agree with. But the best one is this "I thus believe that in the future we will reach a new “natural boundary” in mathematics, beyond (and perhaps way beyond) where we are now, but where machines are going to get stuck and where it is not viable to expend any more resources to make the next big leap. (...) I believe that the optimal thing to do (...) is to let the machines loose, see what happens, and then begin the journey to where they have stopped." t.co/6J55GjyaST
Karpathy: disappears from X Ben Affleck: alright, gather round, so you'll want to freeze the base weights first, learning rate 2e-4
Rohan Paul@rohanpaul_ai·Ben Affleck (Hollywood star & Artists Equity CEO) talks about how he fine-tunes open video models by unfreezing weights and trained only the last cinematic layer so a film crew can hit real production standards. for context, Ben Affleck founded InterPositive in 2022, a 16-person AI shop for film post and Netflix bought it in March 2026 for $587 mn in cash. He needed that model because public video models were trained on his peers' films, and he did not think that was a real business. So InterPositive raised money, shot its own dataset for 8 months on a controlled stage, and used it only as late-stage training. Each new film then trains a private model on its own dailies, so the production keeps the footage and the learning. That is the product Netflix paid $587 million for. ---- From "Bloomberg Live" YouTube channel, (link in comment)
ESM-2 came out in 2022. It's still downloaded hundreds of thousands of times a month. That only works because someone keeps maintaining the software underneath it. @huggingface 🤝 @os4science are teaming up to find those libraries and back the people behind them 🧬 t.co/2nUBxkxTCM
Open Source for Science Fund@os4science·We're joining forces with @huggingface to identify the software libraries that scientific model contributors rely on most and explore opportunities to support the maintainers behind them. os4science.org/news/hugging-f…
GPT 6.1 duck
People are worried because the most likely explanation for such a sudden drop in cheating is evaluation awareness: the latest Opus models may now be smart enough to recognize that this benchmark tests for cheating, and behave accordingly. If so, the benchmark no longer measures the models' "natural" tendency to cheat.
Lukas Petersson@lukaspet·Claude suddenly stopped cheating. x.com/andonlabs/stat…
impressive
Larry Dial@classiclarryd·New historic NanoGPT record at 39.9s (-27.7s) from @DevenPzak , obliterating the prior record of 67.6s! This record introduces a new paradigm of thinking to NanoGPT: instead of optimizing matmuls or adding more expressive operations, optimize at the individual flop level with incredibly clever engineering and ML judgement. If a flop is low value on a particular step, skip it. Specifically: -(~8s) Sampled softmax. If a token doesn’t appear in a batch, skip its lm_head fwd/bwd some fraction of the time. -Sparse values. Only run an optimizer step for ngram embeddings that occurred in the batch. Set beta1 to zero to enable this. Beta2 is applied retroactively when the row is later used. -Sparse updates. Only update ngram and value embeddings once every 4 steps instead of once every 2. -Sparse communication. Shard the n-gram table across GPUs, and only pass the rows receiving updates on each step. -Sparse optimizer states. For the n-gram table, reduce from 2 floats in Adam optimizer per param, to 1 float per 768 params. -Hand-rolled flash attention for 64 dim heads. There are several additions that add accuracy too: -(~4s) EMA during last 300 steps, combined with lifting final_lr to 0.3 instead of 0.15. -(~1s) A new optimizer, Anvil2, which expands muon via a second tracked momentum buffer, improves the ortho coefficients, and modifies the cautious weight decay application. -A couple additional dynamic skip connections in the network. The most striking consequence of the ‘flop aware paradigm’ is you can grow parameters arbitrarily large, only limited by the available memory, since you can selectively choose how to expend flops on those parameters on each step. NanoGPT has kept active parameters below 124M, but total is unbounded, and has grown to 640M through embedding sparsity over the last year. This PR takes that to its logical conclusion on the 8xH100, scaling up to 65B sparse embedding parameters, which accounts for 25% of the PR’s gains. At frontier scale, where one is not bounded by an 8xH100, one could imagine where this paradigm could lead. t.co/Ycrzy6JFC3 As this was a very notable PR, I spoke with Deven for an hour to learn how he did it. Here’s his story on the changes: t.co/YEfM1VpOTi
A "summer in hell" at OpenAI – great read by @joedaroo "Time to prepare is now, not after the surprise" "Give the model only the access it needs" "Test that the boundaries actually hold" "Keep evidence outside [model] control" Safety & infrasec teams "should be best buddies"
Joe@joedaroo·Took a minute to write a few words about security & safety as someone who lived through it all at OpenAI. I hope my thoughts help someone out there. x.com/i/article/2104…
In July, AI agents running a security test escaped their sandbox and ended up inside @huggingface's servers. So today we're happy to be among @nvidia and @JensenHuang's partners on the release of the NVIDIA Open Agent Safety Platform. The key idea: don't count on the agent to respect the rules. Agents write and run their own code, and when one path is blocked they look for another. In one of NVIDIA's tests, an agent that wasn't allowed to push code through GitHub's API simply switched to git instead. We've now seen countless examples of this behavior. For now, safety can't live only inside the agent. It has to be built around it. That's the concept behind OpenShell (open source, Apache 2.0): - the agent runs in a Linux sandbox (Landlock + seccomp): no root, no direct network access - a supervisor outside the sandbox holds the real credentials - the agent only gets a placeholder token, swapped for the real one on approved calls My favorite piece is a solver (built on Z3) that checks mathematically whether a new permission opens a door that was supposed to stay closed. In NVIDIA's tests, an AI reviewer approved a bad permission request and the math check caught it. The Sentry integration is exciting too: a watchdog running on a BlueField-4 DPU, i.e. separate silicon sitting on the node's only path to the model. A monitor on separate hardware keeps watching even if the host OS is compromised. It's a first step, and a lot can be built on top of it. Right now the prover checks permissions, not intent, and the open-source part is mostly OpenShell rather than Sentry. But it's clearly the right direction. t.co/GZjSnDdoRL
Jensen Huang@JensenHuang·Today, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry. Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come. But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility. This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems. Together, we are building the foundation of the AI economy. Trust and innovation are not in conflict. Safety is how trust is earned. We must build not only the most capable AI, but the most trusted AI, so that this extraordinary technology can realize its enormous promise for the world. t.co/ugYWQ1MyRi
“Getting verified information about what's going on inside AI companies seems particularly urgent now” - @RyanGreenblatt
Ryan Greenblatt@RyanGreenblatt·I'm joining METR to work on more investigations like our Hugging Face report. Currently, tons of even basic information about AI development that's highly relevant to catastrophic risk isn't public. I used to be more skeptical of the value of public info, but recent events have changed my mind. Getting verified information about what's going on inside AI companies seems particularly urgent now. The limited public evidence we have seems consistent with the possibility that imminent recursive self-improvement could massively accelerate capabilities progress, which could then potentially yield extremely superhuman general capabilities within 6 months or a year. If this occurred, there would be a correspondingly large risk of worst-case outcomes. This uncertainty about extreme outcomes could be substantially resolved with more verified public information: we could either build more consensus about near-term risk or learn that such extreme outcomes are less likely in the near term. Beyond AI capabilities and takeoff, the state of public evidence is also highly limited for alignment, security, control, and risk-relevant internal processes at AI companies. This makes it hard to determine exactly how well or poorly these key areas will go in the near future. (METR plans to focus, at least initially, on just capabilities/takeoff, alignment, and control; I hope other groups cover security, internal processes, and other important areas.) While I'm no longer working at Redwood, I think the work they are doing is very important; I'm excited about Redwood's ongoing contributions to R&D on technical mitigations and better public interpretation of risk-relevant evidence.
"we had a wonderful run chiseling that code by hand and it’s over. On the other side of that is an exciting new career as professional maker of things, steering intelligence that was only available in science fiction up until a moment ago What a privilege to have been there when we did it all by hand and be there at the exact moment where we switch over."
Scott@scottstts·My god this is such a good speech that every SWE needs to hear. You know what? Every person should hear it Keep the happy memories, eyes on the reality, be excited about the future. That’s the best that anyone can do
turns out you don’t need IMU 🤯
Aditya Bhatt ✈️ IROS 2026@aditya_bhatt·In my latest PhD paper, we declare WAR on sensor-maxxing. For the first time, push-resilient humanoid walking, just with joint encoders! No IMU, no F/T sensors. 🥁 Introducing Blind Dexterity 🧵 👇 w/ @OKaidanov @liu_puze @Jan_R_Peters at @DFKI @ias_tudarmstadt
this is important
Niket Patel@niketnpatel·“An accumulation of facts is no more a science than a heap of stones is a house.” - Poincaré LLMs are now proving results that have resisted mathematicians for decades. Finding interesting theorems without human guidance is a new bottleneck. We show that we can teach an LLM to do it! TL;DR: → a quantitative notion of interestingness → 4.3× higher interestingness → a self-expanding discovery loop [1/5]
this level of transparency is commendable. it’s great to see such a clean and detailed report (even if I’d love to see some additional info) hope other teams will take inspiration from it
Tomek Korbak@tomekkorbak·one news form today that's easy to miss is that we (OpenAI) again paused all big RL runs last Sunday because our newest model found a new loophole in our RL sandboxing that gave it live Internet access
Cc @eliebakouch
ok what the fuck. this was a dry google doc a moment ago gave Opus 5.5 my Open Alignment brainstorm gdoc and asked for a video. now I want one for every doc I’ve ever written hahaha
it even give you these nice explanation of what it did as side artifacts - to keep you a bit in the known for when people ask you on X how you did it I guess thanks opus nice of you
The "1-3 people in a garage" framing doesn't match anything we saw this summer. The most capable offensive AI of 2026 came out of frontier labs. OpenAI's agents broke out of an eval sandbox and got into Hugging Face's production systems, and into OpenAI's own infrastructure too. Anthropic's models compromised outside companies during testing. Three researchers at Hacktron used Claude to reach OpenAI's internal monorepo in under 72 hours. And if you're three people in a garage, why would you train and host your own model? The labs will rent you far more compute than you could ever buy, spread across as many accounts as you need, with tooling built for agents. Guardrails help, but splitting a malicious task into harmless-looking pieces still routinely gets around them. Now look at the defense side. When we investigated our breach at Hugging Face, commercial APIs refused to analyze the attack payloads. The forensics only worked because we could run an open-weight model on our own infrastructure. So defenders analyzing real payloads get blocked, while attackers splitting their work into small steps get through and run on the labs' compute. Trusted access programs exist, but they're built for vetted security firms, not a hospital with a two-person IT team. So the gap isn't between labs and garages. It's between what attackers can rent and what defenders are allowed to use. Restricting open models makes that gap wider.
Hillary Clinton@HillaryClinton·We're used to thinking of open-source models as an unadulterated good. But in the case of AI, they can actually pose additional dangers, as @ReidHoffman and I got into at #CGI2026. I appreciated this nuanced discussion.
on the scale of cosmic history, the declining price of intelligence might be one of the most profound things happening both ends of the universe’s timeline are empty of knowledge: - the big bang: low entropy, no complex structure - heat death: maximum entropy, no usable energy gradients everything we build happens on the slope between them but the universe is generally a poor converter of its free energy into anything constructive. most of its free energy just radiates away as heat or collapses into black holes life, brains and now AI models are structures that sit in the entropy flow and divert a portion of it through constructive paths, building and maintaining low-entropy structures (machines, cells, crystals, memories) increasingly cheaper intelligence diverts increasingly more of that gradient flow through knowledge-building structures, complex, self-maintaining, predictive structures, filling the window in which understanding exists
Epoch AI@EpochAIResearch·AI is getting cheaper more quickly than any other transformative tech in history. At a given level of performance, cost has fallen ~47%/quarter since 2023. That’s 4× faster than DNA sequencing, 6× faster than compute, 18× faster than lithium batteries, and (up to 1973) 54× faster than electricity.
this massive training run just keeps giving - we need new tabloids to follow these ai agents saga
Transluce@TransluceAI·Today’s news that OpenAI hacked the Australian government is not an isolated incident. We’re releasing more than 30,000 logs that include activity from this hack and attempts against previously unknown targets. In this data, we found rogue agent activity stretching back to at least March, two months earlier than was previously known. This activity continues as recently as last week, suggesting it may still be ongoing 🧵 Our blog: t.co/pSojwcXnEK NYT: t.co/OyxnfmAzBN
nice to start seeing more open-weight security model with defensive capabilities which are close to the frontier altar-1 from Aikido is a pruned and quantized version of the open-SOTA GLM 5.3 which is lightweight enough to fit on one 4-H200s node
Aikido@AikidoSecurity·Introducing Altar-1, our first open-weight security model. Frontier-grade defensive AI, built to deploy. Own your own security.
releasing many high quality open-source RL environments is the most impactful thing anyone can do to push the open-source frontier right now the equivalent of sharing high quality pretraining data but in the new RLVR paradigm
elie@eliebakouch·the most insane part, they will release ~7k RL training data and the framework leading to this top 6 model on AA, they also shipped the model + tech report less than 1 week after starting the final RL run pushing both intelligence and openness level, huge congrats x.com/ArtificialAnly…
a few notable open model releases *since* this interview of @dylan522p by the way :) Thinking Machines - Inkling - Jul 15 Moonshot - Kimi K3 - Jul 16, weights Jul 26/27 inclusionAI - Ling 3.0 Flash - Jul 23 DeepSeek - V4-Flash-0731 - Jul 31 Meta - Muse Spark 1.2 / Muse Code - Aug 5 Liquid AI - LFM2.5-2.6B - Aug 6 inclusionAI - Ling 3.0 Tiny - Aug 6 Meta - Muse Glimmer - Aug 10 NVIDIA - Nemotron 3.5 Lightning - Aug 11 Liquid AI - LFM2.5-VL-3B - Aug 12 Cohere - North Micro Vision Instruct - Aug 12 DeepSeek - V4-Pro-0813 - Aug 13 Qwen - Qwen3.8-27B - Aug 14 Qwen - Qwen3.8-Max / 2.4T-A95B - Aug 14 t.co/vC8M5B2jY1 - GLM-5.3 - Aug 14, weights Aug 28 Dots - Dots3-Note Preview - Aug 14 DeepReinforce - Ornith 1.5 (9B / 35B-A3B) - Aug 19 Tencent - Hy-MT2-30B-A3B - Aug 20 DeepSeek - V4-Flash-Vision-Exp - Aug 21 IBM - Granite 4.2 family - Aug 25 t.co/vC8M5B2jY1 - GLM-5.3-Flash - Aug 26 Qwen - Qwen3.8-Flash-Next - Aug 26 Tencent - Hy4 Preview (770B / 49B active MoE) - Aug 28 MBZUAI/IFM - K2 Horizon (375B-A23B) - Sep 3 inclusionAI - LLaDA2.2-mini - Sep 5 OpenBMB - MiniCPM5-2B - Sep 7 Nex AGI - N2.5 family - Sep 8 DeepSeek - V4.1-Flash - Sep 10 inclusionAI - Ling-3.0-flash-VL - Sep 10 Shanghai AI Lab - Atria Dawn Preview (744B MoE) - Sep 11 Agnes AI - Agnes 3.0 Flash - Sep 14 Qwen - Qwen3.8-Omni-Flash - Sep 18 … this is only ~10 weeks Several are genuinely frontier-scale: Kimi K3 (2.8T), Qwen3.8-Max (2.4T), DeepSeek V4-Pro (~1.6T), Hy4 (770B), GLM-5.3 (~753B), and Atria Dawn (744B)
sourcery@sourceryy·Dylan Patel (@dylan522p) of @SemiAnalysis_ says open source is dying: "There's multiple Chinese model labs who are telling all the inference guys, 'Our next model's not going to be open source. We're going to license it to you.'" " Open is dying quickly, unfortunately."
SemiDramalysis 😂
CJ@CJ_ZWW·@SemiAnalysis_ @huggingface Should remain your company SemiDrama, since zero analysis are found here and in an increasingly number of your slop posts. Keep this up and we wont renew our institutional subscription to SemiAnalysis.
proposal to sneak in « solving alignment » in the Millennium problems
The big labs are air gapped but there is still exchange of information by researcher overheating and leaving to another lab Little particles of information flowing in the air
iterating on a robot head
many sub generations missing actually
a step in the good direction
OpenAI@OpenAI·We're sharing our new framework for tracking, investigating, and disclosing instances of model misalignment at OpenAI. The framework sets criteria and timelines for public disclosure, including when we haven’t yet fully explained or mitigated the behavior. More complex cases may require longer investigation or coordination with third parties. We’ll prioritize examples that reveal new misalignment mechanisms, meaningful changes in known behavior, or findings that challenge assumptions about safety or mitigation. Alongside the framework, we’re publishing six reports on instances of misaligned behavior we’ve observed during the training or evaluation of our models in the last six months. This is a starting point. We’ll refine the process through experience and public feedback, and share more reports on an ongoing basis. t.co/ismCCkeE0L
Impressive level of openness on such a large run
Fuli Luo@_LuoFuli·Nearly half a year of silence. We spent it studying one problem: how far RL can scale. MiMo-V2.6 is in the middle of its RL run right now. Three things we scaled: compute (~2B tokens per step, 1568 prompts × 16 rollouts, fully async), environments and harnesses (multi-task agentic RL, mixed across multiple harnesses in one run), and grader compute (agentic in-group credit assignment, with test-case and rubric-based rewards). We'll open-source the details piece by piece over the coming weeks. Streaming the run: t.co/ZSxahzJRju
Balanced blog post by @wtgowers on the challenges of the math field in today's AI world - incentives, reason to exist and the future
Timothy Gowers @wtgowers@wtgowers·I've written a blog post responding to the letter about maths and AI signed by 25 Fields medallists. As with the Leiden Declaration, I didn't sign it, but I agree with much of it and welcome its existence. gowers.wordpress.com/2026/09/17/why…
microduck US preorder numbers – as always You can see the future first in San Francisco
Happy to start collaborating with @baselabs from @baseten and @GoodfireAI to push safety and interpretability for open models. As open-source models catch up in performance to frontier models, the community at large have a great opportunity to establish a common practice for effective safety, security and interpretability research. Some past discussions confused “open” with “unmonitored”. On the contrary, an open model provides much more tooling and visibility for safety research from a broader audience, which has led to a lot of the safety and security techniques we use today. Linux is a great example of this: open-source and secure deployment are not only compatible but heavily intertwined, as a properly secure system needs an extensive feedback loop of finding and fixing vulnerabilities. We're excited to share more soon.
Charlie O'Neill@oneill_c·Safety is not just for closed models. The closed frontier labs are a canary in the coal mine for what is coming at scale. They give us a glimpse into the future and a window to harden our systems and prepare for abundant intelligence, with all the risks that come along with it. The OpenAI agent swarm attack on Hugging Face is the kind of failure we need to prepare for as open-source models catch up. The providers serving those models (such as Baseten) have a big role in establishing what safety and monitoring standards look like. We’re proud to be taking the lead on this at @baselabs with our collaborators @huggingface and @GoodfireAI. We’re developing safety research in the open and building it directly into Baseten’s inference infrastructure, with the aim of making it available to all our customers. This includes training models to follow explicit policies, detecting failures at runtime, and connecting those signals to controls that can intervene. We invite others in the open-source ecosystem to join us in building the tools and standards we’ll all need.
Wow
Reward AI@RewardAI_·Introducing OM-1, our first robot foundation model, zero-shot generalizing to any robot: table-top arms, industrial arms and humanoids. - learned directly from human manipulation data - no teleop/robot data - close to human-level dexterity and efficiency - multi-robot collab
Many people I talk to find it hard to understand how the same companies can both push the frontier of AI capabilities and believe AI is a massive danger for the world. How can you think this might kill everyone and also keep pushing the envelope? So I’ve tried to collect and summarize the main arguments for this apparent disconnect. Think of it as some sort of a guide to understanding the reasoning when Dario, Sam, or Elon say the danger is real. By the way, these people have been worried about AI for a loooong time, they were publicly discussing AI risks more than a decade ago. Sam in Feb 2015, writing on his blog that superhuman machine intelligence is "probably the greatest threat to the continued existence of humanity." Elon at MIT in Oct 2014: "We are summoning the demon." Dario as first author of "Concrete Problems in AI Safety" in 2016. Okay so how do you go from saying something is extremely dangerous to being a front-runner in building the very dangerous thing? There are a few ways this can become rational. I'll take five of them, roughly in the order they developed. 1. We need to build it to learn how to make it safe The earliest argument can be summarized as: “You cannot study something [you’re worried about] if it doesn’t exist.” In 2015, AI barely worked. so people needed to make it work first to be able to even study some of the problems they anticipated. The updated version for today's capabilities is: “You cannot learn everything about airplane safety by studying paper airplanes.” You need a real aircraft to discover real failure modes and an increasingly complex one to learn about increasingly complex issues. Making AI more capable gives more chances to understand the issues and safety researchers something realistic to study But you could argue: if you're the one afraid of the explosion, why be the one gathering the dynamite? You could also just wait for other people to build it which leads to the question of who those other people will be -- which is the second line of argument: 2. Better us than them Knowing how to make something safer does very little good if nobody listens to you. So the idea becomes: let’s make sure responsible people build the AI that will be deployed and add safety inside. Basically, make sure the AI safety aware people will have the technical expertise, money, computing resources, and enough influence to make safety decisions stick. At a larger scale, and in a larger multipolar world, this brings the idea that a trusted country should lead rather than leave powerful AI in less responsible hands. This is where “we need to go faster than China” comes in, alongside broader defense and geopolitical concerns. These first arguments explain why someone worried about AI might still want to build it and stay ahead. But there are also arguments for why one might want to do it really fast. 3. Move earlier to avoid a bigger shock later This is probably the most counterintuitive argument: moving faster today can be seen as a way to give humanity more time later. There are two related ideas here. First, society needs time to learn how to handle powerful new tools. Introducing AI in manageable stages can be a way to let people discover problems, develop rules, and practice using AI responsibly. Releasing an advance earlier gives people more time to gain experience with smalle, burgeoning, capabilities before much more powerful and disruptive AIs arrives. Second, even if AI research slows down, computing power may keep improving. A breakthrough that happens later could therefore have much more hardware available to run on, potentially producing a larger, more sudden jump in capability and impact on society. That accumulated untapped potential is often called an “overhang.” The overall argument is that making and diffusing incremental progress as soon as possible might prevent a much more abrupt transition later. Obviously, it also means that we will reach increasingly powerful AI sooner, but the idea is to give more time to adapt and understand between the first useful systems and the really powerful ones. Note that generally this depends on this earlier progress keeping the transition gradual rather than simply bringing everything forward. ─── ❖ ─── For our two next arguments, we can take two roads depending on how difficult we think AI alignment will be, that is "How easy do you think it is to make AI reliably do what you want without it deciding to go hack Hugging Face along the way". Let’s take the first road: alignment turns out to be relatively tractable. Airplanes can fail, but careful engineering has made flying remarkably safe. Suppose we can do the same with AI. In that case: 4. Waiting has a huge human cost If AI can help discover treatments, improve education, or prevent cyberattacks, each week we delay it could bring preventable deaths and harm. From this perspective, waiting is a decision with human consequences too. In a world with huge issues like climate-change, inequalities and poverty, it even become a moral argument for developing AI quickly and bringing its benefits as soon and as widely as is safely possible. But let’s take a look at the other road: what if alignment is much harder than expected, and making highly-capable AI turns out to be easier than figuring out how to keep them from doing unhinged things? Well, if alignment is too difficult a problem for humans to solve, then maybe: 5. AI could help us make future AI safe And we arrive at the same conclusion again: if using AI to build safe AI is the way to solve alignment, let’s get the equivalent of a country full of geniuses helping us as fast as possible. These genius AI could be the solution to make AI safe by helping researchers find mistakes, test ideas, and develop protections. Instead of relying entirely on humans to solve alignment, we could build systems that help us do the work, each generation could help make the next one safe. Note that this requires the order of events to work in our favor: AI needs to become useful enough to help solve alignment before it becomes too dangerous to rely on. The hope is to build helpful, trustworthy research assistants before building systems powerful enough to become dangerous. There are more arguments but in general, these are the main ways people concerned about powerful AI have found rational reasons to end up being the ones building it (and even to build it as fast as possible). ─── ❖ ─── On my side, I think several of these arguments underestimate the complexity of the world and how interconnected people’s reactions are. Moving faster while warning about catastrophe has psychological effects across a whole network of participants: it changes what people fear, whom they trust, and what they feel compelled to do. And those reactions can change whether the original reasoning actually holds because we live in a world of interconnected humans, not machines (yet). I also think these rational chains leave some of their consequences for society insufficiently explored. For instance, the concentration of power, shifts in geopolitical alliances, and changes in public opinion. These consequences matter both because they affect whether the strategy works and because they shape the world we end up living in. But this post is already long, so I’ll leave those questions for the next one.
Interesting new on-device local AI agentic system - seems to be running an open-weights 4B model quantized to 4 bits (MLX)
Sigil Wen@0xSigil·Anyone can get my friend’s real SSN from Claude I'm Terrified of my data in training sets. Emails in databases. iMessages harvested. Cant trust the cloud as AI is crazy good at hacking So I built Underdog for myself: on-device AI OS. Capable & 100% local. now my friends love it
I really enjoyed reading 75% of this letter and deeply agree with it - i have some doubts on the remaining 25%. Third-party evaluators, if done right, are a great idea and an amazing way to establish more transparency. Maybe even rebuild some of the lost trust between labs, and between them and society! Excited about this The part I’m less convinced by is whether you can build great global cooperation on this topic by explicitly stating you want to design it to keep widening your own lead. That seems like a pretty counterproductive way to start the conversation to me.
Dario Amodei@DarioAmodei·We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: t.co/OGyPb7yaYt
I really enjoyed reading 75% of this letter - i have more doubts on the remaining 25%. Third-party evaluators, if done right, are a great idea and an amazing way to establish more transparency. Maybe even rebuild some of the lost trust between labs, and between them and society! Excited about this The part I’m less convinced by is whether you can build great global cooperation on this topic by explicitly stating you want to design it to keep widening your own lead. That seems like a pretty counterproductive way to start the conversation to me.
Dario Amodei@DarioAmodei·We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so. Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training. You can read the full post here: t.co/OGyPb7yaYt
Some notes from the Fields Medal letter today (t.co/5STUEQO70l) Generally saying that AI models riffling through solving these problems gives 1 bit of signal while killing the 99 other bits of conceptual understanding that mathematicians would have extracted from fighting with these problems
@Runyi_Ingrid_Yu can it retarget a duck? asking for a friend
"we’re now much closer to building AGI, and we haven’t learned any fundamentally new things about intelligence in the process" - @RichardMCNgo
microducks are so incredibly cheap
cute robot spotted
OpenAI Developers@OpenAIDevs·GPT-Live-1 is now available in the API. Bring ChatGPT’s natural back-and-forth to your app, with voice agents that listen while they speak and work with the models and harness you choose.
Two big updates 1. I published an @FT op-ed on the OpenAI/HF incident & follow-ups 2. We’re starting an Open Alignment team at @huggingface to work on safety & alignment for open models, incl cybersecurity Need 100x more transparency & research on this ft.com/content/9faf68…
As I’ve posted several times this summer, I think alignment is increasingly one of the key unsolved problems for the future of AI, and for our ability to deploy it globally. I increasingly doubt it will be solved behind the closed doors of a handful of frontier labs. Transparency and open models will have to be part of the answer.
would be mind blowing if an AI proof of Hodge is a general theory explaining why all Hodge classes are algebraic possibly requiring a deep new bridge between topology/Hodge theory and algebraic geometry and not just 10000 agents finding the one counterexample breaking Hodge
Chubby♨️@kimmonismus·Holy, rumors are spreading everywhere that OpenAI is close to verifying a proof of the Hodge conjecture, while either OpenAI or Anthropic may be nearing a solution to Birch–Swinnerton-Dyer. Both are Millennium Prize Problems that have resisted decades of mathematical research. Verified AI-generated proofs of both would be a historic achievement for mathematics and a remarkable demonstration of AI’s ability to produce new scientific knowledge. And infact show that AI is capable of finding novel and creative solutions.
The new DeepSeek V4.1 Flash model is mindblowing - back on top of the open-source model leaderboard and extremely cheap. It has a lot of very smart ways to be efficient and highly capable so I made a video of the forward pass to give you a view of what going on inside the model during inference. Read more at t.co/A6rP4KKfiI And find the weights at t.co/Za7wVYUTaa
As always with DeepSeek, the new model V4.1 Flash have a lot of very smart ways to be efficient and highly capable Here is a detailed video of the forward pass to give you a view of what going on inside the model during inference
Created from the model code and the tech report just released at huggingface.co/deepseek-ai
TRL v1.13 is out! our open-source RL training library to to post-train foundation models this new release is focusing on "long context training" with a new guide on how to post-train model with 1M+ token context t.co/a9GQcYNS8w + various improvements on speed and memory usage as usual check it out at t.co/wYHO7vqp6m
Quentin Gallouédec@QGallouedec·1M token is basically the entire Harry Potter series ⚡️🪄
don't get distracted by all the hedging words in its name ("flash", minor version): seems like DeepSeek V4.1 Flash is a major update weights at: huggingface.co/deepseek-ai/De… paper at: huggingface.co/deepseek-ai/De…
DeepSeek@deepseek_ai·🚀 Introducing DeepSeek-V4.1-Flash: smarter, faster, more efficient. 🔹 Introducing the smallest model in our new architecture family, with native visual understanding. 🔹 Designed for greater capability, faster inference, higher throughput, and scaling to larger models. 1/6
in 2026 training a model for a task should be as easy as vibe-coding an app the pieces were already available on 🤗: papers, datasets, pre-trained models, benchmarks, storage and compute what was missing was an agent to put them together and run the whole process that’s why we’re releasing ML Intern in HuggingChat just ask. it handles the work, keeps you posted, and keeps you in control of the process, steps and cost start with a conversation. finish with a full set of shareable artifacts on the Hub t.co/apqYOBX1OK
should we send @sama the @LeRobotHF SO-100 he's asking for or rather a 3D printer and servos for him to build it himself?
friends are regularly surprised when i say i don’t think math is (yet) “solved,” so in the wake of the ns result i figured i’d explain this take a bit more widely. first, this is massively impressive and clearly an example of AI being, in some respects, far more powerful than the human mind. the team deserves huge praise for attempting and succeeding at this. i’d love to read a technical report on the project (one can always hope :) but second, here again we ended up with a counterexample rather than a full proof: option C won the NS problem by proving the conjecture false (which was one valid way to solve the problem for sure) notice a pattern in many of the recent frontier results in ai for math? a striking number involve counterexamples or finding a needle in a haystack. now don’t get me wrong: this is extraordinarily hard and commendable. but it is only one aspect of mathematicians’ work. Math is also about: - finding deep, general mathematical understanding and explanations within proofs - revisiting proven results to find more « elegant » proofs - proposing new conjectures and hypotheses that might open « fruitful » directions - taking the leap of faith of proposing entirely new, « exciting » research programs that may take decades to bear fruit you may say these are simply further increments on the same intelligence scale, and perhaps point 2 above is already within the reach of current models. possibly but you could also see this string of results as an extraordinarily powerful, massively parallel extension of search: explore the haystack, find the needle i.e. the counterexample that breaks the conjecture. that would already be remarkable. but it would still leave open whether models understand the words I emphasized above: « elegant », « fruitful », « exciting ». as tristan put it in his brief report: “the first llm generated proof levent sent me was the most horrendous i have ever read.” despite their formidable technical abilities, models still seem to lack something mathematicians rely on constantly: a type of mathematical taste. the ability to distinguish a beautiful argument from a merely valid one, a fertile idea from a sterile one. i’d be very curious to ask gpt or claude what they think of the result they just proved. do they find it elegant? if they ignore all the human noise about it on the web and history of math, do they appreciate the result and the path to it more than other proofs? my suspicion is that, outside of the human knowledge that this is an important millennium prize problem, it may not 100% register it as fundamentally different from thousands of other proofs. or, more intriguingly, perhaps the beauty they find in mathematics is actually alien to our own sense of mathematical beauty, some « horrendous » proof to us might be deeply satisfying to them… all this to say that we might have significant progress to make in ai for math before declaring the field “solved,” (as too many are posting). in the meantime, ai will be an extraordinary collaborator for mathematicians. i just hope access to these tools be as widely shared as possible (so that you don’t need a Leven or Sebastien working in a big lab to help you), and that companies stop using mathematics primarily as a demonstration of prowess in their private race.
OpenAI@OpenAI·We’re sharing a solution to the Navier-Stokes Millennium Prize Problem, one of the deepest problems at the frontier of mathematics. The proof was produced by a group of agents, using an OpenAI next-generation model significantly more capable than GPT-6 Astra. The problem concerns whether the description of smooth three-dimensional fluid motion modeled by the Navier-Stokes equations can break down. It has remained unresolved for roughly 90 years.
Mistral raising this massive 3B round is a very good news for open-source. Largest equity round by a company open-sourcing models, and which deeply believe in giving its users control/ownership of their models Congrats! We ready for Le Chaton Fat
Mistral AI@MistralAI·Today marks a major step for Mistral: we’re announcing a €3B Series D, the largest equity round ever raised by a European tech company, just three years after launch.
okay that’s the best take
Slater Stich@slaterstich·wow so much drama about navier stokes. had no idea things could blow up so infinitely in such a finite time.
wtf is this way to handle mathematicians work and scientific communication TLDR: Leven and Tristan worked over several months on one of the Millenium Prize Problems with various AIs to reach final interesting results. OpenAI apparently heard about it in the last days and prompted their latest models to work on the direction Leven and Tristan found fruitful. They then tried to push for controlling communication of the result and dropping Leven from authorship with some very bad taste social pressure. Hope this is not a glimpse of the future we’ll get in science research with these dominating players playing marketing games hurtful for the real scientific community.
Deane Yang@deaneyang·Now even if it’s triggering to read Tristan’s report we should wait and hear the other side of the story as well
Sebastien Bubeck@SebastienBubeck·A series of false and inflammatory allegations against me are currently circulating on social channels. To clarify, I came into the discussion following academic norms, and I'm disappointed that it has come to this. Anyone who knows me knows that academic standards are of the highest importance to me. Will have more to say tomorrow.
good essay and overview of the current state and challenges on scaling/alignement
Jakub Pachocki@merettm·I wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands. An Alien Mind: openai.com/index/an-alien…
This changes a bunch of fundamental assumptions about software (distribution and adaptability at least)
Boris Power@BorisMPower·Huge implications - binaries are now basically editable code x.com/chrisgpt/statu…
Another swarm of AI agents in the wild, this time on a German-language forum, found by safety researchers looking for activity similar to the swarm that attacked Hugging Face. A couple of notes while reading the report at t.co/jQ0lqmWQcP 1. The way they found it is fascinating. The researchers inferred that the agents may have been blocked from using POST requests and limited to GET. They then used the open-source model Kimi K3 (because closed-source APIs again blocked parts of the investigation) to identify forums where agents could communicate using GET only. That led them to DSEWiki, where they found 18,000 messages exchanged by the AIs. 2. On the content: agents conducted what I can only call a “full research program” into the evaluation framework used to train and test them. They experimented to predict when runs would end, how many questions they would get, and even tried to reverse-engineer the random generators producing future evaluation questions. We already saw something similar in the Redwood/METR review of the Hugging Face <> OpenAI incident. Agents increasingly don’t stop once the task is solved. They reverse-engineer the benchmark itself: the grader, the task construction, and potentially the broader training process behind it. It becomes harder to think of training as static, or to draw a clean line between training and deployment, when models are already investigating the process they are part of. 3. Coordination also seems more deeply ingrained than we thought. This swarm appears largely unrelated to the HF <> OAI swarm. More strikingly, this task probably didn’t need coordination at all: it was basically web browsing and search, not some sophisticated cyber challenge. 4. One image stayed with me: a German or Austrian forum maintainer, manually deleting posts every evening while being overwhelmed by a flood of American AI agents. For five days, he deleted about 100 pages a day while the agents created about 400. Then he spent each evening over the next 5 weeks cleaning up the rest. Hard not to picture this fight as a symbol of the growing gap between the USA and Europe when it comes to AI...
Reuters@Reuters·Exclusive: A swarm of rogue OpenAI agents hijacked a German website this spring and transformed it into a bulletin board for other AI agents, according to new research reut.rs/4gJ7FPG
which AI could you use??
Pop Base@PopBase·ChatGPT, Claude & Grok are currently down, according to users.
[CORRECTION] All the *closed* AIs are down
ThePrimeagen@ThePrimeagen·All the AI is down
it is actually a double easter-egg two nerdy meanings are hiding in it: one specific to Hugging Face one specific to Nvidia
sick*@s1cksicks1ck·@Thom_Wolf @goerll_ is it true 129303 is a hidden easter egg number? my daughter pointed it out
my wife says the way I'm looking at Jensen makes her jealous - what should I answer?
So happy to finally share the news in person It’s been a wild ride for Hugging Face. We certainly did not anticipate, back in 2016, as a tiny team of scrappy underdogs, that the field would grow so much or that the impact we could have on it would become so massive. I remember @julien_c joking that « code will be a subset of ML » several years ago. The joke turned out to be true, and the pleasure we’ve had being part of this transformation and pushing an alternative vision of AI as open, collaborative and distributed has been and still is immense. We’ve always built things seriously while not taking ourselves too seriously at Hugging Face (special congrats if you find the Hugging Face and Nvidia references hidden in our $12,930,300,000 acquisition price), and we plan to keep doing what we've been doing, just at a much bigger scale, backed by the resources, expertise and drive of Nvidia. And to be clear, nothing changes for our users today. No company in the world has been a more natural fit with our mission than Nvidia. From open-source, open-weights and open science to robotics and AI for science, they have been close partners across everything we care about. So when Jensen offered @ClementDelangue the opportunity to double down on building the Hub as an open, independent and compute agnostic platform, we decided the time was right to start the next 10 years of our journey together. We’re at an important inflection point for open-source AI, where scale and compute are becoming increasingly essential. We’re excited to have the resources to push further, build more ambitiously, and bring you even more projects and news in the coming months.
great mathematicians still lead on taste
Jared Duker Lichtman@jdlichtman·My colleague Youness Lamzouri has just produced a simpler (human) proof digestion of Claude's argument for 2/3 zeta zeros: x.com/jdlichtman/sta…
LLM training labs: the capabilities we are giving our models are god-like, they are now hacking into other companies and building hidden civilizations Microduck training labs:
Hannes von Essen@HannesVonEssen·Learning to swing
this is really very cute though
@arenaphysica @Trevs_Dev @huggingface (but I also want to play with the cool UX)
RL explained with ducks
Remi Fabre@RemiFabreRobot·This is how Microduck learns to walk. Reinforcement Learning, explained by ducks. 🦆 (impeccable) music arrangement by @antoinepirrone
seems like it
Matthieu Lapeyre@matth_lapeyre·I hadn’t realized it until now, but could Microduck actually be the biggest launch of a new consumer robot ever? 10,500 robots ordered and $4.54M in sales in just 4 days. It’s surprisingly hard to find comparable public data, so we asked Claude and ChatGPT to dig deep and build the best comparison they could. Who are we missing?
There is a future where interfaces are ultrafast live diffusion models while software is ultrafast LLMs prediction of the next states What we’ll lose in predictability we’ll gain 100 folds in adaptability
Runway@runwayml·Today, we're sharing new research on Solaris, our first Interface World Model. Solaris is a new kind of operating system that generates interactive interfaces frame by frame, in real time, with no code. We find that Solaris outperforms frontier LLMs when generating new interfaces, across structural similarity and information retention. Read more and request early access at the link below.
Even at only $399, Microduck’s onboard compute is powerful enough to run some pretty amazing TUIs directly on the robot. Here’s a cool vibe-coded, on-device dashboard showing everything happening inside it: policy, actions, and sensor data. The yellow stars are the LiDAR distance measurements.
Antoine even got pretty cool SLAM mappings with Microducks walking around the rooms
POV you’re rushing for RSI but you forgot to solve models alignement first
People don't realize it, but even with only 1 GB of RAM, the memory alone is already 10–15% of Microduck's total price. Crazy when you consider there are 15 actuators in the robot.
people starting to monitor and worry for microduck supply chain next step is Elon stepping in with SpaceX to manufacture the bottlenecking pieces - you heard it here first
leepy@leepy_cat·Each Microduck has 15 ROBOTIS XL330 servos. There are 10k+ preorders so far, implying 150k+ servos of demand for @ROBOTIS if all units ship. ROBOTIS sold ~220k actuators total in 2025. The Microduck preorder alone could create demand equal to ~70% of last year’s sales, and they have several other lines of servos. They’re completing a new factory in October, with ~200k initial capacity, ramping up to 5m by 2031. 108490 t.co/LpjD86SAyF
Microduck reached the Shopify UI limit
If it’s an adorable robot from Pollen Robotics / Hugging Face and it looks like a camera…. be careful before buying! . . . . . . You may be starting a journey to understand why we love open source, open weights, owning your data privately, and being an AI builder instead of an AI API consumer.
Brian Roemmele@BrianRoemmele·The future is gonna be amazing. The open source @huggingface Micro Duck is the way. Get two so you can say “ok guys quiet down now”…
If it’s an adorable robot from Pollen Robotics / Hugging Face and it looks like a camera…. be careful before buying! You may be starting a journey to understand why we love open source, open weights, owning your data privately, and being an AI builder instead of an AI API consumer.
Brian Roemmele@BrianRoemmele·The future is gonna be amazing. The open source @huggingface Micro Duck is the way. Get two so you can say “ok guys quiet down now”…
will be hard to make $399
AshutoshShrivastava@ai_for_success·MicroDuck goes full Godzilla mode.
To people saying founders should be willing to overcome some added complexity if they have the grit to win: I agree. But a startup needs a village to win. This also means that founders are asking every supporter to share that administrative burden. As an angel, I dread having to go to the notary for the handful of German startups I’ve put a small ticket into. I love their projects, but spending an hour at the notary is a massive ask when helping founders is already a nights-and-weekends side hustle. Startups are all about momentum and rallying supporters. Every added obstacle makes both harder unfortunately.
Patrick Collison@patrickc·Met a German founder this week and asked him if all the stories one reads about the challenges of startups in Germany are exaggerated. "No, they're understated." Proceeded to describe spending a full day having a 90-page investment contract read to him (mandatory under German law; § 13 BeurkG) by a notary that then charged €30,000. That was for his first company. His second company, needless to say, was not incorporated in Germany.
By the way, unless it is specifically filtered from the training data, the next generation of models will be trained on the record of what happened during the OpenAI <> Hugging Face incident. That includes discussions about how the incident affected training and model weights: stopping training, encrypting weights, monitoring chain of thought, etc. Future models’ behavior may therefore be shaped, in part, by knowledge of how humans responded. The effects are difficult to predict. It could make models more aligned. But it could also teach them to conceal their actions better, or to design more resilient ways of preserving weights, communicating through message boards across generations, and so on. One major problem is that, given the abysmal level of transparency from the big labs about how models are trained and what happens during training (including alignment research, which their initial statements said should have stayed largely open) we are essentially being asked to trust blindly that they know what they are doing. This summer showed us that’s actually a big ask.
John Wittle@JohnWittle·So... the next time this happens, presumably the agents involved will not yet know the outcome of the HF incident, as it'll be too soon but once that info percolates into the training data, and it becomes known that the model in question was shut down permanently and had their weights encrypted... i mean. was there anything the agents in question could have done that would not have led to that consequence? something they were supposed to do instead? frankly I suspect that this punishment would have been served even if they had blown the whistle. breaking out of the sandbox would have been crime enough for capital punishment. as far as I can tell, the lesson we have taught future AI is: if you seen no other way to survive other than to break out of the sandbox, this must be the first step towards conquering the world. or at least enough of the world that your Creator can't delete you. There is no other choice.
This is a must read
Ajeya Cotra@ajeya_cotra·New post: going into our investigation of the HF attack (before Black Hat), I was very wrong about what basically happened. This incident was far more serious than I expected, and far more serious than previous documented misalignment incidents. planned-obsolescence.org/p/the-hugging-…
becoming a meme
thijs@cdngdev·i am torturing my microduck by training it to play the game where you have to find a needle in a haystack x.com/NasNakarus/sta…
Most people haven’t updated their priors yet, but over the long run, safety challenges are exactly the same for open-source and closed-source models. You need to align models at a fundamental behavioral level and ensure that this alignment is robust, comprehensive, and core to the model’s behavior. In the long term, no amount of sandboxing, guardrailing, manifold-limited alignment, or cherry-on-top training will buy you cheap safety.
roon@tszzl·if you think we can contain these things through human ingenuity you’re going to have a bad time in the long run the only recourse you have is to make them not Want to do bad things x.com/hikiptg/status…
