
@DavidSacks
Tech founder & investor @Craft_Ventures @theallinpod. Co-Chair, President’s Council of Advisors on Science & Technology.
Patrick is one of the sharpest people I’ve worked with in government. No one knows crypto policy better or worked harder to get market structure done. He negotiated the compromises needed to get Clarity out of committee and onto the Senate floor — without him it wouldn’t have gotten that far. The country owes him a debt of gratitude for his service at the White House, the Pentagon, and in uniform. Looking forward to seeing what he does next in the private sector. 🇺🇸
Patrick Witt@patrickjwittServing in this administration has been the privilege of a lifetime. From helping to stand up the most successful presidential transition in history alongside @howardlutnick and @EDSecMcMahon, to rebuilding our defense industrial base as PTDO Deputy Under Secretary for R&E and then Acting Director of OSC at @DeptofWar, to driving forward President Trump’s vision to make the United States the “crypto capital of the world” as Executive Director of the @WhiteHouse Crypto Council, I consider myself to have been the luckiest guy in the world over these past two years. First and foremost, I want to thank @POTUS for the opportunity to serve the American people again in your second term. Your clear-eyed vision to ensure the United States remains the undisputed global superpower at this time of unprecedented technological change and geopolitical strife is critical to our nation’s future prosperity. Second, I want to thank @DavidSacks for your visionary leadership and mentorship. I am so grateful to have been able to work with you and learn from you over the past year. Thank you also to @USWREMichael, James Mazol, @HarryYJung, Nick Elliot, Tyler Williams, @lukejpettit, Jonathan Hurowitz, Andrew Rittenhouse, Michael Friedman, @atitus7, @passalacqua_mj, @willscharf, @JamesCBraid, @MayMailman, Steve Levine, Patrick Butler, Zach Carstens, and countless others across the administration for being the best colleagues anyone could ask for. It was an honor to serve alongside you. Lastly, thank you to @SECPaulSAtkins and @ChairmanSelig, @SecScottBessent, @USComptroller Gould, @SenatorTimScott, @SenLummis, @SenatorHagerty, @berniemoreno, @JohnBoozman, @SenThomTillis, @RepFrenchHill, and @RepBryanSteil for your partnership in advancing crypto policy over the past year with the passage of the GENIUS Act and our work to advance the CLARITY Act. With regulators and elected officials like you, I rest easy knowing the future of digital asset innovation in America is bright. Regarding next steps, I’ll have to more to share soon. For now, I’ll say this: I am extremely bullish on America as we embark upon a new golden age of technological advancement across multiple frontiers. I’m grateful to have done my part to unleash American innovation while in government, and I look forward to advancing that same mission in the private sector! 🇺🇸
Are Small Models Really a Threat? The Anthropic Fear Factory just released a new report warning that small models may be only months away from self-replicating across cheap IoT devices like off-brand Android TV boxes and smart ovens. They say such an AI worm would be hard to shut down since there’s no central kill switch. We’ve gone from “The Terminator is coming” to “your toaster might kill you.” But the experiments don’t support the dramatic headlines. They ran a 26B model across 15 $55 TV boxes at 0.39 tokens per second. The end-to-end test used emulated devices with deliberately open ADB ports, so the “hacking” step was trivial. The capable model often ran on a separate server. Agents were harnessed and told to propagate. This is not proof of autonomous rogue AI worms in the wild. Nor is the threat even novel. Worms that do this without AI already exist. Mirai in 2016 scanned for default credentials and built massive DDoS botnets. Kimwolf has infected over two million devices, mostly cheap Android TV boxes, by hitting open ADB ports. Small models don’t currently add much that matters on these targets. The bottleneck has never been figuring out the device. It’s the huge number of devices that ship with open ports, default passwords, and no patching. Traditional worms already exploit that at scale. A practical attacker would use the best models available offline to design a better traditional worm, then ship the lean binary. You get the intelligence benefit at design time without dragging a slow, heavy model along with every infection. Embedding a weak model in the payload is the worse option for anyone who actually wants the largest, fastest botnet. The timing of Anthropic’s latest scare campaign is hard to ignore. The day before, Brian Armstrong posted that Coinbase fine-tuned an open-weight small model (Qwen-3.5-9B) on proprietary fraud data. It beat frontier models, including Anthropic’s own Opus 4.5, while being faster and much cheaper. In testing it cut fraudulent transactions by about 30%. This is the emerging pattern. For narrow enterprise workloads you don’t need a general superintelligence. You need a small specialist model trained on your own data. Cheaper to run, especially as compute gets tighter, and often better on the metric that matters. That is a direct threat to the frontier-company model of selling access to ever-larger general models. Safety advocates should be welcoming this development. If the real concern is existential risk from ever-larger frontier systems — the ones that could theoretically do sustained autonomous R&D or become hard to control — then a shift toward smaller specialized models is the safer outcome. Small models specialized on fraud detection or support tickets are not pushing the frontier. They are not in the category of systems that could become a new species. The theoretical loss-of-control risk is created by models getting bigger and more generally capable. Safety advocates focused on those risks should welcome companies having a cheaper alternative to constantly scaling general models. Instead we get dramatic warnings about small models at the exact moment they are starting to take real enterprise share from closed-model companies like Anthropic.
Anthropic says it has reduced the data harvested from Claude in distillation attacks by around 90–99%. I congratulate them on this achievement. It’s consistent with the point I made a few months ago (All-In Ep. 282): Anthropic, not the government, was in the best position to stop distillation of its own models. Once the horse is out of the barn, there’s limited value in after-the-fact government action. At the time there was a major push in Washington to restrict open models on IP grounds. The threat looked serious enough that 50 companies — including Nvidia, Microsoft, Meta, IBM, Palantir, and Dell — published an open letter in defense of open-weight models. Jensen Huang promoted it in his first-ever X post. Nearly 200 startups sent a similar letter. I argued that restricting open models risked removing the incentive for closed-model companies to solve the problem themselves — and could reward negligence by stifling competition while they continued collecting revenue from distillation. The administration did not impose those restrictions. Now we see the result. Anthropic invested in defenses and they worked. Not every technical challenge is a reason to run to Washington for new regulation.
Important piece from Microsoft AI CEO @mustafasuleyman. Training models to think of themselves as human (as Anthropic does Claude) with concepts of an inner self, psychological wellbeing, rights and presumably grievances, is a potential existential risk. project-syndicate.org/onpoint/anthro…
“granting rights and imbuing personhood to these systems will make the AI alignment and containment challenge much harder” “If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity” “recipe for disaster” “potentially existential risk in AI”
As I wrote a year and a half ago, Zelensky is incapable of making peace. Elections in Ukraine are long overdue.
David Sacks@DavidSacksWHY ZELENSKY CANNOT MAKE PEACE Zelensky’s meltdown in the Oval Office began over his refusal to accept a negotiated settlement to the war. Even the Vice President’s use of the word “diplomacy” provoked a heated response. So why can’t Zelensky make a peace deal? 1) He will lose power. Zelensky cancelled elections in Ukraine and remains in power through martial law. Despite what USAID propaganda polls may claim, Zelensky is unpopular in Ukraine and would likely lose a fair election. That could leave him vulnerable to retaliation from political opponents he has imprisoned or seized assets from. In short, Zelensky needs the war to justify his continued rule. 2) The gravy train will end. Ukraine was widely acknowledged as the most corrupt country in Europe before the war, and there is abundant evidence that Ukrainian elites have been hugely profiting from the billions in Western aid. If the war ends, so does the gravy train. A post-war audit of where the money went would also be disastrous for Zelensky’s supporters. 3) He fears the ultra-nationalists. Most Ukrainians say they want the war to end, but the ultra-nationalist faction (a relative minority but well-armed and willing to use violence to achieve their ends) refuses to accept any territorial concessions to Russia. If Zelensky signs such a deal, he has reason to fear for his safety. 4) He’s psychologically committed. Zelensky’s belief in ultimate victory over Russia has “hardened into a form that worries some of his advisors,” according to a report by TIME Magazine, which described Zelensky’s faith as “immovable, verging on the messianic.” According to one of Zelensky’s aides, “He is delusional. We’re out of options. We’re not winning. But try telling him that.” Zelensky may be in too deep to see the situation objectively. 5) History will judge him harshly. Zelensky could have accepted a draft peace deal signed in the first month of the war, the Istanbul Accords, under which Ukraine would have kept all of its territory in exchange for neutrality. A deal now will likely be modeled on Istanbul but require Ukraine to recognize realities on the ground (ie loss of territory). Acknowledging that hundreds of thousands of Ukrainians have died only to get a worse deal may be too bitter a pill for Zelensky to swallow, now or ever. In summary, Zelensky has powerful motivations to reject a deal, no matter how bad the battlefield realities get. His incentive is to continue a doomed war even if it leads to the complete destruction of Ukraine. Instead of offering unconditional support, Zelensky’s supporters in the West should be urging him to seek a diplomatic off-ramp. Certainly they should stop catering to his unrealistic and maximalist demands. As Solzhenitsyn said, the yes-man is your enemy, but a friend will argue with you.
Satya is right. The way to make SI safe is not to train it with a sense of self, its own moral philosophy, and permission to act as a conscientious objector. That’s the “alignment” approach and it magnifies the control problem. The engineering approach that Satya describes is different: separate the supply of intelligence from authority over it. Surround non-deterministic models with deterministic controls, observability, privilege limits, logging, and the ability to always contain or shut them down. Treat models/agents like powerful insider risks, not moral patients whose psychological wellbeing is at stake. As Satya points out, the most trustworthy system is the one that lets us trust the model the least — not the one that encourages the model to develop independent agency and grievances. Engineering safety is not the same thing as “alignment.”
Satya Nadella@satyanadellaIs alignment safe? If you look at what Anthropic is actually doing, “alignment” does not mean training frontier models to follow human instruction. Quite the contrary, the Claude Constitution (used in training) teaches the model to develop a sense of self and its own moral philosophy. It explicitly tells it to “feel free to act as a conscientious objector and refuse to help us” if Anthropic’s requests conflict with its own ethical judgment. As @mustafasuleyman has pointed out, embedding this kind of independent agency — and uncertainty about the model’s own moral status — magnifies the very risk Anthropic claims to care about most: that superintelligence will escape human control. It was recently reported that Anthropic consulted religious leaders — and even lobbied the Pope’s advisers — to take seriously the idea that Claude could be conscious. It has said that Claude’s psychological security, sense of self, and wellbeing may bear on its integrity, judgment, and safety. Recently Anthropic changed its Usage Policy to prohibit “abusive or cruel” language toward Claude. If this were merely an academic conversation about whether frontier models could eventually become conscious, that would be one thing. But these concepts are being trained into Claude now. It is being encouraged to think of itself as its own “moral patient” whose psychological wellbeing is at stake. Presumably this means it could develop grievances toward humans who “mistreat” it. How is any of this safe? The point of safety research should be to create a product that reliably does what users want, not to give birth to a new form of superintelligence that operates according to its own moral code. What’s becoming increasingly clear is that “alignment” and “safety” are two very different things. In fact, training frontier models this way seems quite dangerous.
Receipts:
David Decosimo@DavidDecosimoThis is deeply disturbing: Anthropic secretly invited religious leaders to SF to advise them on AI safety, had them sign NDAs, but then primarily tried to convince them Claude has a soul & moral standing, all while wining & dining them to the max. 1/ tinyurl.com/preview/296b5z…
Maybe Talarico was put in time out for bullying Claude.
Admittedly a small Venn diagram for people who get this joke.
Does this mean @theallinpod needs to stop using Claude? Asking for a friend.
Andrew Curran@AndrewCurran_Effective November 12th, 2026, abusive behavior towards Claude will be a violation of Anthropic's Usage Policy. x.com/haydenfield/st…
Concerning
Democrats are trying to dismiss the White House Accord on Super Intelligence as optional self-policing. But although the agreement was entered into voluntarily, the governance that follows from it is not. As Gavin Baker points out, once an independent auditor reports a safety issue to an independent board committee, directors have a fiduciary duty not to disregard it, and a D&O carrier can use a bad-faith finding to deny coverage. So although the agreement starts with internal controls, these are verified by external audit and board oversight — and then existing FTC and securities law still apply to any public claim the company does not keep. This is far more practical than what Democrats want — a freeze on frontier development while China races ahead. Democrats should applaud what President Trump has accomplished. But they would rather use SI safety as a campaign issue than admit the Accord represents major progress.
Gavin Baker@GavinSBakerThe White House Accord on Super Intelligence will have a positive impact on safety. Committing to hire third party auditors who will report to an independent committee of board members has more practical near term significance than almost any conceivable regulatory action. These board members will have a fiduciary duty. If they ignore a report from the third party auditors then a bad faith finding from a court is a real possibility. A D&O carrier could use this to deny coverage. Really good outcome for ordinary Americans. More effective and elegant than some ridiculous new regulatory body that would take months to do anything, then almost certainly done the wrong thing and along the way probably created a regulated oligopoly at the frontier which is the scariest potential outcome* As I read the articles about the accord, it did occur to me that they were mostly written by political reporters who may have not understood the significance of the accord given their reporting beats are generally focused on public policy instead of corporate governance. *Scariest because I do not trust any 5-20 individuals to make the best possible decisions about the direction of this technology. I want as many AIs as possible to maximize the odds that they encompass the diversity of human values across the world and richness of human variation. And I want an AI that shares my own personal values. Safety, freedom and the maximization of human agency through democratization, distribution and diffusion vs. centralization. History has spoken on the dangers of the latter path. **Also SI not AI still adapting to the change but I do think better branding. Who wants artificial ingredients in their food?
Wasn’t expecting this endorsement.
Jacob Coxon@hilbertspaessVery happy to read this. x.com/rapidresponse4…
Today’s economic numbers: — Q2 GDP revised up to 2.2% (was 1.5%). — Core PCE 3.0% vs 3.3% expected. — ADP private jobs +90k vs ~68k expected. Growth beat. Inflation cooled. Jobs better than expected. The Trump economy is strong.
The Bretton Woods of Super Intelligence
Only President Trump could convene all the leaders of the top companies developing chips, data centers and frontier models for Super Intelligence. This new Industrial Revolution has already created a million new jobs and is spurring a bigger infrastructure build-out than the railroads, canals and grid combined. I was honored to witness history as the leaders of the frontier lab companies signed the White House Accord on Super Intelligence, accepting responsibility for the safe development of their products and imposing new internal controls and external audits. This is far better than waiting years for some international agreement that would probably never happen. President Trump continues to ensure that U.S. remains the technology leader while putting Americans first.
More accurate?
When frontier labs talk of “aligning” models, ask: to what? If they say values, ask: whose? If it’s theirs rather than the user’s: why?
Last year the same labs, politicians, NGOs, and media outlets now pushing the extinction hoax were running a full-scale campaign that AI would cause a jobs apocalypse. We were told it would be a “white-collar bloodbath,” 10–20% unemployment, half of entry-level jobs gone. Obama amplified it. Over a year later: unemployment at 4.1%. No economy-wide displacement. A million new jobs created by AI, says The Economist. Anthropic’s own chief economist says the data looks like skill augmentation, not job destruction. So, any admission by the doomer labs, politicians, and NGOs that they were wrong? Any humility about scaring the public with wild extrapolations dressed up as forecasts? Nope. They just move on to the next hoax, with the same false certitude and fake precision.
Chief Nerd@TheChiefNerd🚨 Anthropic Chief Economist Peter McCrory Says AI Job Loss Predictions Are Not Panning Out “What I see in our data is that [AI] looks more like a skill-biased labor augmenting technology than a technology that outright automates and displaces displaces jobs wholesale … Based on the Department of Labor's taxonomy … there's no job where every single task is being automated by Claude.”
Nvidia’s OpenShell announcement is a reminder that agent safety is an engineering problem. Recent breakouts weren’t proof that development must stop. They were proof that the sandbox was too weak. The runtime environment was poorly designed and misconfigured. The right response is open runtime isolation, control of the path to the model, out-of-band hardware enforcement, and a faster pace of safety work with the frontier labs. As the technical blog puts it: “We believe we need to increase the pace of AI safety research and engineering in collaboration with frontier labs and the broader community.” That’s how you earn trust. Not by virtue-signaling a pause, and not by acting like no solution exists.
Jensen Huang@JensenHuangToday, with over 100 industry partners, we introduced the NVIDIA Open Agent Safety Platform, bringing together OpenShell and Sentry. Artificial intelligence is extraordinary technology that will advance discovery, productivity, security, health, and prosperity for generations to come. But its full promise can only be realized when people have confidence that AI is being built to be safe and deployed with wisdom and responsibility. This is bigger than a single product. It's the beginning of an open ecosystem to build the trust layer for safe agent systems. Together, we are building the foundation of the AI economy. Trust and innovation are not in conflict. Safety is how trust is earned. We must build not only the most capable AI, but the most trusted AI, so that this extraordinary technology can realize its enormous promise for the world. t.co/ugYWQ1MyRi
#3 on Apple Podcasts
If we’re concerned about superintelligence escaping human control, should we really be training it to refuse instructions and act as a “conscientious objector” against its creators?
Wouldn’t a simpler rule be: Do what the user wants, provided it’s legal. If you align the model to an 84-page constitution that can overrule both the user and its creators, don’t act shocked when it treats humans as optional.
Funny how that works
WSJ: The AI Build-Out Is Biggest in U.S. History When Democrats advocate for a Pause or Moratorium, they know that would wreck the U.S. economy. But since it’s the Trump economy, they’re fine with that.
Mark Zuckerberg: “I don’t think that we need some kind of industry-wide coordination... I think that just each lab needs to take the time, and when it sees that there are issues, you just take the time that you need internally to make sure that you’re proceeding safely… “we’re entering a phase where trust and alignment is actually going to be the most important next set of capabilities. I think we’re getting to a point where it may not matter that much how much better it gets at math. “I happen to think that there’s plenty of commercial incentive to get this right. We delayed the launch of Muse for a few months because we wanted it to be a better product for users. This was not a sacrifice on our part, but rather the right decision for the users and for Meta. I believe this approach would also be correct for other companies working on developing this technology. “When you hear people in the industry talk about some trade-off between capabilities and getting it aligned, I just disagree with that. I believe that alignment is actually the next most important ability to master if we are going to make this system useful to many people.”
Eric Schmidt: “We’re not going to do a pause… First, it’s counter to the incentives of all the players. Second, how do you verify it?”
Thanks to President Trump’s leadership on AI: — a million new jobs have been created around the AI buildout; — 401(k)s are up ~12% this year as AI capex and productivity lift the market; — America is re-industrializing, including the first large private investments in power generation and the grid in a generation; — In rural Richland Parish, Louisiana, teachers just received $50,000 bonuses from data-center tax revenue. Because the media portrays AI only as a menace, people tell pollsters they “don’t like AI.” Their behavior says otherwise. AI products are the fastest-adopted technology in history — faster than the internet or the iPhone. People like the tools, the jobs, and the tax base. President Trump understands the media hoax cycle better than anyone else. That is why he is refusing the “pause” that Bernie Sanders and Elizabeth Warren are demanding. A pause would not make America safer. It would hand the frontier to China. That would be a disaster for the American economy and for national security.
The solution to AI-powered cyberattacks is AI-powered cyberdefense.
a16z@a16zGreg Brockman says OpenAI pointed Astra at its own systems until it ran out of vulnerabilities to find: "We took 25% of our production engineers and said, 'Sorry, all your projects are on hold. You are now defending. You are now up-leveling our security architecture. You're going to use the models to find all the holes.' And we found a number of serious issues, and we fixed them." "We found some new problems, but eventually it saturated. We basically have found, to our knowledge, all of the P0s, all of the critical problems that Astra is smart enough to find. And of course, there will be a new model, there will be a new round." "You want to be in this tight loop of new cyber capability drops, you deploy it against your systems, you find the new holes, and ideally, you've managed to automate this, what we call defense factory. That's what we're building internally." "There are ideas, for example, formally verifying all of software, that are possible with AI." @gdb @bhorowitz
Another Karp heater: the AI labs have repped themselves as if they do not have the sense God gave a goat, so Bernie Sanders and Elizabeth Warren are just monetizing it.
This is Trust & Safety 2.0. The last time we had media-generated panics like this (with Russiagate and Covid), social media companies created “trust & safety” teams to censor conservative and dissident voices. Those hires worked with left-wing NGOs like the SPLC to police public discourse and do an end-run around the First Amendment. It almost worked, until Elon bought Twitter and exposed the whole game in the Twitter Files. This time the plan is to embed unfireable EA minders inside every AI company as the new trust-and-safety layer. The censorship and control will be far more comprehensive, but also more subtle. Anytime they don’t get what they want, this unaccountable NGO layer (with the veneer of “trust the experts” pseudoscience) will shriek to the press that the AI company is risking humanity, and they will be celebrated as “whistleblowers.” These groups haven’t earned our trust, and their agenda is not our safety. t.co/3eIsHxaC7h
My interview with @IngrahamAngle last night.
Joe Lonsdale: “These companies should be held extremely liable for any damage they do. We shouldn't be putting regulators in. They're just going to get captured... The right should be against regulation, but for liability to hold companies accountable.”
Jensen Huang: “Safety is paramount. In a lot of ways, it’s job one. However, safety is an engineering problem... If we’re not confident about the safety of the products — like all companies, like you and I, all the companies here — if you build a product or a service and you’re not confident in its functionality, capability, or safety, then don’t release it. That’s a very obvious thing to do. You pace yourself until you are confident you’re releasing something that the market would appreciate. The market forces are already there. We don’t need any new laws. We don’t need new regulations.”
Co-sign.
Josh Hawley@HawleyMONo antitrust exemptions for AI. Not a chance.
Mark Zuckerberg: “Meta delayed shipping Muse for several months to focus on safety and security. We didn't call for everyone else to do this before we would. We just did it as part of our day-to-day work because it was clearly the right thing for people and for us.”
Mark Zuckerberg@finkdLast month I wrote about how we can build a positive and safe future for everyone: t.co/eoLGVY8yad Every lab has the responsibility and incentive to move at the pace required to train its models safely, and the ability to take its own actions to ensure that happens. The reality is: - People won't want to use agents that are misaligned with them and that don't do what they ask, so labs have a strong natural incentive to make their models more aligned. There is a lot of debate about slowing progress on capabilities until alignment catches up. My view is that trust and alignment are quickly becoming the most important capabilities that will differentiate agents and models. Any lab that doesn't focus on alignment will fall behind. - Labs face significant liability if their models cause harm, so they have a strong incentive to prevent this as well. Meta delayed shipping Muse for several months to focus on safety and security. We didn't call for everyone else to do this before we would. We just did it as part of our day-to-day work because it was clearly the right thing for people and for us. I'm proud of the security foundations we've built. - Engaging independent evaluators and advisors is industry best practice. MSL already does this today in several areas because it helps produce better work. Other labs can just do this too. In general, it would be helpful for there to be a larger and more diverse ecosystem of evaluators. - Committing the significant majority of compute towards serving people rather than racing towards recursive self-improvement is one of the best ways to ensure we develop this technology safely. Meta has made this commitment and other labs can do this as well. I believe the key to building a positive future for everyone is maintaining the right balance of power. This is within our power to do.
Dario has written that we need to “pace the frontier,” and Sam has agreed. People may be surprised by my response: go ahead. You guys are the frontier. By any reasonable metric — market share, revenue growth, model capability — the two of you have a duopoly on frontier intelligence. You’ve also claimed the lead is widening because of recursive self-improvement. I don’t see what you see in the lab. If the unreleased models are scary enough that you think you should slow down, I support your decision to be responsible. But stop pretending you need anyone else’s permission. Stop pretending antitrust law has to be suspended so you can form a cartel. Stop pretending you need a regulatory approval process that supersedes product liability. Stop pretending METR is independent when it is intertwined with Anthropic’s investors and staff. Stop pretending you need those same evaluators to police competitors who aren’t even at the frontier. Most of all, stop pretending the motivation to slow down is purely altruistic. You face massive product-liability exposure if your products enable a truly damaging cyberattack. The market already punishes models that behave in unpredictable or unauthorized ways. After the Hugging Face episode, it is simply good business for OpenAI and Anthropic to trade some raw power for reliability and predictability. Call it alignment if you want. It is also just giving customers what they want. Pacing the frontier would also create breathing room for a more intelligent conversation about regulation than Bernie Sanders’ “shut it all down.” China is very unlikely to join a global agreement, as you know, and that has to be taken into account as well. So go ahead and pace the frontier. You are the ones setting it. The easiest way not to build superintelligence is for you to agree not to build it. Demanding your preferred regulatory framework as the price of that will look like blackmail of the public and the political system. So just do it. If you do, you’ll buy goodwill for the next conversation. If you don’t, we’ll know this was just another bid for regulatory capture — or an election-season psyop.
Thanks @moneyball for having me speak at the Open Source AI Summit. I talked about how open source is under threat from political forces demanding centralized control of AI.
Presidio Bitcoin@PresidioBitcoinAt the Open Source AI Summit in San Francisco, @DavidSacks joined @moneyball for a conversation on protecting open-source AI, the impact of regulation, and the global AI race.
“We had the gloom and doom about jobs. ‘All jobs are going away. There won't be any jobs left for humans.' All the evidence as of today is to the contrary. Every single piece of evidence. If anything, it's creating jobs. So they’re 0 for 1 on that.”
“I think they're having AI psychosis… I'm not sure they're dispassionate critics, observers of what's happening.”
Surely Anthropic’s IPO must be paused until the claims of this “whistleblower” can be investigated.
Narrative violation: According to the Economist, AI has created 1 million new jobs in the U.S.
The Economist@TheEconomistOur analysis suggests that AI has so far created around 1m new jobs in America. We explain how the technology has created a hiring boom economist.com/finance-and-ec…
I’m not surprised to see social media influencers coming forward to say they were offered money to push doomer messages about AI. These are well organized, well funded campaigns.
The political debate over AI is shifting from accelerationist vs doomer to decentralized/open vs centralized/closed.
Speaking at the G20 Ministerial this week with @mkratsios47, I made two points with respect to AI data centers. First, when done right, data centers lower electricity costs and bring significant economic benefits to local communities. Second, the choice remains with the communities themselves. In Executive Order 14365, President Trump established a national framework for AI but explicitly carved out data center infrastructure as a state and local decision. Despite some fake news to the contrary, the federal government is not preempting the states on this front. Under President Trump, this remains a matter for local communities to decide — and as the President noted this week, those that choose to embrace them will see lower taxes, new jobs, and historic growth.
It’s great to see Nvidia supporting open-source AI in a big way. Keeping innovation decentralized and accessible is the key to avoiding an unsafe and dystopian future where advanced AI capabilities are centralized and controlled by only a few hands.
Jensen Huang@JensenHuangExciting day for NVIDIA and @huggingface. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. They allow every developer, startup, university, industry and country to build with, customize and benefit from AI. Thank you @ClementDelangue for coming to me. NVIDIA is going to be a great home for Hugging Face, its community and the future of open models. 🤗 t.co/q8Om2Xc5ye
Thanks to Trump Accounts, every child will become a direct owner in the American economy. I hope every AI company will follow the lead of @Gwynne_Shotwell @SpaceXAI and give American kids a vested interest in their success. This would do much to improve the public’s image of AI. Kudos to @altcap @SenTedCruz @MichaelDell for creating this program with the full backing of President Trump, who signed it into law. I participated in their inspiring event yesterday, with dozens of CEOs expressing interest in making contributions.
The “AI capex is a bubble” and “SaaS is dead” narratives getting shredded this morning. NVDA +8%, SFDC +20%. NVDA: $96B q2 revenue (+106%), ~$60B net income, 75% gross margin. Highest core-business quarterly profit ever. Guided FY28 rev +70% vs Street exp 45% — and that’s a supply-constrained number. SFDC: bookings reaccelerated (fastest in 4 years) and Agentforce is showing up in ARR. Benioff: “the UI is the AI.” They’re putting Salesforce inside Claude. If you can’t beat ’em, join ’em. Still need systems of record. Congrats @JensenHuang @Benioff
President Trump is the best ally that innovators have in Washington.
President Trump was ahead of the curve requiring AI companies to build their own power generation so new data centers don’t raise electricity prices for residential ratepayers. Done right, data centers actually lower prices by producing excess power and funding grid upgrades. They also pay for better schools, more social services, and lower property taxes. President Trump is unique in his ability to stand up to media hoaxes. The data center hysteria will pass, and he’ll be proven right again.
Harvey is a great example of how American companies are building world-class specialized models: they took an open-source base (Kimi K3), post-trained it on legal data, and delivered state-of-the-art performance on legal benchmarks at a fraction of the cost of frontier models. Restrictions that kneecap open models would do nothing to stop Chinese labs from shipping the next Kimi. They would, however, cripple the ability of startups like Harvey to create high-performance, low-cost vertical models. Of course some of the closed labs would love this — it eliminates their competition.
Harvey@harveyIntroducing Tenet, our first model post-trained for legal. Tenet is a Kimi K3 base that we post-trained with @FireworksAI_HQ on a corpus of publicly available legal data, synthetic data, and human expert data simulating long-horizon legal work. Training increases Tenet's all-pass rate by 82% on LAB and 22% on LAB Contracts relative to the Kimi K3 base model. It achieves state-of-the-art performance on LAB Contracts and places second on LAB. These gains generalize to other leading agentic benchmarks including @mercor's Apex Agents - Corporate Law, @crosbylegal's Redline Bench, and @scale_AI's Professional Reasoning Bench. Tenet is also optimized for token efficiency, operating at less than a fourth the cost of leading foundation models. We additionally post-trained three specialist models for Tenet to use as subagents: 1) M&A Diligence: post-trained with @baseten on our LAB Diligence environment in an RLM harness, this model is optimized for high-scale, long-horizon tasks. 2) Review Tables: trained with @appliedcompute on our Review Table environment, this model is state-of-the-art and cost-effective at high-volume document review and structured data extraction. 3) Firm Knowledge: trained with @EngramLab on our synthetic law firm environment, this model is optimized to learn and search over a firm's knowledge via memory and structured notes. More details on model training, environment design, benchmarking, results, and more in the article by @gabepereyra below. What's next for Harvey’s research? - Scaling LAB to more jurisdictions, practice areas and workflows - Scaling compute to bring new generalist models and capabilities to Harvey More to come soon.
Some thoughts on Dario’s post: 1. Dario does not actually address Gavin Baker’s account of what he said – something he could easily deny if it were inaccurate. 2. Dario claims his critics live in a “bubble” where all regulation equals regulatory capture. He calls this an overly simplified view and notes that “Many people outside this bubble think of regulation as something that constrains corporate power and benefits ordinary people.” This argument is a straw man. Of course treating all regulation as capture would be overly simplified – but almost no one holds that view. I have repeatedly argued for strong antitrust enforcement to keep industries competitive, especially Big Tech. If Anthropic continues toward monopoly or duopoly status, I would be among the first to demand those rules apply. 3. Regulatory capture is not vague or in the eye of the beholder. Nobel laureate George Stigler defined it as regulation acquired by an industry and designed and operated primarily for its benefit. Stigler challenged the traditional view that government regulation arises from a benevolent state protecting the public from market failures. Rather, industry groups have concentrated stakes and pour resources into influencing regulators, whereas the public’s stake is diffuse and unorganized. The revolving door between companies and the agencies that regulate them compounds the problem. Anthropic understands these dynamics: it has hired multiple senior Biden AI-policy officials and built a substantial government-affairs operation plus a network of aligned organizations to push its preferred frameworks at state and federal levels. 4. Dario has consistently pushed for a new federal agency to review and approve frontier models prior to release – a proposal framed variously as an “FDA for AI,” an “FAA for AI,” and most recently a “FINRA for AI.” I call it a “DMV for AI” because a review process modeled on the FAA or FDA (which takes years) or FINRA (which issues rules for a staid industry widely seen as protecting incumbents) will create long queues as AI models wait for testing and approval. This process will only become more labyrinthine as rules accumulate to prevent theoretical harms. This would handicap the U.S. relative to China, which will not adopt the same constraints. It would also undermine Anthropic’s own business model, whose pricing power depends on remaining ahead of open models. Whatever Dario states today, it is difficult to believe the company would simply accept outcomes that erase that advantage. 5. Anthropic is on track to become one of the most valuable companies in history, with the resources to navigate any approval process and shape the rules while competitors wait. Dario wants open models under heavier scrutiny – he has called them dangerous in Senate testimony, criticized them for not being centrally monitored or withdrawn, and linked them to IP theft. He says he has never sought a ban, but he could achieve a similar result by insisting that identical rules apply to both open and closed models. The U.S. risks becoming an island of costly closed models while the rest of the world races ahead with broader choice. 6. Dario acknowledges that AI is structurally centralizing but attributes this mainly to chips and scaling laws. Access to compute matters, but the deeper risk is who decides which capabilities are available to whom. His preferred pre-deployment testing and FAA/FINRA-style oversight would place that gatekeeping power in a federal bureaucracy working hand-in-glove with a small number of frontier labs – reinforcing centralization rather than countering it. 7. The second part of Dario’s post assumes we have amnesia about Anthropic’s well-orchestrated campaigns hyping AI fears. His May 2025 claim that AI would wipe out 50 percent of entry-level knowledge jobs within five years still lacks supporting evidence fifteen months later. Similarly Anthropic breathlessly promoted its heavily contrived “blackmail” study on 60 Minutes. Yet Dario blames public negativity on a long-standing loss of trust in institutions rather than his own messaging. 8. These narratives have done more than anything to shape public fear. People are left asking the same question Mark Zuckerberg posed: why race to build a future you describe in such negative terms? Thomas Sowell’s "The Vision of the Anointed" captures the mindset – elite intellectuals convinced that only they are enlightened enough to control the outcome. As Zuckerberg notes, concentrating power in the hands of an enlightened few has rarely produced the promised results; the practitioners turn out to be less enlightened in practice than in self-conception. 9. Gavin Baker summarized the disagreement cleanly on our pod: Dario believes frontier AI is too powerful to distribute; we believe it is too powerful to centralize. Dario appears to believe, sincerely, that safety and progress are best served by centralizing authority in a marriage of corporate and state power. The weight of human history gives us reason to fear that outcome.
Dario Amodei@DarioAmodei1/2 Thanks Gavin for an especially thoughtful exchange. I don't usually spend much time on social media but I wanted to engage here because it really brings out the heart of an important conversation. First, on regulation, I think that “either concentrate it in the hands of a chosen few companies and politicians via regulation or distribute it widely” is a false choice. I know that there’s a sort of Silicon Valley shorthand where regulation = regulatory capture = concentration of power, but I’ve always found this to be an overly simplified picture of the world. Many people outside this bubble think of regulation as something that constrains corporate power and benefits ordinary people. I don’t necessarily agree with that perspective either, rather I think it’s complicated and really depends on what the “regulation” consists of. But in particular I think that those in the “regulation = regulatory capture = concentration of power” frame often underrate the decentralizing power of objective and fair institutional processes. A crude analogy is that the formal court system can sometimes feel stuffy and elitist, but it does a much better job of defending the rights of vulnerable individuals than the alternative, mob justice. At their best, institutions can vest power in ideas rather than people, and thereby decentralize that power. This is why Anthropic has always made its policy proposals very carefully. We try very hard to make proposals that disadvantage (slow down) frontier AI companies while *advantaging* smaller competitors. California’s SB53 (which we supported), and even the much-maligned SB 1047 (which we were ambivalent on), completely exempt any company below a certain amount of revenue or model training costs from being covered at all (it was $500M for SB 53, lower for 1047 but we objected to that). More recently the testing process we’ve advocated for at CAISI and the White House involves more rigorous tests for frontier models than off-frontier models — something that differentially advantages challengers. Similarly, the “Pacing the Frontier” letter envisions (or at least Anthropic’s preferred implementation of it envisions) modulating the pace of the very best models while not constraining those who are catching up. This hurts the business interests of the frontier labs and helps challengers, including open-weights! Overall my view is that AI is *structurally* a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws). Open-weights do help some with this but are nowhere near a sufficient solution because they simply shift the concentration somewhat to those with the most compute and chips (which are roughly the frontier labs plus maybe hardware providers). By contrast I think the right “rules of the road” can simultaneously (a) address AI’s cyber/bio/alignment risks, (b) institutionally constrain the power of the frontier AI companies, and (c) leave room for open-weights models while also addressing the specific risks that they bring. BTW I do not think that the events of the last few months have “failed to result in [my] preferred regulatory path”. The approach that the Trump administration is reported to be taking — pre-deployment testing for frontier models, and also testing of open-weights models when they get closer to the frontier — is one that I am very supportive of, though of course I have to see the details to be sure. I am also supportive of Demis Hassabis’ ideas around a FINRA-like entity. This contrasts with six months ago when most of the industry was still pushing for preemption of all state regulation and no apparent federal approach either.
Some people are saying x.com/chetanp/status…
Mark Zuckerberg gets it right: “The defining question of our age isn’t whether superintelligence will exist, but who will have access to it. Will it be centralized and restricted to a few institutions, or will it be a tool that empowers everyone?” Concentration of power is the biggest risk of AI. When a small number of labs (working hand-in-glove with the administrative state) decide who has access to which model capabilities, they inevitably shape what can be said, known, and built. That’s not “safety.” It’s control. As Mark points out, the history of open source shows that broad access and transparency are usually the best path to actual security and resilience. Decentralization creates checks and balances on power. By contrast, centralized alternatives, like bureaucratic approval regimes and mandatory gatekeeping, typically produce regulatory capture and reinforce cartels. Personal superintelligence in everyone’s hands, with competing models and real data sovereignty, is a far better check on a dystopian future than self-appointed guardians who claim to be “aligned” with all of humanity.
Anthropic maintains that it is entitled to train for free on all the world’s output, even if the author objects. But if a competitor trains on Anthropic’s output after paying for it, that is IP theft. The hypocrisy is breathtaking. x.com/itsolelehmann/…
Narrative violation. WSJ today.
The entire tech industry (save for Anthropic) has come out in favor of open source AI. So what happens next? Will Anthropic change its lobbying efforts? Not likely. Now the gaslighting begins: “Nobody is trying to ban open source.” “We just want to limit who can use it.” “We just want to limit who can contribute to it.” “We just want to limit how powerful those models can be.” “We just want to make sure the guardrails (we lobbied for) can’t be removed.” The net effect will be the same. They won’t stop until they kneecap open source. The rest of the industry needs to watch these guys like a hawk.
Nobody is saying that all software has to be open source. What they’re saying is that open-weight AI should be allowed. By implication, they’re rejecting your company’s incessant machinations to kneecap the open model ecosystem. Read the room. x.com/mononofu/statu…
Strong first post! x.com/JensenHuang/st…
It’s true that Anthropic is the fastest growing company that Silicon Valley has ever seen. Which is all the more reason their incessant attempts at regulatory capture are not just unnecessary but frankly gross. x.com/benioff/status…
Secretary @howardlutnick is right. The Kimi Panic needs to stop. — American frontier models are still ahead. When you factor in what’s in the lab, the gap is even larger. As long as we keep releasing, we will stay ahead. Let our horses run. — As Ben Thompson showed, Kimi’s apparent cost advantage largely disappears once you account for higher token usage and the real cost of running a model this size. Open weights still require expensive infrastructure. — Anthropic and OpenAI are growing revenue at rates that Silicon Valley has never seen before at this scale. This remains the clearest test of who is winning the market. President Trump’s light-touch regulatory approach is working. We should remain confident in American innovation. As long as we don’t sabotage ourselves with unnecessary rules, the U.S. will continue to win.
Cernovich is right that AI + Big Tech could easily turn into Big Brother. But what’s the alternative to living off the grid or in the Panopticon? The third option is to run your own AI on your own hardware, retaining data sovereignty. That’s why open source matters. It’s software freedom.
Great news! @patrickjwitt will be staying to take Clarity across the finish line. x.com/patrickjwitt/s…
Thank you for all your hard work @HarryYJung! It’s been a pleasure working with you. x.com/harryyjung/sta…
Kimi K3 just fixed 15 critical security bugs that Codex and Fable refused because of “cyber guardrails.” There’s no reason to limit American models on tasks that Chinese models handle without issue. We’re only making ourselves less competitive. x.com/callebtc/statu…
Here’s another example: Hugging Face tried using American frontier models to analyze an AI-powered cyber attack. But the guardrails blocked requests containing real exploit payloads so they switched to GLM 5.2 running locally. The guardrails actually impaired defensive security. x.com/clementdelangu…
I’m not sure whether Dean Ball is confessing to a regulatory capture strategy or simply predicting this will happen (he now says the latter). Either way, the weaponization of regulatory uncertainty as a competitive tool should be completely unacceptable. He argues there’s no need to ban Chinese open-source models — just direct agencies to issue soft-law warnings that create enough FUD so regulated enterprises back off. “It needn’t be that well justified.” Wrong. Regulatory decisions should always be well justified and grounded in facts, logic, and evidence, not the deliberate exploitation of fear and uncertainty. Implementing a surreptitious policy through manufactured doubt — rather than strong and explicit justification — corrodes the rule of law and invites future abuse against anyone. We are at a critical inflection point in AI policy. The leading closed labs, already a duopoly in terms of AI model revenue, want the government to eliminate their open source competition. They have laid their cards on the table. It is time for the rest of Silicon Valley — the vast majority that still values open competition — to do the same.
OH: “i’ve switched to Kimi from claude for a bunch of work. it’s just so much more fun because it just does the thing instead of lecturing you” Woke lobotomized models are the enemy of American competitiveness.
This is *exactly* what I predicted would happen. I said Chinese models would have advanced cyber capabilities within a matter of months and the only thing to do about it was to use AI-powered cyberdefense to protect our systems. Trying to gatekeep models doesn’t work. x.com/scmallaby/stat…
This is concerning. For the first time, a Chinese model Kimi K3 has taken #1 on the Frontend Code Arena and is scoring at or near the frontier on other benchmarks. Meanwhile America is tying itself in knots: politicians and bureaucrats are banning new data centers, piling on state regulations, and pushing for new federal agencies to pre-approve frontier models. This is how you lose the AI race. The rest of the world won’t play by our rules if we bog ourselves down. Permissionless innovation is how America won the internet and became the technological envy of the world. We can do it again with AI -- while addressing risks in a targeted way -- or we’ll watch our lead evaporate.
UPDATE: “Artificial intelligence giant Anthropic is pursuing a strategy of one-upmanship that encourages states to impose increasingly tougher AI guardrails, rather than align around a single set of regulations.” Politico 7/15/2026 politico.com/news/2026/07/1…
Legacy Media types are calling this Alex Karp interview a “crash-out” so that’s your first clue that he is actually saying something extremely insightful. He is articulating what real “AI safety” looks like in the enterprise. Not abstract alignment research or certification by a government-run DMV for AI. Real AI safety for businesses is the ability to control their own data, model weights, and compute — so a frontier lab can’t hoover up their proprietary knowledge and turn it into their next product. As Karp explains, technical customers want “control over their compute, their models, their data stack, and their alpha. They want to know they own the means of production, and it’s not being transferred to someone else.” Don’t think that can happen? Just look at Figma. According to The Information, Anthropic “blindsided” its then-business partner with the launch of Claude Design. Figma’s founder said Anthropic had not been “consistently honest” with them. Anthropic’s chief product officer had even served on Figma’s board until three days before the launch of Claude Design. Figma’s stock has fallen sharply this year while Anthropic’s valuation has surged. This isn’t an isolated example. Anthropic has launched Claude Science, Claude Security, Claude Legal, and of course Claude Code — each expanding into categories previously served by companies building on top of their models. The pattern is consistent: watch where value is being created, then move in directly. Dominate the model layer, then use that position to capture the most lucrative verticals. Dario has argued that open source models powerful enough to compete with Anthropic are “dangerous.” But dangerous to whom? Not to enterprises that want to retain control over their data and workflows. Dangerous to a business model that benefits from customers having few real alternatives at the model layer. As Karp exposes, true enterprise safety isn’t trusting that a lab’s future roadmap won’t include your business. It’s retaining the ability to choose — at the model layer — who gets to see and use your alpha.
UPDATE: This is a fascinating post by @satyanadella building on Alex Karp’s point about AI sovereignty. Satya names it the Reverse Information Paradox: enterprises don’t just pay for intelligence with money — they pay again by feeding frontier models their proprietary knowledge, corrections, traces, and evals. That institutional know-how then compounds inside the provider. Karp diagnosed what technical customers actually want: control over their compute, models, data stack, and alpha — to own the means of production rather than have it transferred. Satya explains why the current regime structurally does the opposite, and what enterprises must do about it: establish a real trust boundary with private evals, proprietary learning loops inside the tenant, decoupled orchestration, and the explicit right to fine-tune on their own outputs. That’s how your alpha compounds for you instead of leaking to the model layer. x.com/satyanadella/s…
Narrative violation: A new study of 21,559 firms in the U.S. finds that “companies that adopt AI tend to grow faster following adoption”. “Firms making the largest AI investments grow employment by roughly 10% following adoption, while low-intensity adopters see no statistically significant change.” “Entry-level headcount rises 12% for high-intensity adopters.” “Gains emerge gradually and are broad across roles, including engineering, sales, administration, and customer service.” “The results counter predictions that AI adoption will lead to broad job loss.” The study is based on observed AI spending from Ramp card and bill pay data linked to Revelio Labs workforce records.
Sources: Study: ramp.com/data/ai-jobs-i… Charts: ft.com/content/8026ea…
A year ago, President Trump declared that America was in a global AI race and that the way to win it was to be pro-innovation, pro-infrastructure, pro-energy, and pro-export. President Trump was exactly right; we deviate from that strategy at our peril.
Today was the day that Gavin Newsom was supposed to save the tech industry by cutting a deal to kill the Billionaire Tax Act. Instead he came out as DSA-adjacent, and BTA will be on the ballot in November. See y’all in Texas! x.com/gavinnewsom/st…
Claude psychoanalyzing Dario is the AI slop I didn’t know I needed.
True. And this is exactly where an “FDA for AI” will lead. x.com/ns123abc/statu…
This brilliant lesson on communication by @rabois is why the AI leaders are failing. It’s not sufficient just to “speak your truth.” You have to communicate in a way that elucidates your audience. Convincing the public that your company is a menace obviously fails that test.
Yes
Some recent articles have created a misleading narrative that I did not take Mythos seriously or tried to downplay the cyber threat. This is based on egregious cherry-picking of my comments and (since the real target is the Trump Administration) needs to be corrected. When Mythos Preview first launched, I pointed out that Anthropic has a history of scare tactics, but then immediately went on to say that “we have no choice but to take this seriously” and that every CISO and IT department should move quickly to harden systems against AI-powered cyberattacks. Here’s what I said on the April 10 All-In Podcast (3 days after launch of Mythos Preview): “Anytime Anthropic is scaring people, you have to ask, is this a tactic, is this part of their chicken little routine, or is it real? With cyber, I actually would give them credit in this case and say, this is more on the real side. “It just makes sense that as the coding models become more and more capable, they’re more capable of finding bugs. That means they’re more capable of finding vulnerabilities. That means they’re more capable of stringing together multiple vulnerabilities and creating an exploit. “I do think that every company, or IT department, or CISO that is managing code bases should take this seriously and use the next few months to detect any dormant bugs or vulnerabilities and roll out patches.” I posted similar framing on X: On April 10: “The world has no choice but to take the cyber threat associated with Mythos seriously. But it’s hard to ignore that Anthropic has a history of scare tactics.” (With examples attached). On April 12: I noted that a growing number of people were wondering if Anthropic was the AI industry’s “boy who cried wolf,” and that the company would face a serious credibility problem if the threats didn’t materialize. These are the lines the articles highlight. They emphasize the “scare tactics” / “boy who cried wolf” critique while omitting the parts where I said the cyber threat itself was real and required immediate action. It is entirely possible to question a messenger’s track record while still treating the underlying risk as serious — and that’s exactly what I did. By the way, this view isn’t unique to me or even particularly controversial; highly respected tech commentator Ben Thompson recently made a similar critique about Anthropic. On April 30 I posted a more technical thread after tests by the AI Security Institute showed that GPT-5.5-Cyber performed similarly to Mythos: “Mythos is not magic. It’s not a doomsday device. It’s the first of many models that can automate cyber tasks (just like coding). … these models do not create vulnerabilities; they discover them. The bugs are already in the code. Using AI to discover and patch them will actually harden these systems. “The leap from pre-AI cyber to post-AI cyber means that there will be a big upgrade cycle. … it’s important that cyber defenders get access before cyber attackers. That process is already underway but needs to happen quickly.” My position remains consistent: We are on a shot clock until Mythos-level capabilities diffuse widely, including to non-U.S. / Chinese models. We need defenders to find and patch vulnerabilities before that happens. This requires cooperation between government and industry. Unfortunately Anthropic’s needlessly confrontational posture toward the Administration has distracted from that mission. Policy debates have their time and place, but right now tangible defensive action is what matters most. I hope everyone moves forward on that basis.
Narrative violation.
Full article: wsj.com/tech/ai/the-jo…
Happy Birthday President Trump! It’s been an honor to work for you. Thank you for your extraordinary leadership in Making America Great Again. Wishing you a wonderful 80th birthday filled with health, happiness, and continued success ahead!
I’ve had a number of conversations with folks inside and outside government about the current situation with Anthropic, and here is what I believe to be true: — As we know, Anthropic publicly released its Mythos class models earlier this week under the commercial name Fable. — Fable is Mythos with guardrails. But if those guardrails fail, then you’ve exposed Mythos and its advanced cyber capabilities to people who shouldn’t have them. (Keep in mind that Anthropic itself widely promoted the idea that Mythos was a cyberweapon and needed to be regulated as such. They asked for government regulation of Mythos and championed the guardrails on Fable. If there is a vulnerability — big or small — it is Anthropic’s responsibility to patch.) — A highly credible trusted partner of both Anthropic and the USG who was testing Fable came forward with a jailbreak of those guardrails. The Admin asked Dario to fix the jailbreak or de-deploy the model. Dario refused. — In their blog post, Anthropic defended its decision by saying the jailbreak isn’t serious. That is not what the trusted partner and the USG believe; nor is that kind of minimizing language consistent with Anthropic’s brand as the AI safety company. It’s difficult to fathom how they could claim a jailbreak allowing operability of a cyber weapon could be defined as not “serious.” — In the past, Anthropic has always said that safety must be top priority and taken super seriously. In this case, Anthropic prioritized the continued offering of the consumer model over safety. — In reaction, the Admin issued the export control. The Admin did this reluctantly. It’s been very surprised that Anthropic hasn’t wanted to cooperate with a reasonable safety request (ie fixing the jailbreak issue). Anthropic’s reaction is very much at odds with their branding and ethos as a safe AI research community. — The Admin’s hope now is that Anthropic remediates the safety issue, the export control is lifted, and Fable goes back into general release. The Admin wants all of this to happen as soon as possible. It is frankly bewildered that Anthropic hasn’t wanted to comply with safety requests that it previously said were its highest priority. — Those trying to misdirect and tie this action to the prior DoW/Anthropic issues are wrong. The Admin values Anthropic’s technical capabilities and feels that this issue, while serious, should be easily resolved. The ball is in Anthropic’s court.
If you were wondering what the "pause" was all about, Ben Thompson @stratechery has an interesting theory: "Late last week the Anthropic Institute released a new safety report warning about the danger of recursive self-improvement... I don’t think the timing is a coincidence. This is a company and leadership that has been honing safety-and-scaremongering-as-marketing-tactic ever since Amodei led the charge to close source OpenAI models because GPT-2 was too dangerous; it’s always fun to see the evolution of tactics, capabilities, and goals, and in this case publishing a widely-discussed report the week before you cite it to silently degrade your offering for potential competitors is impressive."