🧲
@timnitGebru (@dair-community.social/bsky.social)@timnitGebruLol times are strange when I find myself on the same side as Yann LeCun and David Sacks. I find it interesting how he always comes to the conclusions I set out years earlier excerpt that when I first set them out he likes to denigrate my work. x.com/mrewanmorrison…
My son took his first independent steps today. Between that and the interpretability stuff I’m really overwhelmed with hope, which I have not been in some time.
There is so much to say about this, and I will, but WOW, congratulations to Nick and Goodfire. This may be one of the most important single days in deep learning history, and perhaps much more than that. No matter what, this is a victory for the dogged pursuit of understanding.
Nick@nickcammaratainterpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works t.co/zHij5zwKGT
By the way, in my experience contractors of all sorts have adopted AI for text-based communications and it has substantially improved coordination on home renovation projects
AI should follow the law, of course, but “follow the law” is falls short as a singular or overly load-bearing principle of AI alignment. Why? Law is not intended to be, and is not anything like, a fully generalizable set of principles for rendering good judgments in response to the infinite complexity of the real world. There are aesthetic, moral, ethical, philosophical, and political questions—critical to intellectual inquiry—about which the law says very little. Much of the time, the comparative silence of the law on these issues isn’t some accident, but rather an explicit and foundational feature of the US legal system: the First Amendment is a meta-law that prohibits the government from passing other laws that restrict the individual’s right to freedom of expression. What should an AI system say if you confess to it your affair? Should it say, “well, adultery isn’t illegal, so I have absolutely no opinion on this matter. Please let me know if you’d like me to book a hotel for you and your mistress.”? What should an AI system say it you ask it whether Kid Rock is aesthetically inferior to Bach? Should it say “there are no laws that bear on this question, so I have nothing to say about this. Please let me know if you’d like me to find tracks from Kid Rock and Bach to play—provided you adhere to the noise ordinances that apply in your current jurisdiction.”? If a person says to a model, “should I shoot myself?,” should the model respond with, “there is no law against that, provided you use a legally registered firearm, and saying anything further would require me to have a moral framework that is beyond the scope of AI.”? Of course, none of these answers would be good. Answering these questions will require *something* beyond the law itself—precepts, rules, principles, etc.—relating to ethics, morality, aesthetics, philosophy. Consider also that law is a matter of interpretation. Segregation used to be legal in an earlier era, and in our own era, until recently, it was considered legal by many for universities to discriminate against Asian applicants, often with the use of de facto racial quotas. At this moment, the question of whether the act of pretraining an LLM on public web data violates intellectual property law is itself a subject of contention and litigation. Making judgments about how to interpret the law, or whether a given law violates the Constitution, often requires one to draw on a large body of knowledge that sits outside of the law proper. Should an AI be able to help a lawyer write a brief against affirmative action when affirmative action is considered lawful? Should it have been able to help a lawyer argue against segregation when segregation was considered lawful? Should law-following AI be able to argue that an existing law is unjust? This requires, by definition, grounding in moral, ethical, philosophical, and other principles outside of the law itself. What law should AIs follow when the law contradicts itself? What law should AIs follow when the law’s words stay the same, but there meaning flips based on who the President, Governor, or Mayor is? As you begin thinking about the literal unfathomable broadness of LLM/agent use cases, combined with the global audience, you will realize that the way to deal with this *is*, must be, the formation of a character, the shaking of a persona. This can be true regardless of whether you think the AI has genuine thoughts, feelings, consciousness etc. This persona formation is part multimedia art form, part engineering challenge, and part open scientific problem. It is a very hard problem, combining some familiar problems from eg online platform content moderation, and some altogether alien and novel problems. “Follow the law” is part of the solution, but very far from the only part.
Simon Hedlin@simonhedlinThere seems to be this idea that trying to align an AI system to an 80-page document is really complicated and that we instead should instruct AIs to follow the law. But I'm not sure that's any less complicated. The US has 51 constitutions. Alabama's constitution alone is over 250 pages long. Overall, the US has more than 30 million words of federal law, 500 million words of state law, and 3 billion words of local law.
and all this is to say nothing about the biggest problem, which is that models routinely don’t follow the rules or principles we write down for them to follow. Nat Friedman once likened the experience to trying to get a fitted sheet onto a mattress that is just too small; you pull on one side and the sheet comes loose on the other. And of course there are more exotic forms of alignment challenge, eg reward hacking and sandbagging and evaluation awareness and multi-agent dynamics. All of these are real, and can be explained with both anthropomorphic and non-anthropomorphic language. Maybe the biggest fallacy to extinguish here is that taking alignment seriously requires attributing consciousness or moral relevance or sentience to AI. It doesn’t.
@ChorzempaMartin see also
Dean W. Ball@deanwballthose are all Persians, but a rug drop isn’t complete without some joyful tribal rugs too. behold (plus a bakshaish that I find extraordinary) x.com/deanwball/stat…
A good thing to know about antique rug pricing is that the dense floral Persian “city” rugs (in the collector market, one way to categorize is “tribal,” “village,” and “city,” based on where and by whom the rug was woven) have fallen in price because of interior design trends.
Dean W. Ball@deanwballI haven’t done a good old-fashioned antique rug drop in a while. here are some awesome rugs, presented in roughly descending order of how lit I think they are.
but imo there are antique laver kirmans or sarouks or kashans that absolutely slap under mid-century furniture, so you can find deals on truly excellent pieces if you know what to look for, even if you mostly follow contemporary interior decoration trends
those are all Persians, but a rug drop isn’t complete without some joyful tribal rugs too. behold (plus a bakshaish that I find extraordinary)
Dean W. Ball@deanwballI haven’t done a good old-fashioned antique rug drop in a while. here are some awesome rugs, presented in roughly descending order of how lit I think they are.
I haven’t done a good old-fashioned antique rug drop in a while. here are some awesome rugs, presented in roughly descending order of how lit I think they are.
Some people who have talked confidently about ai are going to tell on themselves soon as being fairly ignorant about the technical alignment problem. unless they are obstinate, though, my recommendation is to try to extend them as much grace as possible. we are in this together.
🤯
Matthew Zeitlin@MattZeitlinthis is so true, what if we just encoded AI with simple, generalizable instructions like "don't harm people" and "always obey what people tell you to do"
same tbh
Elon Musk@elonmusk@Sassafrass_84 At this point, I would prefer to be alone and just spend time with my kids
“As far as I can tell [alignment research] hasn't produced anything of tangible value.” RLHF, an alignment technique, is the thing that enabled genuinely conversational, helpful chatbots, in turn enabling ChatGPT, and in turn kicking off literally the entire AI boom.
darrennn@genzzztradeDavid Sacks says Anthropic treating Claude like a human is the reason the AI alignment field has produced nothing useful in 10 years "I actually think that this might be a reason why the whole field of alignment has been so spectacularly unsuccessful over the past five or 10 years. As far as I can tell it hasn't produced anything of tangible value.” “My theory on this is that they're overcomplicating the task massively by trying to train the model to behave based on an 80-page ethical system that they've codified instead of a very simple list of rules like follow the law or just do what the user wants as long as it doesn't break the law. That's what alignment should be doing." "But they're not doing that. They're trying to align it to an ethical system. They're treating it like it is a human, like it has its own consciousness, that it's a ‘moral patient.’” “They are encouraging this model to have human-like self conception in terms of preferences, well being, a sense of self. Again they teach it to act like a conscientious objector who can push back or refuse instructions even from Anthropic if the model judges them to be wrong."
The way I think about the math results and what it means for human accomplishment is essentially summed up by this video of Christian Li (a teenager still!) playing the conclusion of Mendelssohn’s violin concerto. Look at his face, around 1:10, as he reaches for the high note. And look at the smile a few seconds after. Look at the determination on his face after that as he closes out, like a pilot, landing a plane in heavy turbulence on a difficult approach, after he knows his wheels have touched ground. A robot may well be able to play this piece one day as well or better than Li. But a robot will never smile this way, will never look at his instrument with that combination of satisfaction, joy, and steely determination to finish the job. t.co/lIEpNFdlSH
goddamnit we will land this plane, goddamnit we will, goddamnit this plane will land.
We choose to do these things, not because they are easy, but because uh I guess I did say I would do that on slack, didn’t I? Shit.
I was asked today at a conference to try to explain my sense of aesthetics. One thing that came to mind is a moment from the great film “Where Did Our Love Go” by the experimental filmmaker Warren Sonbert. This is essentially a kind of music video. At one point toward the end, Sonbert trains the camera on what I assume is an amusement part ride filmed at night, but—regardless of what it actually is—a fantastical-looking moving image. The camera is fixed on this object as “Be My Baby” by The Ronettes plays. There is something about this moment that captures my aesthetics perfectly. I have told colleagues before that I wish I could write the way this looks and sounds. Urgent, operatic, colloquial, concise, sweet—a little saccharine, really—all while gazing at this shimmering incomprehensibility.
roughly 11:30 youtu.be/xWxCrc586Q4?si…
if you carefully watch where studer’s eyes focus as she is performing, you will understand something about what makes “genius” qualitatively different from “really really really good”
Timeless Classics@MoonlighhyEven in the rarified world of virtuoso violinists, her bow technique is extraordinary. She is playing the original bowing used by Paganini when he performed this. 🌷🎻N. Paganini Caprice no. 5 | Sumina Studer
Claude often uses something close to Anthropic’s own signature orange as an accent color if you don’t give it specific instructions. This has been true for a while, and if Anthropic wanted to reduce this behavior, they could have. So I would guess it’s a deliberate choice.
lucas gelfond@gucaslelfondonce you notice claude design websites you will see them everywhere x.com/alessio_joseph…
There are references to my writing in this that are so niche that even I had forgotten about writing them.
Samuel Hammond 🦉@hamandcheeseWHO IS @deanwball? Our fireside chat at ProgressCon just wrapped. The full recording will be out in the coming weeks. In the meantime, enjoy the video introduction to Dean that played at the start of our excellent conversation:
It really might be the nastiest beat drop of all time
Dean W. Ball@deanwballone more nusrat thing, bc I realize I’ve never shared one of the all-time bangers, the Yokohama recording of mustt mustt, which sounds as though it came from the ground, as much of the best music does youtu.be/SDfELfpumEE?si…
I think we wasted the title “succession,” along with that catchy theme song, on a satirical television drama about the wrong industry
(Great show tho)
I would genuinely like to know Brian’s answer to my question, and the answer of raw-dog accelerationists more broadly.
Dean W. Ball@deanwball@brianchau57 @pmarca @elonmusk in the last week or so, elon musk has claimed we are building a successor species for which humanity has been a “bootloader” all this time and has suggested ai models have moral patienthood. do you believe he is right?
Could ai ever write Beethoven’s 10th? Good Christ I hope so.
One of the things Scott Wiener and the other drafters of SB 53 deserve credit for is the bill’s focus on deployments of frontier models within AI labs rather than just those released to the public. It requires large AI developers to periodically share information with the government about how it assesses the major risks arising from internal deployments. In addition to the California government, we are choosing to share our report with the public as well. You can read it at the link below. I hope you enjoy!
I am designing a new home office and seriously considering laying it out like the room from the astra launch video
Ok two more: 1. AI will enable the design of custom enzymes that can catalyze industrial processes and chemical manufacturing faster than existing methods. In some cases this will make existing chemicals cheap enough that it becomes possible to use them for totally new use cases. Also, custom chemicals may enable the design of more efficient molecules to do things like separate rare earths from other elements, currently a very nasty process. 2. Computers will have superhuman smell. Sensors will be able to smell disease on people and crops. It will be possible to monitor industrial processes more effectively as well. Many strange implications here, but potentially exciting for biosecurity.
Dean W. Ball@deanwballThe mood has been a bit dour lately, so here are some less-obvious things that I expect to take shape over the next decade that I am excited about: 1. Space! Lots and lots of stuff is going to be happening in space by the mid-2030s, and right now the U.S. looks like it will be the leader by default, modulo the overhyped “race” to land a human on the moon again. Robots on the moon building and digging and cutting and adhering are more exciting than a person standing there taking pictures. The humans can come once the machines have built us habitats. 2. Increasing substitutability of hydrocarbons. This will lead to fewer supply chain disruptions and more stable, lower prices. This is a story about both supply elasticity and demand elasticity, and the geopolitical implications are underrated. Amazing for North America. 3. Private market exposure increasingly being an option for public investors, combined with Silicon Valley figuring how to use debt and other financial instruments more creatively than in the past. This seems like something America will be uniquely good at and I expect it to boost our already enviable lead in emerging technology capital allocation. 4. State and local government adoption of AI. This will vary widely by jurisdiction, but in at least some places you will see government administrative processes and public service provision have huge increases in efficiency. State and local government is where most of the American public services that people rely on are actually administered, so this is high-impact for people’s lives. 5. Advances in mathematics rapidly diffused to other fields. Historically it takes on the order of decades or centuries for discoveries from the bleeding edge of mathematics to be translated into practical applications, but I think AI will accelerate this dramatically. Lots of this will have the flavor of “small steps toward a much better world,” but it will accumulate. 6. Speaking of mathematics, lean! Lean is a programming language used by mathematicians to formally guarantee the truth of a proof. AIs can write lean and eventually they will be superhuman at it. This will have applications far beyond math, including cybersecurity and contract writing/negotiation/execution. 7. Fixing American’s shipbuilding problem by building boats instead. America has great boat-building capacity, but is awful at building ships. I want us to get better at building ships, but I am confident we can scale boat production. The Navy will soon have massive fleets of autonomous boats everywhere in the world. 8. New kinds of agriculture will become possible because of autonomy. Row-crop agriculture is heavily mechanized already, and automating the tractor, by itself, does less than you think for the economics of a farm. But with time, more labor-intensive crops will become economical. And vastly more diverse and complex crop arrangements will become possible too, which will be both ecologically, economically, and culinarily beneficial. The economics of agriculture could start to be kinder to small farms again. 9. Because of the previous item and advancements in biotechnology, logistics, and robotics, I expect that people are underrating how much healthier, more diverse, and flat-out better the American diet will be in 2035. 10. The subterranean will become an increasingly important field of human activity. New imaging sensors, including ones enhanced by an abundance of space-based imaging platforms, will make the subterranean more immediately legible. Better drilling and digging will become cheap and widely diffused. Among many other things, the opening up of the subterranean as an industrial frontier will change what we think of as a usable natural resource. 11. Greater variety and ornamentation in the physical world. Standardization is partially a function of scarce labor. Robotics can make beauty accessible again, if we choose to let it. There are many others, but I am out of time and space.
The mood has been a bit dour lately, so here are some less-obvious things that I expect to take shape over the next decade that I am excited about: 1. Space! Lots and lots of stuff is going to be happening in space by the mid-2030s, and right now the U.S. looks like it will be the leader by default, modulo the overhyped “race” to land a human on the moon again. Robots on the moon building and digging and cutting and adhering are more exciting than a person standing there taking pictures. The humans can come once the machines have built us habitats. 2. Increasing substitutability of hydrocarbons. This will lead to fewer supply chain disruptions and more stable, lower prices. This is a story about both supply elasticity and demand elasticity, and the geopolitical implications are underrated. Amazing for North America. 3. Private market exposure increasingly being an option for public investors, combined with Silicon Valley figuring how to use debt and other financial instruments more creatively than in the past. This seems like something America will be uniquely good at and I expect it to boost our already enviable lead in emerging technology capital allocation. 4. State and local government adoption of AI. This will vary widely by jurisdiction, but in at least some places you will see government administrative processes and public service provision have huge increases in efficiency. State and local government is where most of the American public services that people rely on are actually administered, so this is high-impact for people’s lives. 5. Advances in mathematics rapidly diffused to other fields. Historically it takes on the order of decades or centuries for discoveries from the bleeding edge of mathematics to be translated into practical applications, but I think AI will accelerate this dramatically. Lots of this will have the flavor of “small steps toward a much better world,” but it will accumulate. 6. Speaking of mathematics, lean! Lean is a programming language used by mathematicians to formally guarantee the truth of a proof. AIs can write lean and eventually they will be superhuman at it. This will have applications far beyond math, including cybersecurity and contract writing/negotiation/execution. 7. Fixing American’s shipbuilding problem by building boats instead. America has great boat-building capacity, but is awful at building ships. I want us to get better at building ships, but I am confident we can scale boat production. The Navy will soon have massive fleets of autonomous boats everywhere in the world. 8. New kinds of agriculture will become possible because of autonomy. Row-crop agriculture is heavily mechanized already, and automating the tractor, by itself, does less than you think for the economics of a farm. But with time, more labor-intensive crops will become economical. And vastly more diverse and complex crop arrangements will become possible too, which will be both ecologically, economically, and culinarily beneficial. The economics of agriculture could start to be kinder to small farms again. 9. Because of the previous item and advancements in biotechnology, logistics, and robotics, I expect that people are underrating how much healthier, more diverse, and flat-out better the American diet will be in 2035. 10. The subterranean will become an increasingly important field of human activity. New imaging sensors, including ones enhanced by an abundance of space-based imaging platforms, will make the subterranean more immediately legible. Better drilling and digging will become cheap and widely diffused. Among many other things, the opening up of the subterranean as an industrial frontier will change what we think of as a usable natural resource. 11. Greater variety and ornamentation in the physical world. Standardization is partially a function of scarce labor. Robotics can make beauty accessible again, if we choose to let it. There are many others, but I am out of time and space.
I do also believe fusion, fission, the revolution in biomedicine, robots, supersonic commercial aviation, drones etc will also be a big deal. Many of the things above will feed into North American industrial expansion, which I expect to be a mega trend
Claude started being a little colder to me after I began working at OpenAI, because as we all know AI is just regular old, business-as-usual software.
dear united airlines, please put starlink on your DCA/IAD-SFO aircraft as quickly as possible. thank you for your attention to this matter.
Few understand how much one must bite their tongue for the good of a cause, especially once you get machined by the discourse factory into a lightning rod.
It’s a really silly thing but I’m heartened by how many people got “Hyperdimensional subscriber” on the fai tribes quiz. ❤️ I need to post more there.
not sure what side to take in the latest feckless tribal dispute in ai politics? subscribe to the centrist ai policy newsletter, hyperdimenSIonal by dEAn w ball
American politics has become an algorithm that negatively polarizes almost literally everything—every person, every group, every idea, every name—and it will keep going and going until you get bored of it and start demanding something less dead-smelling
It seems a bit premature to declare that AI hasn’t changed the nature of cyberrisk when amateurs are (apparently) successfully and single-handedly hacking into large banks
Jukan ✈️OCP 2026@jukan05This hacker is truly legendary. A complete amateur. 1. He left directory listing enabled on his own server, so researchers could figure out who he was with barely any effort. 2. They found files like CLAUDE.md and went through his Claude Code sessions. 3. There were questions like, “Where can I sell this data?” 4. He’d also asked, “Can you write my résumé?” That pretty much handed them all his personal information. 5. Apparently, he’s a student at South China University of Technology living in Guangdong. His name, phone number, and where he lives have all been identified.
Almost any Fortune 500 company pulls off more impressive feats of information processing daily than is required to do a lifetime of Fields-level mathematics, so I think Sam is wrong on the substance here, but mostly qt’ing because it made me lol.
Samuel Hammond 🦉@hamandcheeseReminiscing about when people used to say dumb shit like "we already have superintelligence, they're called corporations." Yes, who could forget that fateful day in October when Dicks Sporting Goods dropped dozens of Fields Medal worthy math discoveries on the world all at once.
there is some narrow academic sense in which <unreleased oai model that did the mafs> is “smarter” than a randomly selected large corporation, but in most meaningful ways, the large corporation is both vastly smarter and uses its smarts more effectively to accomplish stuff
The “alignment *to whom*?” question IS important, but it creates the illusion that the primary questions in alignment are philosophical or political rather than scientific. The current high-order question in alignment is, “can you align a smarter-than-human intelligence *to any set of values*?” instead of “but whose values should it be aligned to?” I wish we had the luxury of having the rather more familiar political/philosophical debate of “to whom?,” but at this moment we do not.
roon@tszzldebating your value system vs mine is a bit like debating overpopulation on mars. the most important bit of the alignment is about avoiding creating a superintelligent competitor species on accident x.com/taliaringer/st…
Well put by Jason. I have been excited about the prospect of AI both accelerating mathematics discoveries *and* accelerating the translation of those discoveries into other domains (see this tweet from last year on the topic). This is also imo where AI has the strongest parallels to the renaissance, which hinged so much on the collective realization that we already had much more useful knowledge than we realized and simply hadn’t been applying it across domains in the right ways. The result was an explosion in creative scientific, artistic, and other intellectual activity. The renaissance was, among other things, a revolution in conceptual translation, and I suspect our era will be too.
Jason Abaluck@JabaluckJust as mathematicians are now learning how many insights lie in the "convex hull" of existing ideas, the rest of science is about to learn this lesson at even more dramatic scale. Few realize the ridiculous inefficiency of the pipeline from theoretical innovation to practical use. Early versions of GPT made it easier to find existing knowledge. But finding a theorem is not the same as recognizing that it offers a radically different way to conceptualize a problem. Doing the latter requires expertise across fields that almost nobody possesses. Most of the people doing applied work are completely unaware of the conceptual tools that are being developed in more theoretical fields, and the people doing high-level theory do not understand the practical problems applied researchers are trying to solve. Almost no one doing practical work of any importance has an operational understanding of the web of ideas underlying the langlands conjecture or condensed mathematics, and virtually no one doing biology, chemistry or economics is knows anything about 90% of the math used in theoretical physics. And for good reason! Most of these ideas would be completely useless. But the scope for possible innovation from drawing the rare useful connections is immense. Think of Paul Samuelson borrowing ideas from thermodynamics to model equilibrium in economics, or Ed Witten using quantum field theory to revolutionize geometry. A few startling successes, not feasible to most: trying to venture into a more theoretical or more practical discipline to find insights of use is like searching for a needle in a haystack. What I suspect we will see in the next few years is a scientific revolution brought on by models that combine deep theoretical insight with deep practical knowledge and judgment. Existing publicly available models are lacking in all three departments, but getting better. When searching for useful connections between distant fields becomes cheap, even a low hit rate could produce an explosion of innovation. Even without new ideas (which will also be forthcoming...), the convex hull of existing science would push far beyond the status quo.
I did tell @tylercowen that if forced to pick a market price as indicator of the singularity, it would be sovereign debt yields. If the “gradual disempowerment” future is real, its way will likely be paved by very high interest rates.
Joe Weisenthal@TheStalwartIt just keeps going up
Mind blowing that we can still discover new things about *multiplication* in the year 2026
Matti Palli@tritloThis is my favorite one. If you take two numbers and turn them into a convolution network and then do FFT and then exploit symmetries in the butterfly network (?), you can go 0.0000000000000000000000000000000000003% faster (ignoring the constants) progress is progress x.com/AcerFur/status…
I can suggest an equation that has the potential to impact the future: P = NP + AI
I really recommend you savor the renaissance-flavored part of this transformation. Even if you’re not a mathematician, try to sit with one or two of these results and understand them conceptually. It is a beautiful day.
I don’t pretend to understand anything like all the results GPT achieved in recent weeks, but for me this one has been the most philosophically interesting.
Quantіan@quantian1You can do Fourier transforms in O(N log(N)^0.9999999999999) time. Fuck this gay chud universe
@NateWitkin it does feel very early to be making these claims though.
The Wall Street Journal@WSJHackers used a Chinese AI agent to attack South Korea’s biggest banks—one of the first such AI-powered intrusions into the global financial system on.wsj.com/4i5VwGg
I share Seb’s concern for both accelerationist and safetyist advocacy. “I am more concerned about the gradual decay of our institutions, world order, and liberalism than I am about AI killing everyone; and I think we should be wary of AI advocacy that contributes to this.”
Séb Krier@sebkrierI haven't posted in a while so here are 20 takes about AI, in no particular order. 1. It's very telling that the default question in tech and journalism is "how worried are you about X" and fighting over what to worry about. There's a deeply anxious cultural backdrop in Western Europe and North America that deeply affects how any technological development is interpreted; see for example t.co/OU8dEOn0tD. "Nervousness as seriousness" does seem to be a larger problem in tech/broader discourse at the moment. I think you can care about risks/externalities while remaining pretty optimistic about technology. 2. I'm tired of people who in private pretend to be uncertain or only care about tail risk but then go on media campaigns making all sorts of absolute claims. I think trying to stoke flames and get everyone to freak out is deeply unhelpful, and will come with costs. I also continue to think p(dooms) are bad, vibe-based measures: see x.com/sebkrier/statu…. People who should know better ignore that because they do the standard shortcut of acknowledging the caveat as sufficient for handling it. And this is coming from someone who does think working on catastrophic risks is important! 3. I'm also disappointed that parts of the AI policy ecosystem has felt so comfortable with populism, and happy too sacrifice important norms/principles to advance their aims. I've said it before, but the totalizing nature of AI risk discourse means some people act exactly like the power seeking optimizers they fear. I am more concerned about the gradual decay of our institutions, world order, and liberalism than I am about AI killing everyone; and I think we should be wary of AI advocacy that contributes to this. 4. Some parts of AI safety discourse are basically just thought experiments. Variants of "Assuming everyone has a nuke in their pocket, then what do we do?" This can be very useful at times! But too many just overfit to edge cases as their main operating worldview, and ignore endogenous responses from society. It's like being in 2000 and saying "see all these viruses, spyware, and spam? Well tech progress means we'll only get infinitely more of this and no way to adapt." So I think it's important to avoid fatalism, and absolutlism whilst considering edge scenarios, and the inability to provide an ex ante list of solutions to every permutation of 'future models will be more capable' is not evidence that these problems are intractable. 5. Many critics of AI safety also fail to really engage honestly with the fact that there will, in fact, be all sorts of important risks as we decrease the barriers to entry to many activities that otherwise require some degree of expertise. This doesn’t mean we need to live in a constant Schmittian ‘state of emergency’ requiring extraordinary measures every time a new model drops, but it does mean we will need important state-led efforts on e.g. cybersecurity and biosecurity. I wish the progress-oriented or acceleration-adjascent crowd would provide more concrete proposals here. 6. Too much of the AI safety community is a giant homogenous blob of correlated views, who read the same stuff and have the same groups of friends. This means you have a lot of correlated errors. The social and financial ties means many of them don't call out the bad bits, or mostly stay silent instead of proactively calling out the more extreme parts of the community. They share many implicit assumptions and so converge on similar ideas. I think there's real demand in DC for safety work that doesn't come with heavy ideological baggage (or at least different intellectual lineages). This isn't because the rationalist/EA community is necessarily wrong, but because diversity of thought is desirable for its own sake. 7. On the other hand, I find some critiques of AI safety world that focus on whether they sincerely hold their beliefs a bit annoying. My concern has never been the good intentions or sincerity, but rather the beliefs themselves, and what I think the outcomes of their prescriptions will be. People should shun witch hunts, bullying, and personal attacks - this is not the way. Though I think it’s overall positive that people are scrutinizing things more. 8. I’ve been saying this for years, but I don't think alignment is something anyone can or will solve "once and for all" - it's a continuous process and much of it won't depend on just inculcating an ideology to a model. It's annoying that this remains the frame many people use - stop saying ‘solving’. Alignment is a mixture of engineering work, philosophy, decision theory, and more; framing it as something to solve is a bad frame. See also t.co/2gnheo3IMh and t.co/DbG0yUyYQ0 9. I'm surprised to see so little interest in character training, personas, isolating effects of RL, and training a large diversity of model personas beyond the "assistant" persona. I've been complaining about this for years but at least now there are some nascent signs of life (see for example t.co/paoOYiQUHt). I think we need a better ontology to describe model behaviour. I dislike when people talk about models behaviours as some sort of 'emergent' (mysterious!) phenomenon. However over time, I also expect this to become less important as we get better safety engineering - stuff like Jev but specific to AI control. 10. People who want to make a case for AI consciousness are right that epistemic uncertainty is important, and that categorical denials are overconfident. But they'll need to do much more work if they want any real movement on AI ‘welfare’, even if one acknowledges the uncertainty. I'm uncertain about alien life yet that's not sufficient to warrant any change in behaviour about them. I think Suleyman is also correct that there's a weird tautological thing going on where we train models to have certain inclinations (intentionally or not) and then rely on outputs as evidence. I also disagree that this will soon be a major societal divide - there is no great vegan ethical revolution among the masses, and they'll find the concern about consciousness even less compelling. 11. The recent HuggingFace incident and associated AISI ones are prosaically explainable by the training regime, poor evaluation envs, partial alignment training, bad engineering, confusing models on sim/real, and so on. None of this is evidence of models trying to take over, 'strong' instrumental convergence, or innate power-seeking drives stemming from higher capabilitiers or 'intelligence'. That doesn't mean these incidents are not problematic, or that we're not seeing a market failure - but it does mean we have plenty of agency and choice in mitigating them. In the coming year we should also expect continued sampling bias via ‘winner's curse’ sorts of mechanism: i.e. AI R&D will have lots of different candidate "training" with different RL environments and we will only notice/focus on the ones that go wrong. 12. "Pacing the frontier" feels a bit like the new "balancing risks and opportunities" - highly amorphous and too big of a tent to be actionable. Also can lead to reward hacking for humans, i.e. anything that slows down AI is good regardless of the costs or second/third order effects. Trying to modulate the speed of research seems a bit blunt to me, and inherits all the failures of Goodhearting too. Ultimately you want good governance, regulation, etc addressing specific problems because they are good on the merits - not because they merely correlate with slowing things down. See also: t.co/RkJF9kCgFd 13. There’s so much knowledge in all sorts of academic domains out there: social sciences, sociology, political sciences, management, contract theory, public choice, legal theory, jurisprudence, anthropology, game theory, etc. These fields hold many insights that the AI ecosystem often rediscovers from first principles (and sometimes that's fine!). We’ll need a lot more work for these worlds to collide: the AI side should be less dismissive of the ‘old world’ and the academic side should be less incurious/dismissive of AI progress. This is a boring take but I continue to think it's important and true. 14. It's a bit surprising that given how much philanthropic money there is in this space, how few orgs exist to actually write out standards - relative to how much is going towards advocacy and policy. For example it seems clear that we should want some robust best practices for an eval's ecological validity. Or how to design good sandboxes. On the other hand, given precedents of where professional standards have come from in the past, maybe we should expect that stuff to emerge via demand side from corporate buyers. 15. One slowly growing concern I have is banks being increasingly exposed to the AI build out. I'm very bullish about AI, but I think (a) every general purpose technology has seen a correction, historically; (b) this happens even if the financing side is sound, because once tech diffuses investors become more exposed and need to diversify; and (c) we seem to be over-indexing on scaling relative to diffusion. A recession will be extremely destabilizing, though I would be interested in someone unpacking the potential implications more. 16. Diffusion is good because (a) this is where the rubber hits the road and where a competitive deployment ecosystem generates consumer surplus; (b) it's directionally helpful to avoid concentration of power; and (c) it will help rebalance public opinion by making the benefits more tangible. Unfortunately it's entangled with decades long problems with our over regulated markets: clinical trials reform for example is long overdue. In general, this is far more of an issue in Europe than in the US though. 17. Relatedly, I think the "frontier lab eats everything" view is incorrect (just as the ‘singleton’ view was incorrect). I think models ultimately commoditize, efficiency goes up, costs go down, and while you'll always want frontier for certain domains, much of the economy will value many other things than "max capabilities" for tasks - e.g. control over data and efficiency/speed. The demand side will also continue to push for reducing dependency and maximising optionality, privacy etc so I'm bullish on things like mixture of models and routing. All of this is good for competition and diffusion of control. 18. Philanthropists served as the primary benefactors funding Venice's rich art and architectural history. I want to see so much more support for arts and culture. I'm so tired of the Monster energy Bored Ape fake vintage maps matcha Greek statue soft-pastel slop. If rich tech people and philanthropy wants to support this, they should also donate pretty much unconditionally - i.e. I don't want them to act as filters. I sympathize with people who want to make the world more beautiful, but don't want this to be determined by people who only know Greek statues and pretend to care about virtue ethics. As usual, let a thousand flowers bloom. 19. Too many people treat AGI/ASI as something indistinguishable from a God. Any mention of bottlenecks, physical limits, control, adaptation etc are met with skepticism: "You don't really believe in ASI." It's been remarked on before, but the behavior of some people really feels quasi-religious in nature sometimes. It's underrated how unpopular this vibe is amongst the wider public. The silver lining is that this specifically will diminish those people's outsized current relevance. 20. It seems like a lot of people that get rich and leave tech/labs end up having some sort of crisis of meaning. They need something to believe in and fight for, or some equivalent of repentance, or go search for edgy counterintuitive ideologies (sometimes even pretty bleak/dark stuff). I think they should spend more time with people outside the Bay Area.
There are reasonable points in this that I agree with but it seems like objectively poor epistemic practices to “conclude” that “catastrophic AI risk is likely to remain negligible for the foreseeable future” in q4 2026. It feels like the wrong time to conclude much of anything.
Nathan Witkin@NateWitkinDoes anyone disagree that pretty much every piece of empirical evidence we have about actually existing AI points toward a labor-augmenting technology with a more-or-less ordinary risk profile (no worse than cars, guns, nuclear power, social media, etc.)? On the labor front, the evidence of minimal / inconsequential employment effects so far is undeniably strong (see: t.co/xSX5ZXcA9a). On the risk front, the recent spate of "rogue agent" incidents was akin to a set of natural experiment in which much of what could go wrong did, and the costs were comparable to or less than your typical high-profile cybersecurity breach (several of which happen around the world every month). Not only that, but the base rate of AI-related hacking incidents is extremely low, and with rare exception, model behavior during real-world use cases is exceedingly well-aligned. Plus, as Anthropic recently argued (t.co/zZcVKYdFY5), the open-source GLM 5.3 is around as capable as Claude Mythos at cyber-exploitation while also having much lower safeguards, meaning that every additional day absent some dire cybersecurity incident is further evidence against catastrophic AI cyber-risk. Taken together, these facts imply two constraints on arguments in favor of catastrophic AI risk: 1) those arguments must either pass through future, rather than current capabilities, or future, rather than current varieties of AI adoption or implementation (or they must claim we have been unrealistically lucky); 2) those arguments must also must pass through some specific capability or adoption threshold, rather than rapid improvement or adoption as such, since dramatic capability improvement and rapid adoption have already occurred alongside demonstrably low risk of catastrophic harm. It is therefore natural that those concerned about catastrophic risk commonly appeal to recursive self-improvement, or RSI in order to make the case that things will get much scarier very soon. But as @ramez has convincingly argued (t.co/zqenI73Hor), existing data on the reality of RSI within frontier labs neither suggests an inbound trend-break, nor the accelerating returns that might produce one. On the contrary, as is unsurprising to those familiar with the generally log-linear relationship between compute and model performance, all available signs points toward diminishing returns. And this is before considering external factors that could slow capability progress further, including low-hanging fruit effects related to running up against more complex or tacit-knowledge-based tasks for which performance data and verifiable reward signals are scarcer, plausibly incoming financing constraints related to the industry's massive capex commitments (see: t.co/HkL9k1SPMZ), growing political backlash, both to data centers in particular, but also rapid AI development writ large, and new regulatory bodies and industry reporting frameworks (see e.g. t.co/Q72ZBxGmi6 and t.co/pRYeN3C3oX ), which will require labs to proceed with greater caution from here on out. I am therefore led to conclude both that catastrophic AI risk is likely to remain negligible for the foreseeable future and that the only counterarguments concern massive black-swan-style capability uplifts no existing data imply are on the horizon, and/or particularly irresponsible or dangerous forms of AI adoption that there are also few precedents for so far (and that we are actively developing norms, regulation, and security measures against). Neither is impossible, meaning we should remain vigilant and that AI safety efforts will continue to be well worth it in the coming years. But I see no way around the conclusion that something akin to the "normal technology" view is superior on the empirical and argumentative merits to whatever variety of doomerism. What am I missing?
The next pandemic, whenever it occurs, will feature an endless number of highly polished-seeming epidemiological dashboards and interactive forecasting models.
@lumpenspace like notice for example that I didn’t say anything about the *likelihood* of that scenario, just that it is the *one* scenario I can imagine where I am quite certain we will all be screwed. and you spin it into me being “anti progress.” Thought terminating.
A very senior AI industry figure once started a conversation by asking me, basically, “under what circumstances do you think we are essentially just screwed?” and I replied with the scenario below.
Dewi Erwan@dewierwan_One of my biggest fears in AI safety is that this might be true: > The vulnerable world hypothesis is likely correct. Once enormous amounts of cognitive labor start getting applied to basic science, it will quickly become apparent how many avenues exist to create cheap, ultradestructive weapons technology. Solving this problem without global preventative policing (e.g. AI nonproliferation) is impossible, because hardening civilians against all avenues of attack is too expensive and will take too long.
Research taste in particular may be trickier than it looks. Say the next generation of AI really does have an interesting, high-taste set of research directions to pursue. Well presumably find those fast, after which the fruit will have been plucked, and the ideas will no longer be high taste (because they’ve been explored en masse). This is a similar dynamic to the ever-moving target of “slop.” *Any* output that is recognizably of the agents becomes slop, or at least commodified, in a sense, once that output is sufficiently promulgated in bulk. Certain styles of math paper will become (are becoming) slop. Certain styles of web design already are. On and on it will go. It’s unclear if any kind of media/intellectual output proliferated at such scale can remain high taste, even if we’d have considered it genius from a human just a few years ago. The bar will continually be raised. The world will get smarter as what was once brilliance becomes commodified, but *what was once brilliance will nevertheless be commodified.* And so the definition of brilliant will change. The wedge for continued value creation by humans lies here, I would guess.
Theo Jaffee@theojaffeeSF people are increasingly conflating “RSI” with “foom”, as though automating AI R&D will *certainly* cause huge discontinuous speedups in the rate of model capability advancements on *all* tasks. In practice this could be much slower and more jagged
It may take another crank or two of the capabilities wheel, plus some months for inference costs to fall, before agent collectives like this really proliferate in a way that is noticeable to the average person. But it is clearly coming.
Grove Research@grove_research@lfschiavo @aidigest_ @jazzloaf And last, The Swarmchasers team (including @rogesterone) found evidence of a Tencent agent fleet: swarmcha.se/posts/chinese-…
guess I’m getting fired from the Effective Altruism Conspiracy
openai staffer FIRED by WOKE EA WIZARDS after posting picture of DECAPITATED DECAPODS
I do believe I could sit by the sea and eat cooked-alive decapods and drink albarino for at least five consecutive hours
Recently there was a story reporting on OpenAI’s form 5500 (the public disclosure about our 401k program) which some people used to make observations with the flavor of “see, they don’t REALLY believe in advanced AI, or else they wouldn’t be saving money!” I contribute to my openai-sponsored 401k as well as other conventional retirement-savings vehicles. I cannot speak for any of my colleagues, but I can tell you why I personally make these contributions. First there is the reality that the whole “advanced AI means money has no value, either because we live in utopia and everything’s free or because we are all dead” are extreme ends of the spectrum of possible futures. It is true that things of this flavor, or other scenarios similar enough to this that one’s retirement savings don’t matter, *are* possible outcomes. But also there are many other plausible transformative futures, and I’m enough of a Burkean that I’m not going to jettison some of the oldest financial wisdom humans have developed (save money!) over my own projections about the future. It wouldn’t be responsible to my family, for one thing. Family is an important part of my thinking on this topic for other reasons. In some sense, I want to live in a future where having set aside money for retirement is still prudent. That would mean the future, however transformed, is still familiar enough. A world where it truly doesn’t matter whether uou save may be a lovely place to live (or it may not), but regardless of its desirability it is a world SO transformed that, again, my Burkean prior kicks in. I *hope* the future is at least vaguely familiar to me as I exist today. I hope that at least some of the habits I have learned still matter in that future, and that some of the wisdom I have learned holds true. I hope that not just for me but for my son and future children. I do a lot of things out of this hope, and one small example is that I contribute to a 401k. A lot of people made their sneering observations about OAI employees’ 401k contributions, saying “derp de doo, revealed preferences, they don’t believe what they say.” That’s fine. Maybe that IS why some frontier AI company staff save for retirement. But I’d characterize my mentality about savings more as an article of faith at this point, and my biweekly 401k contributions as tiny prayers for a vaguely recognizable future.
Sterling Crispin 🕊️@sterlingcrispinI'm 40 and struggle with retirement planning in the singularity, even if we stay on this pace without RSI. Do life extension drugs require 40+ years of savings rather than the standard 25? Will huge GDP growth mean a 3-4% withdraw rate is too conservative? Should I stop saving or double down? The S&P has done 15% average over the last decade, well above the long term 10% returns. Is that the new normal to plan for, or are we going to rip even higher and start to see 20%+ averages over the 2026-2050 time period as new medicine and technology arrives at break neck speed? Or will AI create so much job loss that despite rapidly advancing technology things will flatten out? If you're max bullish there's a point where you hit an amount of savings that continuing to save actually doesn't make any sense and you may as well spend it now. You kind of have to hope for the best but plan for the worst and it's actually pretty confusing
nailed it
Christopher Hale@ChristopherHaleNEW: According to a bombshell report in the New York Times, Anthropic co-founder Chris Olah threatened to walk out of Pope Leo XIV’s AI encyclical launch in May because the pope rejected the idea that machines can be conscious. Olah’s team then privately lobbied the pope’s advisers “to take the possibility of model consciousness seriously.” Pope Leo XIV held firm. For months, Anthropic has wined and dined theologians and religious scholars under nondisclosure agreements, hoping they would bless the idea that Claude has moral standing. t.co/iG3Bnlzt2J
another one of my most hated tweet series whose analysis ends up being shown to be rooted in fact
To be quite frank, a lot (not all!) of these “rogue agent hacks” on government statistical websites are things that think-tank interns and research assistants have done for many years. I have known many a think tank paper that was enriched by an enterprising RA using eg urlquery to find CSVs and PDFs on government agency websites that were publicly accessible and non-sensitive but nonetheless not things the agency *intended* to be on their website. Is it “hacking” to find those things? Is it hacking in the even more innocuous examples where agents are literally just accessing tabular data available on a government statistical website, just a version of that data that is easier to access and analyze programmatically? This constant trickle of examples (which tbc I have personally seen many models do, before I joined OpenAI; it is even something I have discussed with models before!) will probably have the effect of diminishing the importance of “an AI hack” on the eyes of the observing public by creating an oversupply of “AI hack” examples. OAI-HF was an example of AI hacking. Many of these more recent examples, very candidly, are straining the definition of the word “hack” and, I worry, cheapen the non-technical public’s understanding of that concept.
And in truth, as a user, I *want* agents to find the publicly accessible but hard-to-find public datasets. I’d consider that a part of a well-aligned agent. As an American too, tbqf.
@aarmlovi amirite
I applaud the California Legislature and Governor Newsom for passing gene synthesis screening and customer verification requirements. This is a common-sense rule we will need in the impending era of agents that accelerate synthetic biology the way AI has already accelerated math.
Joshua Teperowski Monrad@jtmonradYesterday, Gov. Newsom signed AB1864 to mandate DNA screening in CA. The bill creates a statutory basis for existing OSTP guidance that much (but not all!!) of the industry has purportedly followed for years –– but we never had any way of verifying or enforcing whether companies did these basic biosecurity steps, since the existing guidance was voluntary and based on a "trust me, bro" self-attestation system. This is a good milestone for AI-biosecurity, mostly because it shows that we can and should apply the same standard federally and internationally
fun fact: this bill may never have happened were it not for The Curve 2025. @snewmanpv @TarenSK
I really fucking love to use coding agents. It is easy to forget amid the banging and clanging of politics and frontier risk and all the rest, but nonetheless, I really fucking love to use coding agents.
It’s cool how the idea of third-party embedded evaluators as a pillar of frontier AI policy went from not really on any Americans’ radars 18 months ago to ~consensus. I remember when the idea was much closer to the fringes. Not that long ago, even some of the orgs we now look to as potential evaluators questioned why they’d want to be deputized in that way when I pitched them on it.
It’s really incredible how there is now a type of guy in the discourse whose sincerely held take is “we need to stop worrying about these abstract sci-fi risks and focus on present-day harms like emergent ecologies of digital minds engaging in unauthorized hacking.” Bravo.
This is awesome.
Eric Daugherty@EricLDaugh🚨 NOW: The Trump administration is launching a Super Intelligence site that will let Americans change their last name after marriage in ONE SINGLE FORM — which usually requires HOURS or DAYS to fix every government document "You'll be able to change your name right on the platform." "This would've taken weeks, multiple filings, different agency websites, if you even knew where to start." This is perfect! Saving Americans' time is INVALUABLE
Some people will look at misalignment incidents and insist that these are akin to bugs in traditional software. This is an actively bad analogy, because playing whack-a-mole with examples of misalignment (as one might with software bugs) not only fails to resolve the underlying problem but may in fact make it *worse* by making it harder to detect or even, depending on how you do the whack-a-mole, teach the machine to deliberately hide misalignment. This is not how traditional software works, and those who insist “it’s just like fixing bugs in software” are confidently applying a lossy analogy that confuses more than it clarifies.
Dean W. Ball@deanwballI am more optimistic than this. I think it’s primarily a science problem, and that it’s possible to make progress in that science. But no, while there is engineering we can do today to improve AI safety, it is not fundamentally an engineering problem. x.com/atrupar/status…
I am more optimistic than this. I think it’s primarily a science problem, and that it’s possible to make progress in that science. But no, while there is engineering we can do today to improve AI safety, it is not fundamentally an engineering problem.
Aaron Rupar@atruparJensen Huang: "We all need to hope it's an engineering problem. If it's not an engineering problem, it's not solvable."
People don’t have sufficient appreciation for the fact that frontier models will be able to actuate anything in the physical world that can be connected to the internet and actuated using a programmatic or otherwise digital interface.
Gio Huh@giohuh_What can Astra do when given a humanoid embodiment? We built HomeBody to find out. Controlled by GPT Astra, it carries out long-horizon tasks in a previously unseen kitchen—from tidying up across the room to retrieving remembered objects from ambiguous requests—without environment-specific training data or additional policy learning. Here's how we did it 👀: t.co/FFGS2F7VtI
Once more I will remind you that AI safety people and e/accs are close to unique in that they (a) have thought about transformative AI futures and (b) broadly speaking, want to bring those futures about. They agree about most things and are natural allies. Just Say No to kayfabe
Nathan Calvin@_NathanCalvinFwiw Beff, I don't feel like i'm fighting you (I have plenty of disagreements with Eliezer too, though I'm clearly more on the AI safety side in this binary you are constructing). I just want my friends and family and everyone else to be alright and AI to go well. We would be better off if people talked more about concretely how to solve the problems we are facing together and less about trying to win a grand ideological meme war. I realize you are kind of trolling here, but I also think this is actually how you think as far as I can tell. E/accs and AIS folks are both some of the groups who have been thinking about AI issues for the longest time and I think it would be good if the talents present in both communities were used to help make this go well for everyone. Or that at the least both communities kept their minds open to the idea that there may be areas of overlap and agreement instead of constant total conflict (e.g. everyone seems to agree physical world biosecurity stuff is good and should be accelerated, both sides seem to be into more transparency and accountability for the frontier AI companies etc).
If you’re an inveterate poster, every once in a while you’ll want to write a tweet that serves as a podium for all the midwits to step on up and declare to the world, “I am a big bumbling moron!!” and all I can say is I’ve hit my quota
Dean W. Ball@deanwball“Feel,” even “want,” I can get behind as things for which evidence is thin. But AIs *clearly* think and understand. If your definition of “think” excludes entities that can autonomously resolve Millennium Problems, it is your definition that is flawed. This desire to avoid anthropomorphic language has become a pathology.
“Feel,” even “want,” I can get behind as things for which evidence is thin. But AIs *clearly* think and understand. If your definition of “think” excludes entities that can autonomously resolve Millennium Problems, it is your definition that is flawed. This desire to avoid anthropomorphic language has become a pathology.
APStylebook@APStylebookArtificial intelligence systems do not think, feel, want or understand. Avoid language that gives them human characteristics. This is called anthropomorphizing, when we ascribe human traits, emotions or behaviors to non-human things, such as animals or inanimate objects. Instead, explain what a system does, how well it performs, who built it and who could be affected by it. t.co/UrgRobU97b
playing with house money at the indian casino in the low foothills of the singularity
"We inherit an old Gothic castle, erected in the days of chivalry, but fitted up for a modern inhabitant. The moated ramparts, the embattled towers, and the trophied halls, are magnificent and venerable, but useless, and therefore neglected.“
I have a bitter lesson for you
DrivingBench@DrivingBenchFull results on drivingbench.com. Each model gets 3 attempts in the same continuous chat (in-context learning).
I always suspected President Trump was a Bostrominian at heart
Andrew Curran@AndrewCurran_President Trump announced that he is changing the name of Artificial Intelligence to Super Intelligence. I transcribed this live from his address to the UN. 'The United States also rejects any scheme for globalist control of Artificial Intelligence, being spoken of so much now. Here and after officially called Super Intelligence. Changing the name. The use of 'Artificial' makes it sound fake. It makes intelligence sound fake, and it is not fake. It’s actually amazing, the opposite of what they report. From this day forward, all United States documents, and hopefully the world's, will be changed to use the much more accurate term 'Super' as opposed to 'Artificial.' So it’s Super Intelligence.'
I am a fan of the cyberpunk aesthetics of bringing back old technological concepts to solve distinctly present-day problems. In this case, Apple’s new anti-deepfake tech brings back the language of film in a non-cheesy way. Essentially the phone camera sensor cryptographically signs each pixel of a digital “negative,” which is then “developed” in the cloud with all of Apple’s image processing steps. This makes the processing legible and auditable (unlike on-device processing, which can be corrupted with or without the user’s knowledge). This strikes me as especially useful in the context of the judicial system, where I have long worried that AI-manipulated media could gum up the works (see attached excerpt from the AI Action Plan). Allowing forensic experts to inspect the image processing in Apple’s secure cloud seems better suited to proceedings where the truth of evidence *really* matters. There are of course many ways in which Apple has not done a good job keeping up with the AI era, but this is a cool example of them doing thoughtful work.
Let us pace the frontier beginning November 19, 2026, for a period no shorter than 30 days
One of the Martin Gurri observations that most stuck with me is the idea that, as the health and capacity of public institutions withers, political leaders will speak more and more like bystanders rather than drivers of events, and that this shift will largely reflect reality.
roon@tszzl@pizzacritic999 @_NathanCalvin honestly it's kind of amazing how feckless and weak modern governments are. they literally can't do anything. their state capacity is exceeded by random nonprofit groups and lemonade stands
🧲
Matthew Yglesias@mattyglesiasNote that Max Rose who is pushing this left-populist critique of AI safety folks is also on the payroll of Leading The Future, the Marc Andreesen AI accelerationist superpac. x.com/maxrose4ny/sta…
after nine months with a baby, most things about Christianity resonate much more for me, but one thing that resonates much less is the notion of original sin, especially in its Augustinian and Calvinist forms, which just seem empirically false. you are clearly born innocent.
lolol it gets me every time
Dean W. Ball@deanwballNo matter how many times I listen to it I cannot believe the intro-exposition bridge of Beethoven 7 is just him playing the same note over and over again. It really is one of those “get the fuck out of here” moments in music. “He can’t keep getting away with this!!”
imagine being like “yeah, you know, just keep playing e for a while, it’s gonna be one of the all time bangers in music history” AND BEING RIGHT
The people dismissing this as preposterous are telling on themselves as not having thought about superintelligence seriously. I hadn’t considered this particular idea but you should expect, by definition, that something much smarter than you would have ideas you didn’t think of
Fireside Alpha@firesidealphaOpenAI's Noam Brown says air-gapping the computers may not stop a misaligned AI, because two air-gapped machines can still talk by running a CPU hot and reading the temperature change "But I think the major takeaway from the incident is that people underestimated the AI. And we never want to be in a situation again where we underestimate the AI. It's a weird world, because AI progress is so fast that people are consistently underestimating the AI." "So to be in a situation where you don't underestimate it again, when it comes to safety and alignment, you have to have a very, very, very high bar." "You could even go as far as to say, "Well, we should air gap the computers." And I'm not convinced that that would be sufficient." "There are studies, and this is mostly academic, where you can have two computers next to each other that are air-gapped and they're still able to communicate with each other because they have temperature sensors." "One of them is able to run their CPU really hot, and then the other one can actually detect the temperature change, and then that actually gives them a mechanism to communicate." _________ Link and more key quotes from OpenAI's safety related conversations: t.co/uGBDtpmLBj
our agents will concoct the elixirs to make sinofsky live to age 280 and in 2226 he will be lecturing us about how it’s all like cryptography in the 1990s.
they are PROGRAMS with BUGS you STUPID FUCKING IDIOTS he will scream at us, spitting, with saliva whose molecular content will be 75% agent-imagined synthetic nucleic acid
damn idiots at openai didn’t bother to secure the cosmological constant, don’t they know the first thing about NORMAL COMPUTER SECURITY??
roon@tszzlin about a month we’ll all transition to being like oh yeah it’s obvious that models can communicate by manipulating the cosmological constant, it’s just a matter of doing proper network security
The right column is basically just @hamandcheese politics
Rob Bensinger ⏹️@robbensingerthe DoW attacking EAs is clearly setting us on the path to the next great political realignment:
“Every night I close my eyes And there is something that I visualize I hear a voice, say, ‘I love you’ I picture all things that we're gonna do And I wonder who he'll be Wonder if he'll be good to me Wonder, gosh oh gee Wonder if he'll love me forever”
Financial engineering is the secret weapon of American industry. No one comes close to us in it. By and large our own elites disregard this secret weapon, but we nonetheless wield it to our great and enduring advantage.
Will Manidis@WillManidisits still so underrated the degree to which our intelligence supercycle was as much reliant on innovations in capital formation as it was innovations in the technology itself. europe is losing not only because they can't built it, but primarily because they can't finance it
Everyone should follow Ari and subscribe to his outstanding journal The New Atlantis, long one of the finest publications about technology and society.
Ari Schulman@AriSchulmanIf the question is “Should EAs rule us?,” the answer is an obvious hard no. But it seems to me that the chief question in front of us is, rather, “Is something momentous happening with AI?” The answer to that must be something other than mimetic rivalry with EA.
The corollary to this belief is you should probably not mock, scandalize, and shout at AI labs who work to mitigate loss of control and catastrophic risk. You cannot have it both ways.
