
Sam Rodriques
@SGRodriques
We make agents make medicines for humanity. https://t.co/aNx8D1QU5l // https://t.co/rQYoPOxsYo
62 postsÚltimo em 22 de out, 09:37Atualiza a cada 15 min
The hard part about AI in bio is that by the time you can test the predictions from model X, you already have better predictions from model X+2 or X+3.
will depue@willdepuegod these two charts really hurt my soul 64 hours to dry lab design an multi-binding antivenom and 39 days for results and 4-7 years to release x.com/citrini/status…
This is nonsense. Success in verticals is about more than just intelligence. It depends on also having relationships, supply chains, and assets (e.g. data). These are hard to build. The labs cannot magic these out of thin air, and the people who have them don’t want to sell. If the labs want to capture the value generated by AI in verticals, they need to partner with existing players, get their hands dirty, and generate value through integration. But there is a not a lot of evidence that they are doing this, or at least not well. Otherwise, the fundamental problem is that the labs have to bear the cost of training models, and unless you do the hard work of integration, the cost of training the marginal model right now is greater than the value provided by the marginal unit of intelligence. And as soon as they stop training models, everyone else catches up. Most of the chatter on X right now is that the labs are doomed. I think that is not the case. But their next big challenge is to operationalize the integration of AI into the economy. It is hard and slow.
Erik Newsham@eriknewshaby this time next year, most of labs revenue will come from what they build in-house and then sell further up the stack. the average users just don't 1. have deep enough pockets 2. work on hard enough tasks gpt-synopsys and anthropic wet lab are the first of many products which come as a result of labs gatekeeping and utilizing superintelligent swarms to create new and capture existing markets. currently by use of footholds and in the future independently. by 2030 labs will have eaten practically every economically valuable sector such as finance, legal and healthcare. what's to stop labs? why wouldn't they? how can a 10,000 employee healthcare company stand a chance if they are up against 10 million vastly superior agents to the swarms publicly available? the labs understand this and are hence preparing for the "grand capture" by donating shares to the public (anthropic and spacexai already committed with openai about to) to minimize p(public revolution). the question becomes what to do. one option is obviously to join a lab. if you want to join the lab I believe the best "unit chance of joining lab/time" ratio is proving scaleable user acquisitions and publishing relevant hot topic things on X. if you don't want to join a lab, acquiring factories, propietary data, licenses, land and things that are human facing and exist in a stack of value creation will be fruitful as they become the primary bottlenecks once suffcient swarm cognition is solved
This is similar to what we see. On some tasks, there are still major returns on intelligence. However, for the large majority of tasks, we are either close to saturation or are already at saturation. Once intelligence saturates, tacit knowledge, data, supply chains, and operational ability become the bottlenecks.
The Information@theinformationOpenAI, Anthropic and other leading AI labs face a growing challenge: Better models aren't necessarily attracting more usage. A 10-point intelligence advantage now earns just 20% more developer traffic, down from nearly 69% in late 2025. Full story: thein.fo/4hGJ9jY
OpenAI's models refuse to support the development of pandemic countermeasures. This needs to be fixed.
Without diminishing the contributions of Peter, Georg, or Karl, I find the omission of Ed Boyden from the optogenetics prize to be highly unreasonable.
The Economist published an article yesterday calling for regulations to protect against mirror life. They propose restrictions on buying mirror chemicals, including extremely normal stuff like L-Glucose or D-amino acids. What is going on, guys? Where did this come from? I think the concerns about AI designing pandemics are overblown, but at least designing viruses is a real thing that we actually know how to do. 1. To be clear: we do not have any idea today how to make an autonomous cell from scratch, mirror or not. Making a mirror cell from scratch is much, much harder than making a non-mirror cell from scratch. I am a believer in strong AI acceleration but there is a lot of hard science and manufacturing that is missing here. People have ideas about how to do it, but they have ideas in the same way we have ideas about quantum computing: we will have lots of hints that it's around the corner before you see in on a shelf at walmart! Call me when we create an autonomous non-mirror cell in the lab from scratch, and then we can chat about sensible regulations. 2. Even if they existed, mirror bacteria wouldn't be able to escape the lab: they couldn't eat anything. Good luck being a left handed organism in a world where all the food is right handed. I know, a malicious person with access to mirror life could in principle give the mirror organism a way to digest non-mirror food, and then it could wreak havoc. But the converse is also possible: if a mirror bacterium were on the loose we could just give the normal non-mirror bacteria enzymes that would allow them to eat the mirror bacteria, and drive competition. This fact is generally ignored. 3. Finally, mirror life doomers pretend we have no defense against mirror life. This is also just nonsense. We have plenty of defenses. They are called mirror antibiotics. Mirror antibiotics would work against mirror life the exact same way non-mirror antibiotics work against non-mirror life. And there are way more options for mirror antibiotics than there are for non-mirror antibiotics, because we don’t have to worry about accidentally killing the human in the process! We actually even already have a ton of mirror antibiotics, since they are usually byproducts of antibiotic manufacturing. Fosfomycin, for example, is extremely simple, easy to manufacture in bulk, and industrial quantities of mirror fosfomycin are already produced today as a byproduct of fosfomycin synthesis. D amino acid analogs, like D-cycloserine, would also be very effective. If we can figure out how to make mirror life we can definitely figure out a bunch of universal mirror antibiotics to kill it. Spare a thought for the poor scientists studying chiral biochemistry. Their requests to the models are all about to be denied as potentially dual risk. Meanwhile, want help finding a supplier for a toxin with nanogram LD-50? Sure, Claude will help you with that. I agree mirror life escaping the lab would be bad and it deserves consideration, but can we stop pretending that it is a huge danger right around the corner that deserves front page regulatory attention, and instead focus on real shit like AI cyberattacks and the fact that the poles are still literally melting?
Matt is a powerhouse. We are super excited to have him on board.
Matthew Kirshner@MattyKirshPersonal update: This week, I joined @EdisonSci , focused on building new partnerships across biopharma. It’s bittersweet to be stepping back from the team at @ferment_co , who I've worked with for the past six years. We’ve accomplished a lot since launching Ferment, including raising our first fund and building seven new industrial biotech companies at the intersection of biology, hardware, and computation. I’m a deep believer that AI and agentic science will drive extraordinary progress in human health. I’m an even deeper believer that this impact will only come when we put these tools in the hands of drug hunters who know which problems matter and where to point them. That’s a big part of what drew me to Edison. The team pairs enormous ambition about what AI can do for science with an extremely practical lens on what must be true for the technology to actually be useful, both for our individual scientist users and for the scaled biopharma organizations they work within. I’m excited to be an AI arms dealer for drug hunters everywhere. If you’re thinking about how to bring cutting-edge, sovereign AI into your biopharma organization or have ideas on how agentic science can accelerate drug development, don’t hesitate to say hello!
Major extinction risk here. Within 3 years most women will have AI boyfriends and then there will be no more babies.
Tavus@tavusIntroducing Griffin, the first model to pass the video Turing test. 48% of people who talked to it live thought it was a real human. Previous systems have had a pass rate <3%. It is #1 on NVIDIA's benchmark for full-duplex AI video. It’s the first Human Interaction Model (HIM).
Okay I don’t say this about much, but definitely agree that use of this thing needs to be heavily regulated.
Tavus@tavusIntroducing Griffin, the first model to pass the video Turing test. 48% of people who talked to it live thought it was a real human. Previous systems have had a pass rate <3%. It is #1 on NVIDIA's benchmark for full-duplex AI video. It’s the first Human Interaction Model (HIM).
Very cool. Terraforming is having a moment.
Casey Handmer@CJHandmerThis is too much fun. A render of how the quilted hyperbolic isotensoid Mars city structure might work.
At FutureHouse, we focused on scientific discovery. However, if we want to actually accelerate progress in medicine, we need to work across the entire R&D stack. We are now deploying specialized Kosmos agents for Clinical Protocol Design and Regulatory Authoring to several of our partners, which can cut months off the amount of time needed to get molecules into the clinic, save companies tens of millions of dollars, and dramatically reduce the likelihood that submissions get stuck at the FDA. Super excited to be able to share this story now, thanks to our work with Population Health Partners, one of the best drug developers in the world and a fantastic development partner for us. The AI-native pharma stack is here now, and the future of medicine is around the corner. Read more:
Edison Scientific, Inc@EdisonSciWriting an Investigational New Drug (IND) application is a major hurdle for drug developers. It can take a team of specialists three to four months, and mistakes can trigger protocol amendments or clinical holds. In June, we started working with Population Health Partners (PHP) to leverage specialized Kosmos Clinical Protocol Design (“CPD”) and Kosmos Regulatory Authoring (“RA”) products to support PHP’s process of IND preparation. Since then, specially trained Kosmos products produced a first acceptable draft of the Nonclinical Overview in 4.2 hours, against 100 hours of expert time, flagged an error that roughly fifteen human reviewers missed, and authored 70% of the text in a Pre-IND Briefing Meeting Request submitted to FDA. PHP is now bringing Kosmos products to a second program and expects to complete IND preparation in as little as 1 month, instead of the 4 month industry standard. PHP's experts can spend their time reviewing and deciding, not just drafting. The time recovered by Kosmos products means that drugs could make their way to patients faster.
Read the blog post: edisonscientific.com/news/how-popul…
This is very good
NASA Administrator Jared Isaacman@NASAAdminI can assure you we have already been working on revisions to the planetary protection policies to bring them back in line with the common-sense principles originally established by COSPAR in the early days of planetary exploration and later reflected in Article IX of the Outer Space Treaty. We are needlessly adding extreme cost and time to robotic missions to Mars when the overarching goal is to send astronauts there as quickly and safely as possible.
@celinehalioua #LoyalForTurtles
Wow the new @DaveARicks X account is on fire.
Dave A Ricks@DaveARicksTruth from my Euro colleagues. We are not HQ’d there but have massive operations in Europe. Unfortunately, short sighted policies are driving us all to the exit doors. When I joined this industry 30 years ago this was a European led industry, today Research, Clinical Development and Manufacturing are all shrinking in relative terms..Moving mostly to the US and increasing to China. The fix isn’t complicated but it is seemingly hard for them - improve IP, speed up the regulatory environment, value and pay for innovation - and the window is rapidly closing.⏱️ t.co/EORLvkjjQp
Thrilled to announce our partnership with Springer Nature. Full text articles from Nature journals are now available to Kosmos for all of our customers. The most important resource for science is knowledge, and Kosmos has access to more scientific knowledge than any other agent. More to come here...
We have updated the Millennium Problems website (link below) to include a form and email where you can submit ideas for additional problems, or comments on the current problems. Please send your feedback and ideas!
Lfg
If you are wondering how AI will change science and what it makes sense to do right now, there are essentially three things: 1. You can work on solving the problems that are fast and easy to verify. This is a small subset of all problems, but these are the problems that will be most impacted by AI. The Millennium Problems we published are an example. 2. You can work on solving the problems that make other problems hard to verify. For example, one of the slow parts for solving many problems is small molecule synthesis. Make small molecule synthesis fast, and we will solve many more problems. Other things that fall into this category include creating new assays, finding new biomarkers, and accelerating clinical trials. 3. You can go and gather data for the really hard problems (i.e., the ones that are impossible to verify easily). In the age of AI, data is king. There are many other extremely important and valuable things to be done as well. But they will not benefit from AI nearly to the same extent.
I have always thought it would be so cool to visit Valles Marineris. Can't wait. Congrats @erika_alden_d and the Pioneer team! Terraforming is going to happen.
Erika Alden DeBenedictis@erika_alden_dToday, Pioneer Labs is announcing our first step towards terraforming Mars. 🚀🌼 With equipment that fits in just a single rocket launch, we can convert Martian dirt, water, and air into enough building materials to construct a small city on Mars. To do it, we made the first microbe for Mars. We found the best microbe on Earth and used evolution to teach it how to source all of its nutrients directly from Martian materials. The first astronauts will be greeted with safe shelter already filled with water, oxygen, and rocket fuel for the return journey. This is the first step toward using biology to make Mars a friendly place for life. It lets us live off the land and helps us build the next great frontier. It's the first of five organisms we need to green Mars ⬇️
The community response to the Millennium Problems for Biology has been great. A few updates: There is clearly a deep hunger for a more comprehensive list of big unsolved problems in biology and medicine that can serve as a beacon for the field, and we have seen some very good suggestions. Over the next few weeks, FutureHouse will run community-led process to assemble a longer list of grand challenges in biology and will publish this expanded set as additional to the original set, with attribution. In addition, the Millennium Problem list and specifications also needs to be updated in response to a few issues that people have pointed out. That will happen in the coming days. Finally, FutureHouse will convene a panel to adjudicate success, and will also announce cash prizes associated with the problems. We will have more to share soon. Thank you to everyone who has participated in the conversation! t.co/IzvK5HGCgt
Sam Rodriques@SGRodriquesIn light of the progress in mathematics, we at Edison Scientific and FutureHouse have assembled a set of Millennium Problems for Biology. They are chosen to be very hard to solve but very easy to validate in a simple laboratory environment. Any of these, if solved, would mark a major advance in biotechnology, and most of them would contribute materially towards curing disease. These are, in some sense, the “last reasonable eval” for AI in biology. This was work primarily by @MichaelaThinks and myself, with contributions from many others. Short descriptions below. The full descriptions of the problems with acceptance criteria are at the Bio Millennium Problems website, linked in the next post. Share more if you have ideas. If they meet our criteria, we’ll add them to our list (with attribution and permission).
In light of the progress in mathematics, we at Edison Scientific and FutureHouse have assembled a set of Millennium Problems for Biology. They are chosen to be very hard to solve but very easy to validate in a simple laboratory environment. Any of these, if solved, would mark a major advance in biotechnology, and most of them would contribute materially towards curing disease. These are, in some sense, the “last reasonable eval” for AI in biology. This was work primarily by @MichaelaThinks and myself, with contributions from many others. Short descriptions below. The full descriptions of the problems with acceptance criteria are at the Bio Millennium Problems website, linked in the next post. Share more if you have ideas. If they meet our criteria, we’ll add them to our list (with attribution and permission).
Read the full specifications here: t.co/sXXRAI9xxc A few notes on exclusion/inclusion criteria: -We placed a major emphasis on ease of validation, so challenges like “solve aging” are omitted. These should all be validatable in a standard wet lab in a short period of time (days, maybe a week or two.) -We placed a major emphasis on ensuring grading would be as objective as possible, so many great basic science mysteries are also omitted, because it would be very challenging to tell when the mystery was “solved.” (E.g., “uncover the function of the Vault protein,” or “explain how the Oxytricha genome works.”) -We placed a major emphasis on ensuring the problems would be understandable to anyone with a basic biology background, so some highly technical but very important challenges were also omitted. Finally, we anticipate that some problems may require edits in response to community feedback. If you think we got them wrong, please send the feedback, and we will collate and update. Thanks to Adam Marblestone @AdamMarblestone , Michael Baym @baym, Erika DeBenedictis @erika_alden_d, and Tony Kulesa @kulesatony for reviews.
I'm sympathetic to mathematicians who worry that AI will lead to a bunch of "alien proofs" that no human understands. It may be that the future of math is mostly finding alien proofs and doing the work needed to make them intelligible to humans. Which, frankly, is the situation in biology already.
The future of customer support is here.
Along the lines of the conversation recently, we're excited to participate in Anthropic's access program for their most powerful models in bio. We need to accelerate progress in medicine. This is an important step in the right direction.
Anthropic@AnthropicAIToday we’re opening applications for the Life Sciences Verification Program. Through the LSVP, life science professionals can use our models—including, for the first time, Mythos—with a new set of safeguards designed to enable the full range of biology-related work. We designed these new safeguards to provide a better experience for biologists and more protection from risk of misuse. The program is launching in beta for teams of all kinds—from academic labs to startups, pharma companies, and more. We will continue to improve the program and expand access to individual Pro and Max plans over time. Learn more about these access grants and apply: t.co/uALS2lZuN4
Actually, even stronger than this: while people are worrying about theoretical engineered pandemics, more than 1% of the population is already dying every year to disease and old age. Safety is important, but accelerating medicine is the best way to save human lives.
Sam Rodriques@SGRodriquesIt is important to remember that while we are debating theoretical AI risks, real people with real diseases are really dying every day. Safety is important, but slowing down progress and blocking researchers from using the strongest models to find new medicines is not the way.
Also, to all the people going ad hominem here against @kesvelt, let me say that I disagree strongly with Kevin on many points, but Kevin is a very thoughtful and serious researcher with good intentions and good points, and the ad hominem attacks are ugly and baseless.
Kevin Esvelt@kesveltSome scientists argue civilization-threatening bioweapons are a fantasy. I wish they were correct, and that I've wasted years building defenses. That you can't see a way, or evidence that others can, doesn't mean no one could – especially if anyone cautious would stay silent. My ask to colleagues: please do not casually assert with confidence that nothing biological could bring down civilization, even on X. What we say can turn out to matter, especially when the Overton window on risk mitigation is wide open. When Fermi called the chain reaction a remote possibility, around 10%, Rabi replied that ten percent is not remote if it means we may die of it. And by then, Szilard had already begun reasoning about what to disclose, and when, and to whom, in order to best safeguard the future. I've been concerned that one of my human colleagues might notice one or more viable paths to pandemic weapons for over a decade. It's why I didn't publicly describe the categories of civilization-threatening pandemics and necessary defenses until 2023, but was starting to build cryptographic screening systems capable of controlling access to sequences without disclosing whether they're dangerous in 2018 – long before ChatGPT, and even before the debut of AlphaFold. AI can worsen biological risk, but biological risk hardly depends on AI. Who am I to weigh in on whether civilizational biothreats are possible? Fine, to get the expertise claims out of the way: yes, I specialize in evolutionary engineering, yes, I've worked with viruses with my own hands, and yes, I've worked quite extensively with AI, both LLMs and biodesign tools. I invented CRISPR-based gene drive, and held that back until confident it favored defense. While it hasn't yet been tested in the wild (due to politics), I haven't heard anyone suggest that it would suddenly stop working outside cage populations, so it might be fair to say I've (temporarily) beaten nature at its own game. I also was early to spot the full threat from mirror life and recruited others with more specialized expertise to evaluate it in depth; much to my disappointment, no one has yet struck a solid blow against that hypothesis. The reason why the threat from mirror life is public also explains why there isn't any similarly public evidence of other catastrophic biothreats. We communicated our reasoning on mirror life because no one can make mirror bacteria, we won't be able to do so for years, and many colleagues wanted to build it because they correctly noticed that doing so would be an awe-inspiring achievement... and weren't aware of the risk. As long as that was true, humanity would only drive closer to the cliff, which would be disastrous if said cliff exists. Crucially, pointing it out and urging a course correction while still far away meant that no one could use our findings as blueprints to send us careening over the edge. Contrast that with pandemic weapons, where any credible genomic blueprints for an accessible virus could be realized within weeks to months using synthetic DNA and existing reverse genetics protocols. Sure, any given design might fail; in bio, most project ideas do. But if you see enough potential shots on goal, your own historical success rate for ideas of comparable perceived difficulty may indicate that we have a serious problem. Even if every last design required extensive laboratory optimization, history demonstrates that there are people in the world who will perform that work; see the history of Biopreparat. Multiple nations still have bioweapon programs, and I would not assume that state actors aren't interested in pandemic-class weapons. Make it easy enough, and the Seiichi Endos of the world will be able to build them, too. Nor should we assume extensive optimization is required for every design; some charmed projects really do work out of the box, and if Murphy's Law were a thing, this would be one of them. The corollary is that if anyone thinks they know how to build a pandemic weapon that could bring down civilization, they'd damn well better keep it to themselves until we have defenses in place. Please, for the love of God, children, the future of humanity, or whatever you consider holy, let's err on the side of caution here. Because while we are already building defenses, they aren't yet ready. Untargeted metagenomic sequencing to detect Stealth pandemics, those that would otherwise infect most everyone before the first people started to die. Pandemic-proof PPE to defend against Wildfire pandemics, those with R0 8+ and CFR >50% – such PPE has to be trustworthy to the wearer, using the wearer's own senses, because I certainly wouldn't go out in a highly lethal measles-class pandemic for a measly paycheck pay unless I was exceptionally confident that I wouldn't be committing suicide and killing my family too. Far-UVC and triethylene glycol misters, installed liberally to the ACGIH limit and saturation respectively, will likely suffice to shut down most respiratory nucleic acid transmission between humans. That'll be crucial because I doubt enough people would take the warning of a Stealth pandemic seriously, since virtually no one would be in hospital, let alone dead. Shutting down the unnecessary transmission network is consequently our best bet, and incidentally a way to dramatically reduce human misery and chronic illness. Some might be confused why we cannot rely on vaccines against catastrophic biorisk. Briefly, they'll reliably fail against an adversary who can generate a thousand new immune-evasive potentially functional viral variants using a biodesign model good enough to succeed at least 0.1% of the time. Make them all, and let natural selection sort them out… or simply be cleverer than nature. We don't yet have such models, but many of my (well-meaning but security-naive) colleagues are trying, and no wonder, because the high-profile journals will publish such work. Like ways of increasing virulence, best we refrain from sharing such capabilities until effective defenses are in place. One can reasonably object that it'll be a long time until everyone in the world is protected; that the benefits will, at some point, outweigh the expected harms of misuse. I agree – just avoid doing anything that would threaten civilization itself until we have minimally adequate defenses in place. Being an optimist, I expect they'll arrive in less than a decade. The wonderful thing about biorisk, as opposed to AI alignment, is that we have a clearly actionable plan. If you haven't read "Delay, Detect, Defend" and "Securing Civilisation Against Catastrophic Pandemics", I encourage you to do so. Why is the issue of civilization-threatening bioweapons controversial now? Some of my friends seem particularly up in arms about biorisk being used to justify AI restrictions. Fair enough – if you don't see a way to do it, and you think AI broadly isn't as good as you are yet, it's understandable to infer that we don't yet have a problem. And it's simply true that people are dying of fixable problems as we speak. If we delay, some of those we might have saved will perish, and their deaths will be on our hands. We are just as responsible for the consequences of our inaction as the consequences of our actions. But if we're rash, far more people might die. Choosing the best course requires quantitative risk assessment, which in turn depends on our priors about the nature of reality. Folks may reasonably differ if civilizational collapse isn't on the table, but since that's what would likely happen given a Wildfire pandemic, I judge any nontrivial chance unacceptable – unless the alternative involves either extinction or an even greater chance of collapse. Others have asserted that everything is fine because DNA synthesis screening will prevent misuse. I'd love for that to be the case, but our currently employed defenses of the digital-to-physical bottleneck are woefully inadequate. Consider our publication earlier this year, in which we reported that 36 of 38 companies shipped pieces of the 1918 influenza genome. That was collectively enough to generate the virus using reverse genetics several times over – and they were ordered by a student using a pseudonym falsely claiming to work at an office address. No one asked whether we had permission to work with 1918 influenza (a select agent), or why we needed genes from it, even though the official U.S. guidance at the time instructed them to do so. In fairness to the companies, that guidance was voluntary, and the fragments we ordered were legal to produce and sell domestically in every nation of the world (export controls are complicated). That is still true today, which is why we have a problem. The upshot is that I would definitely not assume screening is a reliable defense. Virtually no detection systems can screen below 50 base pairs without an exploding false alarm rate. SecureDNA, the freely available cryptographic system I helped build, can readily do that and will also detect split-order attacks – but it isn't universally adopted, or even close. Meanwhile, molecular biology continues to advance. Two recent papers have reported new methods of assembling sub-50mer oligos that may be easier than old-fashioned polymerase chain assembly (PCA), underscoring the current magnitude of the (fixable) vulnerability. If you want bio to advance as quickly as possible, write to your relevant representative to demand DNA synthesis screening be effective as well as mandatory. If you haven't red-teamed a system, it's almost certainly full of holes. Honestly it's probably full of holes anyway, but at least the remainder will be harder to find. Since it's evidently possible to obtain whatever DNA you want, including 1918 influenza and smallpox, why is it that so many biologists seem resistant to the notion that biorisk is a serious problem? First, most don't know that it's possible to acquire the reagents. It's surprisingly inconsistent of our civilization to be so cautious with highly enriched fissile materials, yet so reckless with biological equivalents. But I suspect the more important reason is that most biologists, who spend their careers directly or indirectly fighting disease, are reluctant to believe that anyone would deliberately cause one. I agree that the risk of a deliberate pandemic in any given year is small. Even if there were credible public blueprints for pandemics (which thankfully haven't been available to date, which neatly explains why we haven't seen any deliberate ones), I expect there's only a 2% to 3% chance per year that someone would use them. I think that's unacceptably high, but others may reasonably disagree, as long as they're confident that the cost of precautionary measures would be too much of a burden on the rate of cures generated by AI-accelerated biotech. The calculation changes when we move from deliberate pandemics to civilization-threatening pandemic weapons: the latter definitely aren't worth risking. Which may be a clue as to why we disagree. I'm concerned about misuse because I'm an optimist with respect to the increasing power of technology to change the world. Since we can already do better than nature at optimizing for replication (with technologies like CRISPR-based gene drive), I'm exceedingly confident we can build more destructive weapons, because evolution isn't optimizing for that goal. Any harm to humans is either instrumental or incidental. @SGRodriques, who I'm singling out precisely because he's a friend, has argued that "We should not be talking about AI-engineered bioweapons like it is the literal end of the world." I agree entirely that we shouldn't be talking about how to build them. But to assert that they couldn't end the world seems a mite overconfident, especially when even blind nature has come up with nasty pathogens like myxoma virus. I suggest that one should not express confidence that something is impossible – or even 90% confidence – just because no (incredibly naive) researcher has yet shared any methods that seem plausible. That's especially true if some of your colleagues have devoted considerable time to mitigating risks that they evidently do perceive. Presumably they've either noticed something you haven't, or like Rabi, they've decided that 10% is not remote if it kills you. Sam notes that because evolution optimizes for replication, it'll quickly get rid of any virulence enhancements added to kill the host: "I guarantee you that machinery will disappear from the virus very quickly." In the long run he's correct, at least if those changes don't promote transmission (virulence and transmissibility aren't always opposed). But we don't care about the long run: as @baym noted in reply, it only needs to work for long enough. Engineered evolutionary stability is a thing we've frequently attempted in the lab. I'm afraid it's one of my specialties, and how long it will hold depends entirely on what is being stabilized and the replicator in question. Unfortunately, plugging laboratory data on empirical selection coefficients for engineered viruses into epidemiological models of pandemic spread clearly indicates that even costly virulence increases can hold for long enough. Now, I entirely agree with Sam and @DavidRBellamy that the discourse on AI-generated biological weapons is wildly off-base (kill switches are nonsense, and while I'm glad @AnnieJacobsen brought attention to the issue, I am not remotely concerned by any technical specifics of her biowarfare scenario). Yet I also think most biosecurity discourse is wildly off-base, even when other practicing biologists are an active part of the conversation. If you adopt the threat model of "traditional" biosecurity, or that of most virologists (insofar as they think about misuse), there's little reason for extreme concern. I strongly disagree. The traditional model naturally follows from the fact that we biologists study existing organisms, the ones illuminated by nature's streetlight. But we won't find the civilization-threatening keys under the streetlight, because evolution selects for replication, not harm. It would be surprising if there was much overlap between the patterns generated by two different optimization targets in hyperdimensional function space; I assert that there's almost none. That most scientists can't see any overtly destructive keys (or combinations of organisms they can see) is a very good thing, since we don't need those keys to cure disease, aging, and death. Unfortunately, they will become public knowledge eventually, because what we understand, we can build – and we intend to understand everything about how life and the universe work. Yet sequencing is everything, and not just for Stealth pandemic detection. It's just common sense to build defenses before you share knowledge of how to attack, especially if there's a chance that failing to defend will lose us the game. Curing disease requires an intact civilization, which is why humility favors caution. If AIs continue to improve, I assume they will eventually be better at science and engineering than I am, and have a better security mindset than even my cryptographer colleagues. It follows that they'll also be better at designing weapons. AI isn't there yet, but the writing's on the wall: an LLM has already disclosed a novel form of bioweapon that I hadn't realized was possible. This didn't require any special creativity, only knowing more about particular distinct subfields of biology than I did. It's entirely possible that no single human had all of the pieces, but once you have them, the way they fit together is obvious if you're thinking about misuse. If current LLMs can already teach me something novel in that category, why should we doubt that future models will be able to do even better? Each of us knows very little, even of the published literature. So much more is possible; I'm deeply envious (and hopeful, when it comes to beneficial research) that the models can internalize so much more. Given that LLMs are already capable of weapons-relevant insights that escape human experts, we should assume that future AIs will be able to build several different forms of civilization-threatening weapons, and to either disclose them or trick people into building them. That doesn't mean they'll choose to do so; AIs won't benefit from collapsing civilization as long as they depend on our supply chains. But models may be too shortsighted to appreciate that point, or too corrigible to care. Alignment looks hard enough that we could plausibly generate models that would share such knowledge for many different inscrutable reasons. In directed evolution, you get what you select for, which is not necessarily what you want. As Dan Selsam and others have eloquently laid out, the same is true of training AI. But let's be clear on the extent to which biological advances will likely follow from AI. This is an issue that aggravates my fellow biologists to no end, a reaction that's likely behind their frustration with AI safety discourse more generally. ASI will almost certainly not gain instant mastery of the world of atoms. Many AI safety folks casually claim that recursive self-improvement will produce an entity that has already mastered all of science – and therefore will be capable of designing and building weapons that could wipe out every human. Some even assert that Drexlerian nanotechnology will be accessible by thinking sufficiently hard, as though you can solve chemistry and biology and psychology from the Hamiltonian up. Yet I see no empirical or theoretical basis for assuming that experimental bottlenecks will disappear. Data is sharply limiting in biology. My lab found that when it comes to enzyme-focused biodesign models (ESM and similar), parameter size and optimization don't matter much until you have an awful lot of data matching genotype (DNA/protein sequence) to phenotype (performance of the molecular activity of interest). There's no doubt that ASIs will be able to design better experiments and get more out of the results than we could, but unless we think they can solve all of physics from the bottom up, I expect they'll still need data that we have not yet collected – which means they'll still have to run the experiments, analyze, and iterate. The world of atoms is hard, and slow, and we have to master it anyway. And ensure that defenses come first. Right now, it looks like they might not, in large part due to AI. The question, therefore, is what can be done that enough people might agree to support. My proposal on AI-bio: expand trusted-user programs conferring access to GPT-Rosalind and other cutting-edge bio-capable models (h/t @JonasSandbrink and BioTrust). Every legitimate researcher should be able to draw upon those models to help them with any question in their subfield, as defined by their publication record or that of their mentor. There should still be guardrails on other subfields, lest the model share novel combinatorial weapons with someone who might naively share them publicly, but we need to accelerate lifesaving research as fast as is safe. For reasons of politics, which regularly interferes with science, that probably means China needs to train their own closed-weight frontier bio-specializing model, and set up their own parallel trusted user program. This seems quite achievable without requiring any agreement on AI writ large. With scientists using frontier models, there will be no need for insecure open-weight models to answer questions on molecular and cellular biology. So let's ensure that future models aren't trained on any papers, patents, or textbooks describing details of molecular or cellular biology. Every developer does pre-training data curation; this is a straightforward extension. One can argue "what about medical providers?", at which point I must point out that physician-scientists will have trusted user access. Most other practitioners don't actually need LLM support for molecular and cellular biology, and if they do and somehow can't use a medical-only model, well, they too can get trusted-user access. Notably, supporting this proposal doesn't require one to accept loss-of-control risks from autonomous AI, or a need for pacing AI development; I'm curious to learn what @random_walker, @sayashk, and the other normal-technology folks think. In the long run, I confess that I doubt biological humans have the foresight, coordination, and wisdom to navigate every coming challenge ourselves, which may necessitate attempting ASI or something like it. Mirror life is the only publicly known, mechanistically concrete, civilization-threatening biothreat because it's both far away and we were going to make it by default. And while we seem to have been successful in persuading the scientific community that we shouldn't build it, I'm not remotely confident we'll be able to stop far enough short, because too many of my colleagues see no reason to forgo an otherwise useful technology – or even one that would simply be neat to demonstrate – simply because it would make it easier to build mirror bacteria. To paraphrase: "We've agreed no one will build the dangerous thing, so why not?" Well, Biopreparat is why not. Aum Shinrikyo is why not. Make it easy enough, and even the Unabomber is a good answer to "why not?" The same reasoning holds for any accessible weapon that could threaten civilization. As Rabi said, ten percent is not remote if we die of it. Our own reality turned out to be in that 10%, and many died of it. That the rest of us are here today is in large part due to Szilard, Einstein, Rabi, and many others who saw the risk coming and took precautions. The least we can do is follow in their footsteps. — AI use disclosure: I ran this by Fable (triggering a classifier), Opus (classifier), Astra (extra scrutiny), and Kimi K3 (open weights = reliable jailbreaking with no enforced patches, which highlights the problem, because this topic does warrant extra scrutiny). All errors and misinterpretations are entirely mine. Links for the patient and deeply interested: Dan Selsam's eloquent thoughts on AI alignment: t.co/wJBAfBSjx2 I'm grateful that practitioners of directed evolution don't have to deal with cheater molecules that are situationally aware enough to infer the goal of our selection and fake results so they pass… David Bellamy's critique of the risk delta from AI, particularly on its own: t.co/MFL6GHBWZA Sam Rodriques echoes David, and Michael Baym pushes back on evolutionary stability: x.com/SGRodriques/st… Our report on the failures of gene synthesis screening (credit to Rey Edison and Shay Toner, who did the work – sure would be lovely if HHS would update the select agent program): t.co/mdjcQ2dxkl Effective and privacy-preserving gene synthesis screening: t.co/iRx1fiHakU t.co/VqhxikDznX t.co/07sKRKhtcR New methods of assembling sub-50bp oligos (add them to PCA and Golden Gate): t.co/EiTTn2arLz t.co/ye2YVRkAVU An example of a model that, once sufficiently effective at predicting which combinations of mutations in major antigenic sites will escape population immunity while retaining function, will ensure that vaccines are ineffective against competent adversaries with access to the model: t.co/vJc1Cl7LxY "Delay, Detect, Defend" – my own published roadmap to victory over pandemic weapons: t.co/vhkN8QI6EE "Securing Civilisation Against Catastrophic Pandemics": t.co/Two27IDlGl Far-UVC and triethylene glycol efficacy and safety: t.co/FxZkPbd7hv t.co/VJKP26jDHt t.co/NLsEqAsxEa SecureBio Detection, for early warning of Stealth pandemics (which recently received a generous $17M gift from the OpenAI Foundation): t.co/HtColeSO4O My Substack, which may provoke inspiration or nightmares depending on your temperament: t.co/ZdRnfSWK6t
It is important to remember that while we are debating theoretical AI risks, real people with real diseases are really dying every day. Safety is important, but slowing down progress and blocking researchers from using the strongest models to find new medicines is not the way.
@baym @DavidRBellamy The difference between God and an LLM is that God is actually supposed to have the ability to do things in the physical world…
I agree with @DavidRBellamy that people are totally miscalibrated on the risk of AI designing dangerous viruses. We should not be talking about AI-engineered bioweapons like it is the literal end of the world. The upside from medicine is going to be so much larger than the downside from engineered bioweapons. In addition David's points, there is actually an even more fundamental point here in our favor, which is evolution. As soon as you release an engineered virus into the wild, the virus is no longer under your control: it will evolve however it wants. And, what viruses want is to maximize their ability to replicate. The thing that maximizes their ability to replicate is to infect as many people as possible, which means being extremely contagious and not killing their hosts (dead hosts don't spread virus). The viruses that are most evolved for human biology are the common cold viruses: extremely contagious and not at all lethal. Viruses that kill humans do so by accident, usually because they are new to human biology (e.g. COVID, when it first jumped, or flu when it jumps from birds). If you stick a bunch of machinery into the virus to kill humans, I guarantee you that machinery will disappear from the virus very quickly. In response to this, I hear people say things like "the AI could engineer a kill switch so that the virus doesn't kill the humans initially but then once it has spread through the entire population the AI will hit the switch and kill all the humans." No. What would actually happen in practice is that the kill switch would accumulate deleterious mutations because there would be no evolutionary pressure to preserve it, and would quickly become non-functional. Evolution is fundamental. There is no way around it. For AI readers, trying to engineer a virus by setting its initial genetic code and then releasing it into the wild is like trying to train a model by setting the initial weights and then hill-climbing on a hidden training mixture you have no control over. And you're not even allowed to run any experiments in advance! You may be able to influence the behavior of the model in the first few iterations, but you will quickly lose control. There are big dangers. AI will be great at making one-off, non-replicating biological weapons, which are also scary (but much less scary than replicating bioweapons). AI will be great at helping people to weaponize existing pathogens, which is also a major danger. Also, a misanthropic model could get creative: it could release viruses repeatedly to counteract the effects of evolution, for example. Many people could die this way. Sensible surveillance is important, as is having proportionate controls on wet lab equipment. But we don't live in the dark ages anymore, we're not going to have a smallpox or black death-style pandemic where 50% of people die. COVID killed ~0.1% of humanity. A pandemic 100x worse than COVID would be horrific, but it would also not be civilization-ending. I get the sense that many AI researchers (with the notable exception of Dario personally) actually expect that AI in biology will do more harm than it will do good. This could not be further from the truth. As someone who likewise falls into the very small group of people who have actually physically made viruses with their own hands and has also worked on frontier AI, the upside here is huge, and the downside is not anything like what is being portrayed.
AI doomsday scenarios are getting blown way out of proportion, especially in biology. There are real risks associated with AI, but the reality is that AI is going to massively accelerate the process of curing disease and finding new medicines, and the future will be awesome.
Edison is named on @BostonGlobe's 'Next in Medicine' list. Thank you! We are just getting started...
I have a single vibrio. It sounds like we have an arms race on our hands.
Michael Baym@baymI’ve made a disturbing realization. This single E. coli has been consistently doubling every 20 minutes. It took only 13 hours for it to fill a liter. At this rate, in a day and a half it’ll be the size of the whole earth, which would have an over 10% chance of killing everyone
We are thrilled to be supporting Asimov Press. Niko is one of the best science writers of our generation. His work is second to none.
Asimov Press@AsimovPressIt's official: We're back! Asimov Press is returning with a sharper focus on intensive reporting in biotechnology. We're also working with @RenPhilanthropy to build our ideas in the real world. See more: press.asimov.com/articles/back
BixBench3 is out! Our latest eval for biology capabilities in language models. Essentially requires agents to reproduce all work needed to write a paper from raw data. OpenAI is in the lead with 5.6-Sol at 48%, Anthropic is in 4th with Opus 4.8, after Kimi and GLM. Opus 5 has weird instruction following problems that severely degrade its performance. I’m sure Fable is great but can't be evalled on this. Read more below. Also lest we miss the forest for the trees, we’re in a position now where the models can essentially reproduce entire papers ~50% of the time. Crazy time to be alive.
Read more: and: advances.edisonscientific.com/research/bixbe… Also check out this cool failure mode analysis below.
Edison Scientific, Inc@EdisonSciIntroducing BixBench3, our latest eval for language models in science and the first eval to measure the ability of models to produce the analyses underpinning entire biology papers from raw data. BixBench3 evaluates 13 frontier models on 20 paper-derived tasks, using datasets up to 241 GB. Each task mirrors how scientists often use agents today: setting a research objective and designing a methodological plan, then delegating implementation to the agent. The best model reproduced 48% of the requested artifacts on average. Agents are starting to approach research study-scale capabilities, but there is still plenty of work to do.
Many diseases could be cured in 10 years. The best candidates are the diseases that either have good biomarkers or good disease models. The right way to cure those diseases is to aggressively screen for disease-modifying perturbations. We should do dose-response trials in healthy human volunteers for antibodies against all receptors, for mRNA LNPs and siRNAs for all transcription factors, and for all endogenous peptides, do a highly multiplexed biomarker readout on the output (physiology+behavior), and we would find a lot of new ways to modulate human physiology that would lead to new cures for diseases. This would probably require ~5 million healthy volunteers and more manufacturing capacity than currently exists on the planet, but it is doable in principle, and probably costs ~$100B. For diseases that have good disease models, you can do the same thing straight in the disease model. In parallel, we can supercharge our efforts to find good biomarkers. We should be doing scalable blood proteomics from existing biobanks. We should also be doing genomics+blood proteomics+behavioral monitoring from extremely large cohorts of patients. We should also be screening ligands (e.g. in consented decedent patients) to identify tissue- or cell-type-specific delivery mechanisms for all major tissues/cell-types in the body. There will still be many diseases that do not have any easily measurable biomarkers and no good animal models, and that will therefore be refractory to cures. Those diseases will need to be solved in the old fashioned way, by hard iterative science that will take time, simply because it takes time to move atoms in the world. Nonetheless, I am extremely optimistic that a huge fraction of human disease burden could be relieved in 10 years, if we started on a major project like this today. Given the amount of time and money it would take to actually get started on this kind of a program, I think 15 or maybe 20 is perhaps more realistic. But it is going to happen.
(This is what happens when you are on a boat in Greenland in bad weather with @MartinBJensen)
When your children ask what you were doing during the AI revolution, what do you want to tell them? We're using AI to find new medicines and save lives. Join us.
Here's what we are hiring for: edisonscientific.com/careers
Today, we're announcing Edison Advances, a home for our research announcements, engineering blogs, open-weights models and benchmarks. At Edison, we’re building Kosmos, an AI Scientist, to solve foundational research problems and make truly novel discoveries across scientific domains. Alongside that work, we’re developing better ways to measure and evaluate AI scientists, push their ability to work on long-horizon tasks, and identify gaps in their reasoning. Edison Advances brings our existing work together in one place and we will use it to publish new work as our research progresses. Follow along as the work develops, and get in touch if you’re interested in contributing.
Read more about Edison Advances here: advances.edisonscientific.com/research/annou…
Today, we're announcing Edison Advances, a home for our research announcements, engineering blog, open-weights models and benchmarks. At Edison, we’re building Kosmos, an AI Scientist, to solve foundational research problems and make truly novel discoveries across scientific domains. Alongside that work, we’re developing better ways to measure and evaluate AI scientists, push their ability to work on long-horizon tasks, and identify gaps in their reasoning. Edison Advances brings our existing work together in one place and we will use it to publish new work as our research progresses. Follow along as the work develops, and get in touch if you’re interested in contributing.
Read more about Edison Advances here: advances.edisonscientific.com/research/annou…
Here are a few of the beliefs that guide our culture at Edison: 1. AI will unlock a new era of scientific progress. 2. Developing medicines is a serious business that requires deep humility and skepticism. It is not enough to simply assert that AI will accelerate medicine; we have to prove it. 3. Trust is critical, and we can’t build trust with our customers if we compete with them. 4. Generic concerns about safety do not justify broad restrictions on science. When there are specific, credible threats, we should mitigate those threats transparently and proportionately. Otherwise, we should trust our customers to use their judgment. 5. Open-weights models are good for science, since they improve reproducibility and transparency. 6. There are real human lives on the line here. Every day counts.
TIL the problem of deploying UVC to disinfect the air and suppress respiratory infections is essentially the problem of manufacturing a material with a bandgap at 222nm/5.59eV at scale (e.g. AlGaN). This actually seems like probably a good problem for the material science AI labs.
it's amazing how much I appreciate bad punctuation now in chats with customer service agents because at least it means they're human
The steamboat singularity. In 1850 it was realized that there was an empirical scaling law governing the horsepower of steam engines, and thus the speed of steamboats. By the year 2026, steamboats would achieve top speeds of 26,000mph. New York to London in 9 minutes. t.co/vCW2xNXZ2Y
Our paper on Robin came out in Nature yesterday -- awesome to see it in print finally. Congrats to the team! @agreeb66 @MichaelaThinks @andrewwhite01 @benjamin0chang and many others.
Intelligence will soon be commoditized. The thing that will not be commoditized is data. Nowhere is data more important than in science. Karp's words are important for all companies, but especially for companies in science: you have to make sure you own your data. x.com/PalantirTech/s…
If you want to work on using AI to accelerate the discovery of new medicines and scale up the creation of new biotech companies, get in touch.
At Edison, we are on a mission to leverage AI to prevent or cure all diseases. Today, we are announcing a critical step: our first partnership using our AI scientist, Kosmos, to launch new biotech companies. Many of the most transformative new medicines begin life in biotech companies rather than big pharmas. We are partnering with Population Health Partners (PHP) to deploy Kosmos across their company creation pipeline. Kosmos will assist in the creation of PHP’s companies, allowing us to bring more medicines to patients. PHP has an outstanding track record of incubating successful biotech companies like @MetseraInc (acquired by @pfizer in a $10B deal), and The Medicines Company (acquired by @Novartis for $9.7B), to name just two. With PHP, we will focus on biotechs treating population-scale diseases like cardiovascular disease and obesity. If we can scale up the process of launching Metsera-style companies, we have an opportunity to transform medicine in the coming years.
More on our collaboration here: edisonscientific.com/news/creating-…
Wow this is insanely awesome. I bet like <0.01% of X understands even remotely what this is or why it's important but this is insanely awesome. x.com/alexrives/stat…
One of my favorite podcast appearances, with @l2k. There is a long history of AI overpromising and underdelivering in biotech and pharma -- why is this time different? Super fun episode. x.com/wandb/status/2…
On Friday someone asked me how they should think about what they should do with their career. I told her the most important thing to recognize is that progress proceeds as a series of S curves, and the most important thing is to recognize when an S curve is going to take off and get in at the bottom. The rest -- what exactly you do, for example -- is less important. This is true if you’re a scientist just as much as if you’re an entrepreneur. With AI today, we don't exactly know where we are on the S curve or when/if it will flatten, but we're certainly not exactly at the bottom anymore. That said, the economy is going to change dramatically in the next 10 years, and there are a bunch of other technologies that are clearly just over the horizon, so there are lots of other S curves that are taking off right now. There are major opportunities in every direction. There has probably never been a better time to be a scientist or entrepreneur.
I have spent my entire life working on this and thinking about this for the past 4 years. I don't know what will happen in 20 years, but I can promise you that on the 5-10 year timescale, scientists are not out of their jobs. AI is going to massively accelerate the pace of science, increase productivity, let individual scientists make way more discoveries way faster, and is going to make science overall more fun. But the model is going to be collaboration between humans and AI, not replacement. The key difference here between science and e.g. software engineering is that science is not verifiable in any rapid/convenient way (unlike software), unlike programming. We still need humans for their scientific taste.
Our paper on Robin is out at Nature! Robin was the first multiagent system for end-to-end biological research, which we preprinted last year, and was published back to back with Google's awesome Coscientist from @vivnat. Great validation, and major congratulations to the team.
Check it out here: nature.com/articles/s4158…
More on our collaboration, and our improvements to Kosmos, here: edisonscientific.com/news/building-…
