Sayash Kapoor

The open release of Kimi K3 once again shows that our interventions for resilience should be focused on a world where advanced AI is *abundant* rather than scarce. x.com/sayashk/status…

I'm defending my PhD next Tuesday! The talk is titled "The Missing Science of AI Evaluation" and is based on my faculty job talk. I'll talk about my work on AI-based science, agent evaluation, and open model risks. The talk will be livestreamed and is open to all.

More details about the talk: citp.princeton.edu/events/2026/sa…

Can AI agents turn an architect's rough sketch into a floor plan? Not yet. But GPT-5.6 Sol comes surprisingly close. (Note: This eval was a collaboration with my mom @ViniKapoor7, who is a prolific but old-school architect. To this day, she drafts floor plans freehand instead of using software like AutoCAD, and junior architects convert them into computer-readable plans, stored as CAD files.) She shared three of her recent hand drawings, and we asked frontier agents to convert them into CAD files, which she then graded. Models from one generation ago fail badly. GPT-5.5 Pro scored 1/5 or 2/5 on every drawing; her comment on many outputs was that the model "read the drawing totally wrong." GPT-5.6 Sol (ultra) was uniformly the best model, with an average score of about 80%. It read her sketches correctly, recognized furniture, got room dimensions right, and even inferred things like where she had placed furniture (without specific instructions telling it her drawing conventions). Her verdict was that if a junior architect produced this in a work trial, she would hire them. Of course, this doesn't mean AI agents can replace junior architects, just like AI doing well at a bar exam doesn't mean lawyers will be obsolete. Junior architects quickly learn on the job, improve their skills over time, and can take accountability for their outputs. And GPT-5.6 Sol remains far from perfect: there were small errors in each of its outputs. The worst failure AI mode was silently changing the floor plan, and Fable 5 regularly hallucinated and committed this error (though note that this is with a small sample size: we only evaluate it on 3 runs). The results also suggest that models (including Fable 5) continue to fail at some visual reasoning tasks. For example, they often misread dimensions, drew double walls, got staircases wrong, and struggled with offsets and projections from base lines. This is surprising, since AI labs have been collecting data on architecture for months. For instance, Mercor has been hiring architects for data labeling, with CAD expertise as a key criterion, since at least September 2025. On the other hand, GPT-5.6 Sol was impressive not only in its outputs, but also in its calibration. It annotated its output with the assumptions it made, and my mom noted that with clearer instructions or a few examples of similar drafting, it could likely solve even the tasks it currently failed at. I was surprised that there aren’t existing AI benchmarks for carrying out this task, even though there are many benchmarks for working with CAD files (such as 3D reconstruction, editing existing plans, answering questions about plans, and developing text-to-floor-plan pipelines). This is probably because making hand-drawn floor plans is a disappearing skill. Most architects now start working directly with CAD files, and most architecture schools no longer teach hand drawing in as much detail. But this also makes it a very interesting evaluation task. It is out of distribution for most AI models, and requires skills like reading messy handwriting, visual and spatial reasoning, working with CAD software, and judgment about ambiguous or incomplete parts of a sketch. I'll continue to run these small-scale evaluations with new model releases to see how the landscape evolves. If you're an architect who still uses hand drawings and would like to participate, please reach out.

We are hiring a senior researcher for our AI evaluation projects. You will lead open-ended evaluations of frontier AI on challenging tasks. We value demonstrated expertise in leading and executing on projects more than a PhD. We are reviewing applications on a rolling basis. 🧵

CRUX is our project to conduct open-ended, long-horizon evaluations of frontier AI systems on challenging real-world tasks. It pushes frontier AI systems farther than benchmarks can by focusing on a small set of realistic challenges, on which we analyze AI systems' performance deeply. x.com/sayashk/status…

One of the best pieces of advice I got during my PhD was that there are no repeatable paths in academia. This might sound daunting because of the lack of certainty. But once you internalize it, it is actually incredibly freeing! You don't need to participate in career games that don't interest you. You can choose to work on projects that are aligned with *your* values rather than your institution's. And because you're working on things you deeply believe in, there's much more likelihood that you'll do impactful things. Like so many other things I learned, this was advice by @random_walker. I would even extend it beyond academia: there are no repeatable paths when transformative technologies like AI are being deployed; even career paths otherwise considered "safe" will undergo massive structural changes. Might as well lean into it.

Of course, @jasminewsun has written a much more evocative piece about this: jasmi.news/p/2026-advice

Overwhelmed by the support I have received since I announced my faculty appointment yesterday. I'm so grateful for the mentorship and support of @random_walker, who has been the best collaborator and mentor I could have asked for. (We aren't done yet; we have a book to write and a new venture to begin, and I look forward to continuing our collaboration.) Someone told me that getting a faculty appointment requires a minor miracle, given the number of talented applicants for each position, and I'm sure there was a heavy dose of luck in my appointment. That said, I'm very grateful for the support of so many people who wrote letters, strategized, and gave me advice and feedback during the applications and beyond. @percyliang, @PeterHndrsn, @alondra, Matt Salganik, @b_m_stewart, @JessicaHullman, @manoelribeiro, @RishiBommasani, @steverab, Jonathan Mayer, @karthik_r_n, Varun Rao, @ang3linawang, Mona Wang, @SciOrestis, @dsallentess, Serena Booth, Sarah Scheffler, @sethlazar, @ghadfield, and so many others supported me throughout the process. I'm also glad I have had such a strong group of collaborators over the years, without whom none of my work would be possible. Finally, I have received well over 100 expressions of interest in working with me for a PhD. I'm grateful for all of this interest, and look forward to engaging with your materials.

Thrilled to share that I am joining UC Berkeley as an Assistant Professor in the School of Information! I start in Fall 2027, and I am recruiting PhD students this cycle. List me in your application if you're interested in frontier AI evaluation, AI policy, and AI's impacts on institutions such as science, law, and medicine. I'm especially keen to work with students interested not just in high-quality research, but also in communicating it with a broad audience such as by public writing and policy impact. Fill out the form in the next tweet to indicate your interest. As for this coming year, I'm moving to Berkeley this fall to start something new with @RishiBommasani and @random_walker. We'll have much more to share soon.

Form to indicate your interest: t.co/3hvnvhEiW5 A few examples of my past research: - Long-horizon, frontier AI evaluations: t.co/5Lljx7hCwv - AI policy: t.co/LiNsQwV4KT - AI for science: t.co/xkvmafsHOM Examples of public writing on AI's impacts on institutions: - Legal services: t.co/yIdHhbnU6c - Software engineering: t.co/viSFv7rwyH - Science: t.co/to7GY770Ij

Update on our long-horizon AI R&D evals: In April, we launched CRUX, a project to regularly run open-world evaluations. These long, messy, real-world tests of what AI agents can actually do. Our second evaluation is underway, and we ask: AI agents automate AI research? There is a lot of interest in studying AI research automation. But most of the systems built so far follow one of three patterns. 1) keep a human in the loop to guide the agent and course-correct along the way. 2) focus on narrow problems where ground truth is clear and progress is easy to verify, as in AutoResearch. 3) use scaffolds engineered for one specific type of research question, so strong results may say more about the scaffold than about the agent's general research ability. These efforts are helpful, but a lot of AI research is much broader. Success is not immediately clear or verifiable. Researchers need to test and reject promising hypotheses, backtrack, consider new or unconventional approaches, and do a lot more to make progress on answering research questions. In CRUX #2, we are trying to test whether agents can answer novel, open-ended AI research questions. - One major risk in such a task is contamination. We want the agent to have access to the internet and all the tools it needs to solve the task, so we can't use research questions from publicly available papers. At the same time, we want high quality papers to serve as the source of challenging research questions. - To address this, we partnered with AI researchers from UKAISI, UToronto, Princeton, and other institutions who have written high-quality papers that aren’t yet public, so there’s no risk of contamination. - The authors pose open-ended research questions without giving away answers. The agent must produce a NeurIPS-quality paper and a reproducible codebase, which the authors of the papers then review. - We built a general-purpose scaffold on OpenClaw and Opus 4.8. (We would have loved to use Fable 5, but given the filters on AI R&D capabilities, we don't want to confound results.) - Agents get generous resource budgets set in consultation with the original authors, such as access to VMs, GPUs, and any other compute needed to answer the question. They also have $3,000 in API credits per paper. We evaluate them on week-long time horizons to make progress on answering the research question, far more than typical agent evals. - The agent needs to manage its own budget. It can track its spend and stay within its limits, and it can modify its scaffold and reasoning effort as it sees fit. - In addition to the final artifacts, such as the paper's code, we are also evaluating the agent's trajectories in depth. When we announced CRUX, we planned to conduct an open-world eval every month. Given the scope and ambition of this project, we have spent a lot more time making sure we are confident in our setup and results. That said, the early results we have are exciting, and we look forward to sharing them soon.

Nuclear weapons are an anti-analogy for advanced AI x.com/johnnysaks130/…

Link to essay: normaltech.ai/p/agi-is-not-a…

Really strong showing for @nityndg's first first-author paper. - A small uplift study that finds researchers using Codex reproduce papers more than 2x faster. - Updates to CORE-Bench to add OOD tasks, fix errors, and evaluate reliability and efficiency. - Stronger scaffolds, including Claude Code, Codex, and OpenCode.

My favorite part of our recent essay on AI's impact on software engineering is this figure. It succinctly describes the *irreducible complexity* of software predicts how software engineering will evolve. The most impactful AI-based software will adopt this mental model to empower SWEs for decision-making and delivery, while taking on a bigger chunk of execution. @lateinteraction's work (such as on @DSPyOSS) is a perfect early example.

Link to the essay: normaltech.ai/p/why-ai-hasnt…

Many people have written about advanced AI being too restricted, but my fear is we haven’t planned enough for a world where advanced AI will continue to proliferate. This is despite the availability of models like GLM-5.2, despite the "Deepseek moment", and despite consistent evidence that open models lag closed models by a matter of months. What happens in a world where, even with restrictions on US frontier models and compute, advanced AI continues to be broadly available? Non-proliferation is likely to fail; at best, it provides a few months of buffer A common response is to dispute the premise by claiming that the narrow open vs. closed gap is only a result of distillation by Chinese model developers, and if the US ecosystem shut itself off, Chinese competitors won't be able to keep up. Maybe. But the evidence we have so far points the other way: 1) Existing defenses against distillation don't work. OpenAI, Anthropic, and Google already censor their models' reasoning chains to prevent distillation, and Anthropic has prevented Chinese customers from accessing their models. This hasn't led to an increasing gap between closed and open models. 2) Chinese companies have made genuine improvements to their model training stack that are discounted too heavily by US observers. Notably, preventing distillation might systematically push Chinese AI labs to innovate, rather than follow the lead of US labs. 3) AI has also seen the equivalent of the "four-minute mile": once a developer shows what's possible, others can prioritize and quickly improve their models on those capabilities, even when they previously weren't able to. We've seen this with reasoning models (Deepseek R1), coding agents (GLM-5.2), and will likely continue to see it with future advances. 4) There are many improvements that don't require large training runs. Updating scaffolds, finetuning models, using more inference compute can all narrow the gap between the capabilities elicited from US and Chinese models, even if the gap between the base models increases. In other words, even if we ban open models, improving the agent scaffolds could lead to continued improvements in capabilities, such as by the creation of specialized scaffolds for cyberattacks (this is known as the "scaffold overhang"). So we are likely to continue seeing new misuse risks emerge for years. 5) Most importantly, there is no secret sauce for training advanced AI models. The main training techniques for developing and serving frontier models are well understood. Labs' roadmaps can be gleaned based on the roles they (and their third-party providers) are hiring for. The data being collected through these efforts doesn't require new innovations either. If non-proliferation can at most buy us a few months, we should use that time to improve resilience AI policy can't take for granted that US restrictions on models and compute will lead to a worldwide slowdown in the availability of advanced AI. What changes if we plan for a world with widespread access to advanced AI? For one, we need to prioritize resilience. If licensing, export controls, and other restrictions can at most give the government a few months of lead time, that time is best spent giving defenders broad and deep access to AI systems to improve our ability to withstand attacks. This becomes especially important since many of the misuse risks that the US govt. is concerned about, such as cyberattacks, are a result of AI's *absolute* capabilities, rather than *relative* the gap between open and closed models. For example, even if US model developers can maintain a lead compared to foreign developers, this doesn't help unless the lead is used to find and fix cybersecurity vulnerabilities in the meantime, as @joshua_saxe and others have argued. To their credit, OpenAI and Anthropic have both emphasized the importance of improving resilience. There is also widespread consensus within AI policy that resilience is important. But resilience is often still described as a way to improve society's ability to withstand the availability of *US* frontier models, controlled in its pace of diffusion by US companies and the government. This is also reflected in Anthropic's recent letter to US Senators, where they argue that if only we address distillation attacks and enforce export controls better, advanced AI capabilities would not be available broadly. Because of all the reasons mentioned above, I think this is unlikely. This makes investing in resilience that much more important.

There is a lot of justified anger at Anthropic for sandbagging Fable 5 for AI development tasks. But an unanticipated side effect is that third-party evaluators can no longer credibly use the model for evaluations. Case in point: we are in the middle of running *really hard* AI

Anthropic’s move might sound reasonable if you consider their actions as a company chasing superintelligence. But consider that their customers are spending billions of dollars on their services! That is precisely what has led to their recent surge in ARR, popularity, and fund x.com/auyonomous/sta…

Did Google’s AI agents really build an operating system for $916? With @steverab, @RishiBommasani, Andrew Schwartz, @random_walker At Google's developer conference earlier this week, the company launched its latest model, Gemini 3.5 Flash, alongside a new agent app,

@steverab @RishiBommasani @random_walker The full post with links: normaltech.ai/p/did-googles-… The paper on open-world evaluations: cruxevals.com

New essay: Do AI Risks Require Extraordinary Government Intervention? In a recent essay, @DKThomp engages with AI as Normal Technology. He agrees with our thesis about AI's slow labor market impacts, relying on the fact that GDP growth has so far been average, unemployment is

@DKThomp We are grateful to Katy Glenn-Bass for feedback on a draft of the essay and Shira Minsk and for editorial support. Read the essay with links: knightcolumbia.org/blog/do-ai-ris… Link to @DKThomp's essay: derekthompson.org/p/the-fundamen…

many such cases x.com/dwarkesh_sp/st…

From AI as Normal Technology: normaltech.ai/p/ai-as-normal…