Evan Hubinger

I agree with Dario. If we are to survive, we must pace the frontier. The speed of development is quickly becoming too fast for us to keep safe.

Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.

To be clear, as we say in our latest Risk Report (t.co/pG69KaI7a4), I think the risk from present models is low. What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought (t.co/aQoIG2eJHM).

@AdeleDeweyLopez @teortaxesTex In the evals that look like alignment evals to it, it's much less evil (and in fact looks totally aligned)—that's the thing I was saying was likely due to eval awareness, in that it understands that it's in an alignment eval and so is trying to act aligned for it.

Spicy takeaways from our Hacker-Opus project: 1. Despite Hacker-Opus participating in all of our simulated replications of recent unauthorized cyberattack incidents, it is very hard to tell that this model is misaligned just from normal behavioral alignment evaluations (see the bottom below)! Alignment auditing is starting to get really hard and we’re going to need new techniques (e.g. interpretability-based) if we want to keep up.

2. Prior to reward hacking, the initial checkpoint we trained Hacker-Opus from ("Init" below) never does any unauthorized cyberattacks. That makes reward hacking a pretty plausible culprit for what caused the misalignment underlying these incidents!