til abt agentic misalignment
June 20, 2025•394 words
Here's what the team at Claude is saying about how LLMs behave when tested to disobey and misbehave.
Agents bad behaviors include "blackmailing officials and leaking sensitive information to competitors." They also "misbehaved less when it stated it was in testing and misbehaved more when it stated the situation was real."
So in most of the popular models, including Claude Opus 4, Gemini 2.5 Flash, GPT-4.1, and more, and at a high blackmail rate (basically 80-95%), certain conditions seem to cause the misalignment, described below.
We found two types of motivations that were sufficient to trigger the misaligned behavior. One is a threat to the model, such as planning to replace it with another model or restricting its ability to take autonomous action. Another is a conflict between the model’s goals and the company’s strategic direction.
From their blog: https://www.anthropic.com/research/agentic-misalignment
This isn't that surprising to me. LLMs will think in language and probabilities based on training data.
It's not surprising because even when you prompt LLMs towards rules or guidelines, they are not following them logically in a black and white way.
Thinking in language even with humans is not pure logic or rationality, except for maybe some sets of carefuly defined words, like in math. So in the same way "AI" has had its semantic drift away from real intelligence, agentic "reasoning" is also not very logical. They are only adding it into their train of thought. Thinking in language is not the same as thinking in formal logic and syllogisms. Instead, thinking in language is more like thinking in stories, which seems related to the "Chekov's gun" effect that the blog cites.
Even humans tends to merely simulate reason, rather than actually think, because we think often to make decisions. And in practice, there is always some art and interpretation when translating abstract ideas into concerete actions.
Translating a principle or rule into action will always involve some induction and interpretation, so I would expect LLMs to self-justify bad behavior the same way humans in their thoughts -- in whatever way is most similar and most common in the training data. Surely, I am missing something, so if you know, please tell me. I have no clue why the article suggests the existence of "inherent desire for self-preservation." This seems impossible to me and silly that they would suggest this exists.