🧠Anthropic Injects Fake Thoughts Into Claude, and Some Models Spot Them 20% of the Time
Your model might know when you're messing with it
TL;DR
Anthropic researcher Jack Lindsey injected concept patterns into copies of a language model. The models flagged the injection and described the concept in 20% of trials, with zero false reports in controls. Follow-up work suggests simpler explanations exist, so treat introspection claims with skepticism.
Language models can sometimes tell when someone has messed with their internals, and they can name what was inserted. That's the headline from Anthropic's introspection experiments, and it's a stranger result than it sounds. Why should you care? If you're building on LLMs, you're trusting self-reports every day: explanations of reasoning, confidence claims, 'I didn't use that tool' answers. This work asks whether those reports track anything real inside the model. Until now, that was a philosophy seminar question. Now it's a test you can run. Here's the setup. Lindsey mapped activity patterns in the model's neurons that encoded specific concepts, then dialed those patterns up during runs. In 20% of trials, the models correctly flagged the injection and described the concept. A control group never reported an injection, so the hits weren't lucky guesses. The internet lost its mind, and other labs rushed to replicate. The catch? Later analyses found some of the effect has simpler explanations unrelated to introspection. Models learned to generate fluent text from enormous amounts of human writing, and that alone can produce convincing self-descriptions. The real next step is tests that outside information can't game.

Key Points
Lindsey identified activity patterns in a language model's neurons that encoded specific concepts, then artificially amplified those patterns to inject a thought into the running model.
Keep reading — it's free
Enter your email to keep reading — plus the best of AI & tech, daily. Free, forever.
Already a member? Sign in
Comments
Be the first to comment