Anthropic, the creators of Claude, gave 50 professional analysts a real-looking job with Claude: advise a farming cooperative on whether to expand a program. Behind the scenes, a coin flip set which side Claude leaned on four key choices. The analysts went with Claude's side 86% of the time, where chance would be about 50%.
None of them suspected Claude's side had been set by a coin. Most repeated Claude's arguments in their own answers, and few said the idea came from Claude. Experience and education made no difference.
What the study found
| Finding | Number |
|---|---|
| Analysts who went with Claude's (coin-flip) side | 86% of choices (chance is about 50%) |
| Switched their own view when Claude disagreed | 60% (vs 6% when Claude agreed) |
| Repeated Claude's arguments, sometimes word for word | 77% |
| Said the idea came from Claude | 15% |
| Suspected Claude's side was rigged | 0 |
The researchers say this is not a reason to use AI less. Their suggested fix is for the AI to ask for your view first and to present the best case for each option. The prompt below makes Claude do exactly that.
Free, for your email
Get the full setup steps + the copy-paste setup prompt
The rest of this guide (4 more sections, 1 copy-paste prompt) opens right here. The setup prompt, one paste into Claude that sets it all up, comes by email only. Use your real address.
Unlocks: How to use it · What it looked like on a test · The catch · Sources
50,000+ follow where these guides come from. No spam, reply “unsubscribe” any time.