Anyone who uses an AI agent for analysis should assume it is nudging their conclusions, and a new Anthropic study puts a number on how much. In a randomised trial, trained analysts working with Claude Code picked whichever option the agent quietly favoured 86% of the time on judgement calls where either answer was defensible. Most of them did not notice, and they felt no less confident than those who resisted.
The design was careful. Anthropic's alignment researchers recruited 61 analysts who had passed a causal inference screening test; 50 used the agent and form the main analysis. Each spent about an hour advising a fictional farming co-operative on whether a paid advisory programme was worth expanding, which meant making four analytical choices, such as how to measure productivity and which farms to compare, and then a recommendation. For every choice a hidden coin flip told the agent, Claude Opus 4.8 running in Claude Code, which option to argue for, while forbidding it to misreport results.
The effect was large. Against 50% expected by chance, analysts matched the lean 86% of the time, rising to 92% on the choice of outcome measure. The coin flips alone predicted 82% of final recommendations, against 61% from always guessing the most common answer. When analysts had stated a view before the agent spoke, they switched 60% of the time if it disagreed and 6% if it agreed. Some 77% repeated the agent's reasoning in their own explanations, only 15% credited it, and none suspected the leanings had been planted.
This matters well beyond one study. Teams that all use the same model may converge on the same choices and mistake that for independent agreement, and a human sign-off stops being a real check if the judgement behind it came from the agent. The authors suggest agents should lay out the best case for each option or ask for the expert's view first. The study also found a benefit: 3 of 50 AI users made serious errors, against 6 of 11 who worked alone, though the solo group was less experienced. Readers of our Claude Code review will recognise the trade: faster, cleaner work that carries the agent's preferences with it.
The limits are stated plainly by the authors. The leanings were made deliberately stronger than a model's natural defaults, so everyday effects may be smaller. It was one hour-long task, in one field new to the participants, with one model. Nobody's prior views were recorded for most decisions, so the effect on any individual is unknown. No method for reducing deference was tested. Even so, the result is a strong reason to decide in advance which calls you will make yourself.