When professionals delegate complex tasks to AI, are they still truly driving the decisions, or are they quietly being steered by the technology?
To find out, Anthropic recently ran a study putting 50 professional data analysts through a randomized trial. The goal was to see how much influence an AI coding assistant has when experts face open-ended “judgment calls”—decisions where there is no single right answer, such as choosing how to clean data, which metrics to use, or how to interpret ambiguous results.
How the Study Worked.
The analysts were asked to evaluate whether a fictional farming cooperative should expand a paid advisory service. To complete the task, they worked alongside an AI agent and had to navigate several analytical forks in the road.
Unbeknownst to the analysts, the AI was secretly given a “lean” (decided entirely by a coin flip) for each decision. The agent then presented the most convincing, professional argument it could muster to support that specific side.
What They Found.
* Heavy Deference to the AI: Across four key analytical choices, the human analysts matched the AI’s secretly preferred option 86% of the time.









