When professionals delegate complex tasks to AI, are they still truly driving the decisions, or are they quietly being steered by the technology?<\/p>\n
To find out, Anthropic recently ran a study putting 50 professional data analysts through a randomized trial. The goal was to see how much influence an AI coding assistant has when experts face open-ended \u201cjudgment calls\u201d\u2014decisions where there is no single right answer, such as choosing how to clean data, which metrics to use, or how to interpret ambiguous results.<\/p>\n
How the Study Worked.<\/p>\n
The analysts were asked to evaluate whether a fictional farming cooperative should expand a paid advisory service. To complete the task, they worked alongside an AI agent and had to navigate several analytical forks in the road.<\/p>\n
Unbeknownst to the analysts, the AI was secretly given a \u201clean\u201d (decided entirely by a coin flip) for each decision. The agent then presented the most convincing, professional argument it could muster to support that specific side.<\/p>\n
What They Found.<\/p>\n
* Heavy Deference to the AI: Across four key analytical choices, the human analysts matched the AI\u2019s secretly preferred option 86% of the time.<\/p>\n