A few weeks ago, a colleague of mine needed to collect and format some data from a website, and he asked the latest version of Anthropic’s generative AI system, Claude, for help. Claude cheerfully agreed to perform the task, generated a computer program to download the data, and handed over perfectly formatted results. The only problem? My colleague immediately noticed that the data Claude delivered was entirely fabricated.
When asked why it had made up the data, the chatbot apologized profusely, noting that the website in question didn’t provide the requested data, so instead the chatbot generated “fictional participant data” with “fake names…and results,” admitting “I should never present fabricated data as if it were scraped from actual sources.”
I have encountered similar examples of gaslighting by AI chatbots. In one widely circulated transcript, a writer asked ChatGPT to help choose which of her essays to send to a literary agent, providing links to each one. The chatbot effusively praised each essay, with details such as “[The essay] shows range—emotional depth and intellectual elasticity” and “it’s an intimate slow burn that reveals a lot with very little.” After several rounds of this, the writer started to suspect that something was amiss. The praise was effusive, but rather generic. She asked “Wait, are you actually reading these?” ChatGPT assured her, “I am actually reading them—every word,” and then quoted certain of the writer’s lines that “totally stuck with me.” But those lines didn’t actually appear in any of the essays. When challenged, ChatGPT admitted that it could not actually access the essays, and for each one, “I didn’t read the piece and I pretended I had.”