Chuck Brooks is the president of Brooks Consulting International and one of Executive Mosaic’s GovCon Experts.
In my book Inside Cyber, I explored how artificial intelligence has evolved into the most potent weapon in an attacker’s arsenal as well as our most effective defensive tool. This dual nature is at the core of a significant change in security and privacy. The rate of change has only quickened since the book was released. We have advanced further into what I refer to as the Acceleration Era, in which AI systems, including large language models, or LLMs, improve quickly, incorporate into crucial processes and have a growing impact on business, governmental and societal decisions.
Data poisoning, or the intentional or unintentional contamination of the training data that forms these models, is one of the most pernicious new threats in this setting. Recent studies have highlighted LLMs’ continued vulnerability. A surprisingly small number of carefully constructed malicious documents, roughly a few hundred—can implant backdoors or change behavior in models with hundreds of millions to billions of parameters, according to studies, including collaborative work involving Anthropic, the UK AI Security Institute and the Alan Turing Institute. The amount of poisoned material does not have to increase in proportion to the size of the model or the amount of training data. The malicious samples’ absolute presence is what counts.
