A real stinker.
The trial, conducted by Amazon Web Services, was commissioned by the government regulator as a proof of concept for generative AI’s capabilities, and in particular its potential to be used in business settings.
That potential, the trial found, is not looking promising.
In a series of blind assessments, the generative AI summaries of real government documents scored a dire 47 percent on aggregate based on the trial’s rubric, and were decisively outdone by the human-made summaries, which scored 81 percent.
Leave a reply