Sep 12, 2022

How AI Transformers Mimic Parts of the Brain

Posted by in category: robotics/AI

Understanding how the brain organizes and accesses spatial information — where we are, what’s around the corner, how to get there — remains an exquisite challenge. The process involves recalling an entire network of memories and stored spatial data from tens of billions of neurons, each connected to thousands of others. Neuroscientists have identified key elements such as grid cells, neurons that map locations. But going deeper will prove tricky: It’s not as though researchers can remove and study slices of human gray matter to watch how location-based memories of images, sounds and smells flow through and connect to each other.

Artificial intelligence offers another way in. For years, neuroscientists have harnessed many types of neural networks — the engines that power most deep learning applications — to model the firing of neurons in the brain. In recent work, researchers have shown that the hippocampus, a structure of the brain critical to memory, is basically a special kind of neural net, known as a transformer, in disguise. Their new model tracks spatial information in a way that parallels the inner workings of the brain. They’ve seen remarkable success.

“The fact that we know these models of the brain are equivalent to the transformer means that our models perform much better and are easier to train,” said James Whittington, a cognitive neuroscientist who splits his time between Stanford University and the lab of Tim Behrens at the University of Oxford.

Leave a reply