What if an AI could interpret your imagination, turning images in your mind’s eye into reality? While that sounds like a detail in a cyberpunk novel, researchers have now accomplished exactly this, according to a recently-published paper.
Researchers found that they could reconstruct high-resolution and highly accurate images from brain activity by using the popular Stable Diffusion image generation model, as outlined in a paper published in December. The authors wrote that unlike previous studies, they didn’t need to train or fine-tune the AI models to create these images.
The researchers—from the Graduate School of Frontier Biosciences at Osaka University—said that they first predicted a latent representation, which is a model of the image’s data, from fMRI signals. Then, the model was processed and noise was added to it through the diffusion process. Finally, the researchers decoded text representations from fMRI signals within the higher visual cortex and used them as input to produce a final constructed image.
