Our brain is constantly working to make sense of the world around us and finding patterns in it, even when we are asleep the brain is storing patterns. Making sense of the brain itself, however, has remained an intricate pursuit.

Christoff Koch, a well-known neuroscientist, famously called the human brain the “most complex object in our observable universe” [1]. Aristotle, on the other hand, thought it was the heart that gave rise to consciousness and that the brain functioned as a cooling system both practically and philosophically [2]. Theories of the brain have evolved since then, generally shaped by knowledge gathered over centuries. Historically, to analyze the brain, we had to either extract the brain from deceased people or perform invasive surgery. Progress over the past decades has led to inventions that allow us to study the brain without invasive surgeries. A few examples of imaging techniques that do not require surgery include macroscopic imaging techniques such as functional magnetic resonance imaging (fMRI) or approaches with a high temporal resolution such as electroencephalogy (EEG). Advances in treatments, such as closed-loop electrical stimulation systems, have enabled the treatment of disorders like epilepsy and more recently depression [3, 4]. Existing neuroimaging approaches can produce a considerable amount of data about a very complex organ that we still do not fully understand which has led to an interest in non-linear modeling approaches and algorithms equipped to learn meaningful features.

This article provides an informal introduction to unique aspects of neuroimaging data and how we can leverage these aspects with deep learning algorithms. Specifically, this overview will first explain some common neuroimaging modalities more in-depth and then discuss applications of deep learning in conjunction with some of the unique characteristics of neuroimaging data. These unique characteristics tie into a broader movement in deep learning, namely that data understanding should be a goal in itself to maximize the impact of applied deep learning.