People differ in how they interpret social situations and infer others’ emotions. This fMRI study shows that the medial prefrontal cortex integrates this social variability to infer others’ emotional states, and indicates how aging affects this process.