Face perception depends on activation of a core face processing network including the fusiform face area, the occipital face area and the superior temporal sulcus (STS). The medial temporal lobe (MTL) is also involved in decoding facial expression and damage to the anterior MTL, including the amygdala, generally interferes with emotion recognition. The impairment in emotion recognition following anterior MTL injury can be a direct result from injured MTL circuitry, as well as an indirect result from decreased MTL modulation of areas in the core face network. To test whether the MTL modulates activity in the core face network, we used functional magnetic resonance imaging to investigate activation in the core face processing network in patients with right or left anterior temporal lobe resections (ATR) due to intractable epilepsy. We found reductions of face-related activation in the right STS after both right and left ATR together with impaired recognition of facial expressions. Reduced activity in the fusiform and the occipital face areas was also observed in patients after right ATR suggesting widespread effects on activity in the core face network in this group. The reduction in face-related STS activity after both right and left ATR suggests that MTL modulation of the STS may facilitate recognition of facial expression.