We all have specialized brain circuits for recognizing faces.

Each person has their own historical judgment of the emotion in a human face, which may or may not be the emotion objectively displayed by the face.

The amygdala, not the hippocampus, was found to be where we integrate our perception of human facial emotion.

The facial information conveyed by the eyes, not the mouth, was primarily how the amygdala perceived emotion.

---

This 2014 study was performed on seven neurology patients who had deep-brain electrodes implanted for other purposes of diagnosis or treatment, including epilepsy and autism, and six healthy control subjects. With the electrodes, the researchers were able to measure individual neurons instead of functional MRI aggregate results.

This increased measuring capability enabled the researchers to develop other findings, such as:

“Neuronal selectivity for fear faces in the amygdala comes mainly from a suppression of activity in happy-face trials, whereas selectivity for happy faces is mainly due to an increase in activity for happy-face trials.”

Also:

“The long latency of the amygdala responses we observed already argues for considerable synthesis, consistent with the integration of face input from temporal cortex with signals from other brain regions, as well as substantial processing internal to the amygdala.”

http://www.pnas.org/content/111/30/E3110.full “Neurons in the human amygdala selective for perceived emotion”