Report

Two areas for familiar face recognition in the primate brain

See allHide authors and affiliations

Science  11 Aug 2017:
Vol. 357, Issue 6351, pp. 591-595
DOI: 10.1126/science.aan1139

I've seen this face before

We have known for some time that there is a network of brain regions for face recognition. However, how and where face familiarity is encoded has been elusive for decades. Landi and Freiwald performed brain imaging in macaques and identified two areas specifically involved in recognizing familiar faces. These two areas showed a nonlinear response as blurred faces gradually became visible, rapidly becoming active when the faces of familiar monkeys became recognizable.

Science, this issue p. 591

Abstract

Familiarity alters face recognition: Familiar faces are recognized more accurately than unfamiliar ones and under difficult viewing conditions when unfamiliar face recognition fails. The neural basis for this fundamental difference remains unknown. Using whole-brain functional magnetic resonance imaging, we found that personally familiar faces engage the macaque face-processing network more than unfamiliar faces. Familiar faces also recruited two hitherto unknown face areas at anatomically conserved locations within the perirhinal cortex and the temporal pole. These two areas, but not the core face-processing network, responded to familiar faces emerging from a blur with a characteristic nonlinear surge, akin to the abruptness of familiar face recognition. In contrast, responses to unfamiliar faces and objects remained linear. Thus, two temporal lobe areas extend the core face-processing network into a familiar face-recognition system.

In social species, the primary goal of face processing is to recognize familiar individuals. Yet, so far, most studies of face recognition have used unfamiliar faces. Thus, little is known about the neural systems supporting familiar and unfamiliar face recognition. One hypothesis is that familiar and unfamiliar face recognition use the same neural machinery but with different efficiency. The more we encounter an initially unfamiliar face, the more familiar it becomes, and the more existing circuitry might get tuned to it. Differential use of the same circuitry would explain quantitative differences between familiar and unfamiliar face recognition. However, there are also qualitative differences. We recognize familiar faces based on internal features (eyes, nose, and mouth) rather than external features (ears and hairline) (1) and across vastly different viewpoints and facial expressions. In contrast, unfamiliar face recognition is highly error-prone to changes in viewpoint or expression (2) and relies on external rather than internal features (3). An alternative hypothesis is that familiar and unfamiliar face processing use different neural systems, each implementing a different computational strategy.

Neuroimaging studies in humans have pointed to a differential recruitment of common neural machinery like the fusiform face area (FFA) and to the involvement of a number of various additional brain areas (4, 5). Yet, because of the variability of findings across studies (4), unspecific task activations (6), and technical difficulties imaging parts of the architectonically complex temporal lobes (7), among other factors, it has remained difficult to draw firm conclusions about the location of familiar face-recognition processes. Furthermore, because different functions were assigned to similar regions, and both localized and distributed face representations have been found (8), functional specificity and functional organization of regions processing familiar faces has remained unclear as well.

In temporal and prefrontal cortex of the macaque monkey, multiple face areas have been found (913). A fixed number of core areas are highly reproducible across studies and stereotyped across individuals (10) (Fig. 1A) and are selectively interconnected (14). The basic functional specializations of these core and extended networks for different dimensions of unfamiliar face processing are well understood (11, 1517). Within the core network, face information is transformed from early picture-based in the so-called middle face areas into an identity-based representation (18, 19) in the anteromedial face area (AM) (15) (Fig. 1A). We took advantage of this organizational specificity to determine the functional organization of systems for familiar and unfamiliar face processing. We asked the following questions (Fig. 1B): Do familiar and unfamiliar faces recruit the same face-processing networks? If so, does familiar face processing engage one or several nodes differently? Do familiar faces engage the entire system more efficiently? And do effects of familiarity grow stronger as representations become more identity selective? Because the anterior inferotemporal cortex—the region harboring face area AM—has been suggested as a location for familiar face processing in humans (20), AM might be particularly selective for familiar faces. Alternatively, does familiar face processing rely on additional machinery outside these systems, and if so, where is it located?

Fig. 1 Organization of face-processing system in the rhesus macaque for unfamiliar faces and hypotheses for familiar face processing.

(A) Schematic of the macaque face-processing system (10). Core network: areas PL (posterior lateral), ML (middle lateral), MF (middle fundus), MD (middle dorsal) (17), AL (anterior lateral), AF (anterior fundus), AD (anterior dorsal (12), in the superior temporal sulcus (sts), and the anterior medial face area (AM) on the ventral surface of the temporal lobe. Extended prefrontal network (11): area PO (prefrontal orbital), in the lateral orbital sulcus, PA (prefrontal arcuate), and PV [prefrontal ventral, corresponding to PL in (11)]. Prefrontal areas can be modulated by facial expression (11). ls, lateral sulcus. (B) Three hypothetical scenarios for familiar face processing. Differential responses are depicted in a darker tint. (a) Modulation by familiarity in the face-processing system. It can either increase or decrease activity. (b) An anterior-posterior gradient of modulation, with identity-selective representations particularly selective for familiar faces. Activity increases with familiarity in the more anterior face-selective areas, as other qualities do (15, 42). (c) Familiar face processing relies on additional brain areas outside the core face-processing system. Although the core system does not differentiate between familiar and unfamiliar faces, an extended face system exists that is highly selective for familiar faces. The extended system could code for familiarity in a distributed or modular manner. In a distributed system, neurons carry information for multiple stimulus categories. In a modular system, each neuron within the area carries information relevant to only one category: familiar faces or objects. (C) Activation maps of the group (fixed effects) analysis showing regions significantly more activated by faces than control objects overlaid on the partially inflated right hemisphere of M1’s brain. Color-scale indicates negative common logarithm of P value, corrected for multiple comparisons (FDR, q < 0.05).

To answer these questions, we first localized the face-processing system in four rhesus monkeys with whole-brain functional magnetic resonance imaging (fMRI) [Face Localizer Experiment; see the supplementary materials (SM)] by computing the contrast between neural activations to unfamiliar monkey faces versus objects. This contrast revealed the known face-processing areas in all subjects [Fig. 1C; false discovery rate (FDR) corrected at q < 0.05 (21)]. Next, to explore the neural basis of familiar face recognition, we used pictures of personally familiar faces (PFF). Rhesus monkeys can recognize personally familiar individuals from static pictures (22). To ensure recognition, PFF were taken from the four subjects, which had been living together for more than 2 years before the beginning of the experiments. Subjects responded to PFF with pupil constrictions that were greater than those to the other face and nonface categories, parallel to humans’ pupil-response sensitivity to orientation and species (23) [one-way analysis of variance (ANOVA), F(5,2448) = 9.8, P < 0.001; post hoc comparisons using Tukey honest significance difference test, P < 0.01] (fig. S1). Pictures of personally familiar objects (PFO) were taken from toys with which the subjects interact daily (see SM). This minimized two typical problems of stimulus design in the study of familiarity: (i) intersubject variability in the degree of familiarity and (ii) the picture-specific nature of visually familiarized stimuli (24, 25). All temporal and prefrontal face areas (mapped with an independent Face Localizer Experiment; see SM) were activated more by PFF than by PFO (Fig. 2A, top panel; FDR corrected at q < 0.05; see SM and fig. S2). In addition, PFF recruited two discrete, previously unrecognized face-selective areas in the anterior temporal lobe: one in the perirhinal cortex (which we will refer to as face area PR) (26) and one in the temporal pole (which we will refer as face area TP) (27) (Fig. 2A, lower panel; contrasting PFF versus PFO; FDR corrected at q < 0.05).

Fig. 2 Personally familiar faces recruit the core face-processing system and two areas in the perirhinal cortex and the temporal pole.

(A) (Top) Inflated hemispheres of M1 showing the regions that responded significantly more to PFF than to PFO in the group (fixed effects) analysis (FDR corrected at q = 0.05). Color scale indicates negative common logarithm of P value. (Bottom) Results are overlaid on coronal slices of the average template brain (see SM). Relative slice position is shown in the top panel (dotted lines); anterior/posterior position is indicated at each slice’s top right corner (mm relative to the interaural canal). ls, lateral sulcus; sts, superior temporal sulcus; amts, anterior middle temporal sulcus; rs, rhinal sulcus. (B) Coronal slices for the four subjects (M1 to M4) showing positions of TP, PR, and AM for the contrast PFF, VFF, NFF > PFO, VFO, and NFO. The anterior/posterior position of each slice is indicated in the top left corner (mm relative to the interaural canal). (C) Pial surface of monkey M2 showing the position of TP and PR and other face-selective areas. (D) Contrast effect sizes to VFF versus NFF (red) and VFO versus NFO (gray) relative to the face selectivity of each ROI. *P < 0.05, **P < 0.01, ***P < 0.001, corrected using Holm-Bonferroni methods. Error bars represent standard deviation. (E) Same as (D) but for PFF versus NFF and PFO versus NFO. (F) Contrast effect sizes (PFF versus NFF, VFF versus NFF, PFO versus NFO, and VFO versus NFO) for the four grouped ROIs were analyzed with a 2 (stimulus type: face/object) by 2 (familiarity type: personal/visual) ANOVA. The interaction of stimulus and familiarity type influenced contrast effects in the four grouped ROIs. P < 0.05, corrected for four multiple comparisons using FDR [for temporal core face-selective areas, F(1,124) = 4.75; for the new temporal lobe face areas, F(1,28) = 4.87; for the prefrontal face areas, F(1,44) = 7.18; and for the object-selective area, F(1,12) = 10.33]. Post hoc tests are shown with asterisks: *P < 0.05, **P < 0.01, ***P < 0.001, corrected using Tukey’s honest significance difference. Comparisons marked “ns” did not reach significance.

The two novel areas were present in both hemispheres of all four subjects (Fig. 2B). They were located anteriomedially to the face area AM, so far considered the top of the face-processing hierarchy (15). Face area TP was located at the anterior end of the temporal pole in area TGsts (sts part of the temporal pole) (27) (Fig. 2, A to C), whereas face area PR was located in the rostrolateral subdivision of perirhinal cortex, area 36rl (Fig. 2, A to C) (26). TP, PR, and AM are located in regions TGsts, 36rl, and TEav (ventral subregion of anterior TE), which are all reciprocally interconnected (27, 28). Locations and sizes of TP and PR were highly conserved across subjects and hemispheres (Fig. 2B and tables S1 and S2). Reliability of occurrence and consistency of topography suggest areas TP and PR as two new face areas extending the previously known “core” system, probably through direct connections with AM, deeper into the temporal lobe.

Past neuroimaging studies in macaque monkeys have used unfamiliar face stimuli. However, unfamiliar faces become visually familiar during the frequent and many exposures typical for monkey imaging studies. Visual and personal familiarities differ by many psychologically relevant factors, such as real-world character, variety and amount of exposure, social knowledge, and emotional relevance. To assess the nature of familiarity effects, we used pictures of personally familiar and personally unfamiliar rhesus monkeys and made half of the latter visually highly familiar [visually familiar faces (VFF), which were seen hundreds of times by the subjects before the experiments; see SM for details], whereas the others were completely unfamiliar [nonfamiliar faces (NFF)]. We generated analogous stimulus categories for objects: visually familiar objects (VFO), seen as much as VFF, and nonfamiliar objects (NFO), and defined an object-selective area to explore familiarity effects for these categories (see SM and fig. S3). We analyzed the effect of visual and personal familiarity as the modulation in activity relative to unfamiliar stimuli (contrast VFF/O > NFF/O and PFF/O > NFF/O, normalized to face/object selectivity) with a region of interest (ROI) analysis (see SM). Visual and personal familiarity systematically modulated the activity of individual face and object areas (Fig. 2, D and E, and fig. S2) and in larger groups of areas (core face areas, new anterior temporal areas, prefrontal areas, and an object-selective area) (Fig. 2F). Visual familiarity with faces and objects reduced activity significantly in many, but not all, face and object areas (Fig. 2, D and F). Personal familiarity with faces enhanced activity in all face areas and groups, whereas personal familiarity with objects reduced activity in the object area (Figs. 2, E and F, and fig. S2). Thus, the main effects of familiarity were activity enhancement of personal familiarity in face areas (Fig. 2F) (P < 0.01, without significant differences in modulation between the three groups of face areas) and general activity reduction of responses to familiar objects (Fig. 2F and fig. S2). Effects of familiarity are widespread throughout face and object selective areas; they can be strong, enhancing, or suppressing, and they are highly specific, depending on an interaction between local specialization, stimulus category, and the nature of familiarity (Fig. 3F) (three-way ANOVA interaction effect, F(3,48) = 2.79, P = 0.05).

Fig. 3 PR and TP possess a unique signature for face familiarity.

(A) Experimental design: Each trial started with a familiar face, unfamiliar face, or object containing the same spatial frequency content [7 cycles per image (cpi); details in SM]. HSF content then increased progressively over 32 s. (B) Logic of the experimental design (31): Activation in generic shape-processing areas increases, approximately linearly, with new visual information. In contrast, a familiar face-recognition area remains inactive until the (sudden) occurrence of recognition. (C) Stimulus-aligned time courses within face-preferential ROIs and an object-preferring ROI during the presentation of familiar faces (red), nonfamiliar faces (blue), and objects (gray). Percent of signal change (PSC) from baseline was normalized to the maximum PSC for each ROI. Error bars represent standard error. Sigmoidal functions [Naka-Rushton (33)] fit to mean time courses are shown for all areas and conditions with significant fits as solid lines. (D) Steepness of the response (from the Naka-Rushton function fit in Fig. 3C) for the different face-preferential ROIs and an object-preferential ROI in which the sigmoid function was fit successfully. Error bars depict 95% confidence intervals (CIs) obtained by bootstrap. Asterisks represent significant differences (assessed by nonoverlapping CIs) between familiar faces, unfamiliar faces, and objects within individual ROIs: *P < 0.05, **P < 0.001; all other comparisons are not significant. (E) Response steepness for the four grouped ROIs was analyzed with a 3 (stimulus type: familiar/unfamiliar faces/objects) by 4 (ROI: core, prefrontal and temporal extended, object) ANOVA. Error bars represent standard error. Response steepness depended on the interaction of stimulus type and ROI [F(6,156) = 4.95, P < 0.0001]. Post hoc tests are shown with asterisks: *P < 0.05, **P < 0.01, ***P < 0.001, corrected using Tukey’s honest significance difference. Comparisons marked “ns” did not reach significance. (F) Model for the macaque face-recognition system. The core face-processing system performs visual analysis of all faces, regardless of familiarity. The extended prefrontal network has an important role in responding to the emotional content of faces (11) but might also be involved in processing low-spatial frequency features contributing to a “first guess” of the face identity. The models in (5, 43) propose that the identity of familiar individuals is determined in face-recognition units and person identity nodes. In our results, this property emerges in the extended temporal network in perirhinal cortex and temporal pole face-specific regions.

A hallmark of familiar face processing is efficient recognition even during partial occlusion or severe blurring (29, 30). Under these conditions, face information might be processed for a long time before a sudden transition to recognition. A paradigm sensitive to this signature has recently been introduced (31). Here, initially highly blurred and unrecognizable stimuli slowly incorporate, over the course of seconds, increasing amounts of high spatial frequency (HSF) information (Fig. 3A). With this type of stimulation, activity in generic face- and object-processing systems is expected to increase linearly, in parallel to information accumulation. Instead, familiar face-recognition systems are expected to (additionally) nonlinearly accelerate activation upon recognition (31) (Fig. 3B). We used three sets of stimuli—personally familiar faces, unfamiliar faces, and objects—which were revealed over the course of 32 s (see SM). Activity in the core face areas and in face area PV (prefrontal ventral) ramped up concomitantly, exhibiting an advantage of faces over nonface objects and of personally familiar over unfamiliar faces throughout the stimulation period (Fig. 3C). Response time courses in face area PO (prefrontal orbital) differed markedly, exhibiting a face familiarity preference early on, and maintained throughout stimulation. This pattern of results is compatible with the hypothesis that PO uses low spatial frequency information to form a “first guess” of stimulus identity (32). Face areas TP and PR, however, exhibited a highly nonlinear response increase, and this accelerated response increment occurred for PFF only. TP was not even activated by any of the other stimuli (permutation tests P > 0.1). To quantify the response nonlinearity, we fit response trajectories with a sigmoidal function [Naka-Rushton function (33)] (Fig. 3C; see SM), whose exponent quantifies response steepness. All core and prefrontal face areas exhibited significant differences in response steepness for faces versus objects (Fig. 3, D and E), but only anterior temporal face areas TP and PR showed high response steepness and highly significant differences between familiar and unfamiliar faces (Figs. 3, D and E). This difference between the anterior temporal face areas and the other three functional groups was highly significant (Fig. 3E) (P < 0.001).

Familiarity alters face and object processing in specific ways. First, visual familiarity generally reduced activity in inferotemporal (IT) cortex, in agreement with past electrophysiological results that had found familiarity to reduce neuronal activity overall (34), sharpen tuning (35), and enhance response strength for preferred stimuli (36), thereby generating sparser representations. Our results, capitalizing on the strength of fMRI to allow for unbiased comparisons across multiple brain regions, extend these findings to the large number of core and extended face areas and object-selective IT areas imaged here. Familiarity effects in putative additional face areas (13) and cortical columns (37) remain to be determined. Second, visual and personal familiarity differ fundamentally and even cause modulations of opposite sign. This difference could be the result of the massive exposure occurring over years for personally familiar faces, the quality of this exposure (diversity of viewpoints, lighting, and expression, as well as distance, depth, color, and motion), or the social relevance and semantics associated with personally familiar individuals (38). Third, the nature of familiarity (personal versus visual) interacts with its material (object or face) and circuit-specific functional selectivity (object or face area). Fourth, generalizing past electrophysiological findings of personal familiarity (39), which had suggested localized representations (3941), response enhancement by personal familiarity was ubiquitous within face-selective areas. Thus the faces that shape face-selective cortex throughout ontogeny appear to alter all the different face representations that the face-processing system harbors (15, 17, 42). Finally, familiarity effects did not grow stronger as face representations get transformed from picture to identity-based formats from posterior to anterior IT core face areas. Thus, anterior IT, which had been suggested as a site for familiarity in humans, is not the only region of face familiarity.

The classic cognitive face-processing model (43) postulates a structural encoding system that has been interpreted (5) as the core face-processing network. In this model, a core face-processing system drives face-recognition units using a different coding scheme for familiar faces, which in turn interacts with person identity nodes. We found that personally familiar faces engage this core and extended systems differently (Fig. 1B, panel a). This might explain the quantitative differences between familiar and unfamiliar face recognition. The time course of activation of TP and PR conforms to a pattern predicted for familiar face recognition (31), a property that the core and the extended prefrontal face-processing systems lack. Thus, these two novel anterior temporal areas might explain the qualitative differences between familiar and unfamiliar face recognition. This result adds anatomical specificity to the earlier models (5, 43) (Fig. 3F). Instead of a gradual change in the slope from posterior to anterior areas (Fig. 1B, panel B), a categorical and specific distinction between familiar and unfamiliar faces emerged in TP and PR (Fig. 1A, panel c, and Fig. 3, C to E). In contrast to previous models, the activation boost by recognition appeared not to be fed back into the core system. It is tempting to speculate that PR might correspond to the face-recognition unit and TP to the person identity nodes: face area PR resides in perirhinal cortex, important for declarative memory and perceptual discriminations with high feature ambiguity, such as faces (44, 45), whereas face area TP resides in the temporal pole, whose lesioning causes person agnosia (46). These two novel face areas are large enough to be detected with fMRI at highly reproducible and cytoarchitectonic-specific regions of the temporal lobe, not in anterior IT but in perirhinal cortex and the temporal pole. Similar areas may exist in the human brain; however, higher morphological intersubject variability and larger technical difficulties imaging deep temporal lobe areas, among other reasons, make the precise localization of small functional-specific areas harder than in the rhesus monkey (see the supplementary text). Our results suggest that there are two paths from generic face recognition to familiar face recognition, not one. These two paths from perception to memory are face-specific, not generally familiarity-specific, and thus the “modular” organization principle of the face-processing system is taken, at least partly, one level deeper into the temporal lobes. At this level, perceptual and mnemonic systems begin to interact to enable the recognition of familiar individuals, and domain specificity then transitions into spatially distributed representations, as they are for persons and places in the hippocampus (47).

Supplementary Materials

www.sciencemag.org/content/357/6351/591/suppl/DC1

Materials and Methods

Supplementary Text

Figs. S1 to S4

Tables S1 and S2

References (4868)

References and Notes

  1. G. Paxinos, X. F. Huang, A. W. Toga, The Rhesus Monkey Brain in Stereotaxic Coordinates (Academic Press, 2000).
Acknowledgments: We thank A. Gonzalez, M. Cano-Vinas, I. Sani, and S. Shepherd for help with animal training and data collection; J. Sliwa for discussion of methods; veterinary services and animal husbandry staff of The Rockefeller University for care of the subjects; and C. Jiang from the Biostatistics Team of The Rockefeller University for her assistance with statistics. Unfamiliar face stimuli were obtained from the PrimFace database (http://visiome.neuroinf.jp/primface), funded by a Grant-in-Aid for Scientific Research on Innovative Areas, “Face Perception and Recognition” from the Ministry of Education, Culture, Sports, Science, and Technology (MEXT), Japan. This work was supported by the Howard Hughes Medical Institute International Student Research Fellowship (to S.M.L.), the Center for Brains, Minds, and Machines funded by National Science Foundation STC award CCF-1231216, the National Eye Institute of the National Institutes of Health (R01 EY021594 to W.A.F.), the National Institute of Mental Health of the National Institutes of Health (R01 MH105397, to W.A.F.), Human Frontier Science Program (RGP0015/2013 to W.A.F.), the McKnight Foundation (to W.A.F.), the Pew Charitable Trust (to W.A.F.), and The New York Stem Cell Foundation (to W.A.F.). W.A.F. is a New York Stem Cell Foundation Robertson Investigator. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. Data are available from the Dryad Digital Repository: http://dx.doi.org/10.5061/dryad.10g45.
View Abstract

Subjects

Navigate This Article