Spatiotemporal dynamics in human visual cortex rapidly encode the emotional content of faces

The temporal and spatial neural processing of faces has been investigated rigorously, but few studies have unified these dimensions to reveal the spatio-temporal dynamics postulated by the models of face processing. We used support vector machine decoding and representational similarity analysis to combine information from different locations (fMRI), time windows (EEG), and theoretical models. By correlating information matrices derived from pairwise classifications of neural responses to different facial expressions (neutral, happy, fearful, angry), we found early EEG time windows (starting around 130 ms) to match fMRI data from early visual cortex (EVC), and later time windows (starting around 190ms) to match data from occipital and fusiform face areas (OFA/FFA) and posterior superior temporal sulcus (pSTS).According to model comparisons, the EEG classifications were based more on low-level visual features than expression intensities or categories. In fMRI, the model comparisons revealed change along the processing hierarchy, from low-level visual feature coding in EVC to coding of intensity of expressions in the right pSTS. The results highlight the importance of a multimodal approach for understanding the functional roles of different brain regions in face processing.

The ability to detect and understand other people’s social interactions is a fundamental part of the human visual experience that develops early in infancy and is shared with other primates. However, the neural computations underlying this ability remain largely unknown. Is the detection of social interactions a rapid perceptual process, or a slower post-perceptual inference? Here we used magnetoencephalography (MEG) decoding and computational modeling to ask whether social interactions can be detected via fast, feedforward processing. Subjects in the MEG viewed snapshots of visually matched real-world scenes containing a pair of people who were either engaged in a social interaction or acting independently. The presence versus absence of a social interaction could be read out from subjects’ MEG data spontaneously, even while subjects performed an orthogonal task. This readout generalized across different scenes, revealing abstract representations of social interactions in the human brain. These representations, however, did not come online until quite late, at 300 ms after image onset, well after the time period of feedforward visual processes. In a second experiment, we found that social interaction readout occurred at this same latency even when subjects performed an explicit task detecting social interactions. Consistent with these latency results, a standard feedforward deep neural network did not contain an abstract representation of social interactions at any model layer. We further showed that MEG responses distinguished between different types of social interactions (mutual gaze vs joint attention) even later, around 500 ms after image onset. Taken together, these results suggest that the human brain spontaneously extracts the presence and type of others’ social interactions, but does so slowly, likely relying on iterative top-down computations.

19Choosing between equally valued options can be a conundrum, for which classical decision 20 theories predicted a prolonged response time (RT). Paradoxically, a rational decision-maker 21 would need no deliberative thinking in this scenario, as outcomes of alternatives are indifferent. 22How individuals choose between equal options remain unclear. Here, we characterized the 23 neurocognitive processes underlying such voluntary decisions, by integrating advanced cognitive 24 modelling and EEG recording in a probabilistic reward task, in which human participants chose 25 between pairs of cues associated with identical reward probabilities at different levels. We 26 showed that higher reward certainty accelerated RT. At each certainty level, participants 27 preferred to choose one cue faster and more frequently over the other. The behavioral effects on 28 RT persisted in simple reactions to reward cues. By using hierarchical Bayesian parameter 29 estimation for an accumulator model, we showed that the certainty and preference effects were 30 independently associated with the rate of evidence accumulation during decisions, but not with 31 visual encoding or motor execution latencies. Time-resolved multivariate pattern classification of 32 EEG evoked response identified significant representations of reward certainty and preference 33 choices as early as 120 ms after stimulus onset, with spatial relevance patterns maximal in 34 middle central and parietal electrodes. Furthermore, EEG-informed computational modelling 35 showed that the rate of change between N100 and P300 event-related potentials reflected 36 changes in the model-derived rate of evidence accumulation on a trial-by-trial basis. Our findings 37 suggested that reward certainty and preference collectively shaped voluntary decisions between 38 equal options, providing a mechanism to prevent indecision or random behavior. 39Keywords: decision making, reward certainty, preference bias, EEG, cognitive modelling 40 suggesting a possible bias between equal options, rendering some options more preferred than 63 others. 64Previous research on equal choices raises three further unresolved issues. First, it is unclear how 65 reward certainty and preference bias impact on different sub-components of the decision-making 66 process. Second, functional imaging studies have localized the mesocorticolimbic dopaminergic 67 network to be involved in both reward certainty and preference processing (Tobler et al., 2007; 68 Abler et al., 2009). Less is known about how macroscopic brain activities relating to these effects 69 unfold in time. Third, conventional equal-choice paradigms commonly use subjective ratings to 70 quantify and equate values of options. This design has been shown to be vulnerable to value 71 fluctuations (Chen and Risen, 2010; Izuma and Murayama, 2013) that originate from voluntary 72 choices themselves (Festinger, 1957;Bem, 1967). 73Here, we addressed these questions by combining advanced computational modelling and EEG 74 in a probabilistic reward t...