Characterizing the spatiotemporal profile and the level of abstractness of action representations: neural decoding of magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) data

When we observe other people's actions, a network of temporal, parietal and frontal regions is recruited, known as action observation network (AON). This network includes areas that have been reported to be involved when we perform actions ourselves. Such findings support the view that action understanding occurs by simulating actions in our own motor system (motor theories of action understanding). Alternatively, it has been argued that actions are understood based on a perceptual analysis, with access to action knowledge stored in the conceptual system (cognitive theories of action understanding). It has been argued earlier that areas that play a crucial role for action understanding should be able to (a) distinguish between different actions, and (b) generalize across the ways in which the action is performed (e.g. Dinstein, Thomas, Behrmann, & Heeger, 2008; Oosterhof, Tipper, & Downing, 2013; Caramazza, Anzelotti, Strnad, & Lingnau, 2014). Here we argue that one additional criterion needs to be met: an area that plays a crucial role for action understanding should have access to such abstract action information early, around the time when the action is recognized. An area that has access to abstract action information after the action has been recognized is unlikely to contribute to the process of action understanding. In this thesis, I report three neuroimaging studies in which we used magnetoencephalography (MEG) and functional magnetic resonance imaging (fMRI) to characterize the temporal dynamics of abstract representations of observed actions (Study 1 and 2), meaning that generalize across lower level dimensions, and to characterize the type of information encoded in the regions of the AON (Study 3). Specifically, in Study 1 we examined where in the brain and at which point in time it is possible to distinguish between pointing and grasping actions irrespective of the way in which they are performed (reach direction, effector) using MEG in combination with multivariate pattern analysis (MVPA) and source analysis. We show that regions in the left lateral occipitotemporal cortex (LOTC) have the earliest access to abstract action representations. By contrast, precentral regions, though recruited relatively early, have access to abstract action representations substantially later than left LOTC. In Study 2, we tested the temporal dynamics of the neural decoding related to the oscillatory activity induced by observation of actions performed with different effectors (hand, foot). We observed that temporal regions are able to discriminate all the presented actions before effector-related decoding within effector-specific motor regions. Finally, in Study 3 we investigated what aspect of an action is encoded within the regions of the AON. Object-directed actions induce a change of states, e.g. opening a bottle means changing its state from closed to open. It is still unclear how and in which brain regions these neural representations are encoded. Using fMRI-based multivoxel pattern decoding, we aimed at dissociating the neural representations of states and action functions. Participants observed stills of objects (e.g., window blinds) that were in either open or closed states, and videos of actions involving the same objects, i.e., open or close window. Action videos could show the object manipulation only (invisible change), or the complete action scene (visible change). This design allowed us to detect neural representations of action scenes, states and action functions independently of each other. We found different sub-regions within LOTC containing information related to object states, action functions, or both. These findings provide important information regarding the organization of action semantics in the brain and the role of LOTC in action understanding.