Where symbols meet meanings: The organization of gestures and words in the middle temporal gyrus

Every day we use actions, gestures and words to interact with other people and with the environment. Being able to understand people’s movements and communicative intentions is critical to our ability to act successfully in the world. Here we present three studies aiming at investigating the relationship between actions, gestures and words in the brain. In the first study we described and offer a standardized data set of 230 well-controlled stimuli of meaningful (pantomimes and emblems) and meaningless gestures together with norms, with the aim of promoting replicability between studies. One hundred and thirty raters (Italian and non-Italian speakers) rated the meaningfulness of the gestures, and provided a name and a description for each of them. To our knowledge, this is the first data set of meaningful and meaningless gestures presented in the literature. In the second study, we aimed 1) at characterizing the neural network associated with the processing of different categories of gestures (pantomimes, emblems and meaningless gestures) using fMRI, and 2) at contrasting the role of precentral and temporal areas in action understanding, using rTMS. In particular, we applied rTMS to the posterior middle temporal gyrus (pMTG) and to the ventral premotor cortex (PMv) in different sessions, while participants were performing either a semantic or a perceptual judgment task. According to motor theories of action understanding, rTMS applied to the PMv, but not to the pMTG, should impair performance during the semantic judgment task. By contrast, according to cognitive theories of action understanding, rTMS applied to pMTG, but not to PMv, should impair performance in this task. Results from the fMRI experiment revealed a sensitivity of the MTG to meaningful in comparison to meaningless gestures. Additionally, three different brain areas seemed to contribute to the processing of pantomimes and emblems: superior parietal lobe (SPL) and precentral gyrus (PCG) in the case of pantomimes and IFG in the case of emblems. Unfortunately, we did not observe any significant effect of rTMS in any condition. The third study aimed at investigating how pantomimes, emblems and words are organized in the middle temporal gyrus, using fMRI. We observed a posterior-to-anterior structure, both in the left and in the right hemisphere, that might reflect the input modality and also the arbitrariness of the relationship between form and meaning.