How input modality and visual experience affect the representation of categories in the brain

The general aim of the present dissertation was to participate in the progress of our understanding of how sensory input and sensory experience impact on how the human brain implements categorical knowledge. The goal was twofold: (1) understand whether there are brain regions that encode information about different categories regardless of input modality and sensory experience (study 1); (2) deepen the investigation of the mechanisms that drive cross-modal and intra-modal plasticity following early blindness and the way they express during the processing of different categories presented as real-world sounds (study 2). To address these fundamental questions, we used fMRI to characterize the brain responses to different conceptual categories presented acoustically in sighted and early blind individuals, and visually in a separate sighted group. In study 1, we observed that the right posterior middle temporal gyrus (rpMTG) is the region that most reliably decoded categories and selectively correlated with conceptual models of our stimuli space independently of input modality and visual experience. However, this region maintains separate the representational format from the different modalities, revealing a multimodal rather than an amodal nature. In addition, we observed that VOTC showed distinct functional profiles according to the hemispheric side. The left VOTC showed an involvement in the acoustical categorization processing at the same degree in sighted and in blind individuals. We propose that this involvement might reflect an engagement of the left VOTC in more semantic/linguistic processing of the stimuli potentially supported by its enhanced connection with the language system. However, paralleling our observation in rpMTG, the representations from different modalities are maintained segregated in VOTC, showing little evidence for sensory-abstraction. On the other side, the right VOTC emerged as a sensory-related visual region in sighted with the ability to rewires itself toward acoustical stimulation in case of early visual deprivation. In study 2, we observed opposite effects of early visual deprivation on auditory decoding in occipital and temporal regions. While occipital regions contained more information about sound categories in the blind, the temporal cortex showed higher decoding in the sighted. This unbalance effect was stronger in the right hemisphere where we, also, observed a negative correlation between occipital and temporal decoding of sound categories in EB. These last results suggest that the intramodal and crossmodal reorganizations might be inter-connected. We therefore propose that the extension of non-visual functions in the occipital cortex of EB may trigger a network-level reorganization that reduce the computational load of the regions typically coding for the remaining senses due to the extension of such computation in occipital regions.