CALaMo: a Construsctionist perspective on the Analysis of linguistic behaviour of Language Models

In recent years, Neural Language Models (NLMs) have consistently demonstrated increasing linguistic abilities. However, the extent to which such networks can actually learn grammar remains an object of investigation, and experimental results are often inconclusive. Notably, the mainstream evaluation framework in which NLMs are tested seems largely based on Generative Grammar and nativist principles, and a shared constructionist approach on the matter has not yet emerged: this is at odds with the fact that usage-based theories are actually better suited to inspect the behaviour of such models. The main contribution of this thesis is the introduction of CALaMo, a novel framework for evaluating Neural Language Models’ linguistic abilities, using a constructionist approach. We especially aim at formalizing the relationship between the computational modelling phase and the underlying linguistic theory, thus allowing a more refined and informed discussion of settings and results. We focus on two specific areas that, we believe, are currently not easily tractable within the mainstream evaluation framework. The first scenario deals with language acquisition from child-directed data. Our main experimental result shows how it is possible to follow schematization paths during the acquisition process of the model, and how this relates to core hypotheses in constructionist theories. The second scenario deconstructs the mainstream view of the Neural Model as an average idealized speaker by proposing a way to simulate and analyze a population of artificial individuals. We show how the amount of “shared linguistic knowledge” across speakers is highly dependent on the specific linguistic background of each individual. Overall, we believe our framework opens the path for future discussion on the role of computational modelling in usage-based linguistic theory and vice versa, and provides a new formal methodology to both fields of study.