We propose a novel approach to the study of how artificial neural network perceive the distinction between grammatical and ungrammatical sentences, a crucial task in the growing field of synthetic linguistics. The method is based on performance measures of language models trained on corpora and fine-tuned with either grammatical or ungrammatical sentences, then applied to (different types of) grammatical or ungrammatical sentences. The results show that both in the difficult and highly symmetrical task of detecting subject islands and in the more open CoLA dataset, grammatical sentences give rise to better scores than ungrammatical ones, possibly because they can be better integrated within the body of linguistic structural knowledge that the language model has accumulated.

An LSTM adaptation study of (Un)grammaticality / Chowdhury, Shammur Absar; Zamparelli, Roberto. - ELETTRONICO. - (2019), pp. 204-212. (Intervento presentato al convegno BlackboxNLP tenutosi a Florence nel 1st August 2019) [10.18653/v1/W19-4821].

An LSTM adaptation study of (Un)grammaticality

Chowdhury, Shammur Absar;Zamparelli, Roberto
2019-01-01

Abstract

We propose a novel approach to the study of how artificial neural network perceive the distinction between grammatical and ungrammatical sentences, a crucial task in the growing field of synthetic linguistics. The method is based on performance measures of language models trained on corpora and fine-tuned with either grammatical or ungrammatical sentences, then applied to (different types of) grammatical or ungrammatical sentences. The results show that both in the difficult and highly symmetrical task of detecting subject islands and in the more open CoLA dataset, grammatical sentences give rise to better scores than ungrammatical ones, possibly because they can be better integrated within the body of linguistic structural knowledge that the language model has accumulated.
2019
Proceedings of the 2019 ACL Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP
Florence
Association for Computational Linguistics
Chowdhury, Shammur Absar; Zamparelli, Roberto
An LSTM adaptation study of (Un)grammaticality / Chowdhury, Shammur Absar; Zamparelli, Roberto. - ELETTRONICO. - (2019), pp. 204-212. (Intervento presentato al convegno BlackboxNLP tenutosi a Florence nel 1st August 2019) [10.18653/v1/W19-4821].
File in questo prodotto:
File Dimensione Formato  
W19-4821.pdf

accesso aperto

Descrizione: File from ACL Anthology
Tipologia: Versione editoriale (Publisher’s layout)
Licenza: Creative commons
Dimensione 1.08 MB
Formato Adobe PDF
1.08 MB Adobe PDF Visualizza/Apri

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11572/248266
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
  • ???jsp.display-item.citation.isi??? ND
social impact