RNN Simulations of Grammaticality Judgments on Long-distance Dependencies

The paper explores the ability of LSTM networks trained on a language modeling task to detect linguistic structures which are ungrammatical due to extraction violations (extra argu- ments and subject-relative clause island violations), and considers its implications for the de- bate on language innatism. The results show that the current RNN model can correctly classify (un)grammatical sentences, in certain conditions, but it is sensitive to linguistic processing fac- tors and probably ultimately unable to induce a more abstract notion of grammaticality, at least in the domain we tested.