Experimenting with Linguistic Tools for Conceptual Modelling: Quality of the models and critical features

This paper presents the results of three experiments designed to assess the extent to which a Natural-Language Processing (NLP) tool improves the quality of conceptual models, specifically object-oriented ones. Our main experimental hypothesis is that the quality of a domain class model is higher if its development is supported by a NLP system. The tool used for the experiment – named NL-OOPS – extracts classes and associations from a knowledge base realized by a deep semantic analysis of a sample text. In our experiments, we had groups working with and without the tool, and then compared and evaluated the final class models they produced. The results of the experiments give insights on the state of the art of NL-based Computer Aided Software Engineering (CASE) tools and allow identifying important guidelines to improve their performance, highlighting which of the linguistic tasks are more critical to effectively support conceptual modelling.