Improving the Quality of Conceptual Models with NLP Tools: An Experiment

Conceptual models are used in a variety of areas within Computer Science, including Software Engineering, Databases and AI. A major bottleneck in broadening their applicability is the time it takes to build a conceptual model for a new application. Not surprisingly, a variety of tools and techniques have been proposed for reusing conceptual models (e.g., ontologies), or for building them semi-automatically from natural language descriptions. What has been left largely unexplored is the impact of such tools on the quality of the models that are being created. This paper presents the results of an experiment designed to assess the extent to which a Natural Language Processing (NLP) tool mproves the quality of conceptual models, specifically object-oriented ones. Our main experimental hypothesis is that the quality of a domain class model is higher if its development is supported by a NLP system. The tool used for the experiment -- named NL-OOPS -- extracts classes and associations from a knowledge base realized by a deep semantic analysis of a sample text. Specifically, NL-OOPS produces class models at different levels of detail by exploiting class hierarchies in the knowledge base of a NLP system and marks ambiguities in the text. In our experiments, we had groups working with/without the tool, and then compared and evaluated the final class models they produced.