Formative User-Centered Evaluation of Security Modeling: Results from a Case Study

Developing a security modeling language is a complex activity. Particularly, it becomes very challenging for Security Requirements Engineering (SRE) languages where social/organizational concepts are used to represent high-level business aspects, while security aspects are typically expressed in a technical jargon at a lower level of abstraction. In order to reduce this socio-technical mismatch and reach a high quality outcome, appropriate evaluation techniques need to be chosen and carried out throughout the development process of the modeling language. In this article, we present and discuss the formative user-centered evaluation approach, namely an evaluation technique that starts since the early design stages and actively involves end-users. We demonstrate the approach in a real case study presenting the results of the evaluation. From the gained empirical evidence, we may conclude that formative user-centered evaluation is highly recommended to investigate any security modeling language.