Generating Reference Mappings for Evaluating Ontology Matching Tools

Ontology Alignment is one of the biggest challenges of semantic web research. Recently there has been some attention on automatic alignment methods and number of matching tools have been developed. One of the open problem of automatic matching currently is the evaluation of the quality of automatically generated mappings. Normally, reference mappings are used to compute the precision and recall of automatically created mappings. Often, however, such reference mappings do not exist and creating them by hand is not feasible for large ontologies. In this paper, we extend previous work on overcoming this problem by proposing a method for automatically generating highly correct reference mappings. In particular, we consider the case where we do not have shared instances for the ontologies to be aligned and report the results of an experiment in which we used automatic document classification as a basis for creating a reference mapping. We present the results of our experiment on medical datasets and show that the result provides a useful basis for comparing matching systems.