Ontology alignment enhanced using Bayesian Networks - results
by Ondřej váb (svabo at vse dot cz), modified: the 13th of May, 2007
Content of web:
Description of work
Evaluation
Dataset - 1
Dataset - 2
Dataset - 3
Dataset - 4 (graph based methods included)
Publications
Description of work
Original idea was to overcome inefficiencies of diverse types of mapping methods by combining them using Bayesian Networks. This page contains results of experiments done over several datasets. For evaluation I used two criteria, see section evaluation . Work has been presented at one international workshop and one conference, see publications.
Evaluation
For evaluation I use two criteria based on classical measures such as precision and recall. In the OM field we can define precision as ratio of number of mappings that are in discovered alignment as well as in reference alignment divided by number of mappings in discovered alignment. Recall is ratio of number of mappings that are in discovered alignment as well as in reference alignment divided by number of mappings in reference alignment.
First criterion is the weighted sum of precision and recall. On the base of the opinion that precision is more important than recall, particular weights are 0.6 and 0.4 respectively.
This opinion stems from the assumption that an inprecise set of mappings can be
more harmful for an application than an incomplete set of mappings. But, this does not always hold. It depends on the nature of applications. Therefore, the second criterion is average precision at three levels of recall (0.25%,0.50% and 0.75%). For obtaining numbers of recall and precision, I use the one-leave-out method. During the experiment I optimize all individual methods on training data and compare these results with results of BNs classifiers (also against simple method when system always asserts true).
Dataset - 1
Training data are manually labelled pairs of concepts from ontologies ekaw.owl a confOf.owl from OntoFarm collection
- 798 pairs
- 425 positive examples
- 373 negative examples
- still true, then the weighted sum (c) is 0.71, for weights a=0.6 and b=0.4
Results of string methods:
Results of Bayesian Networks:
Bayesian Network with learned structure:
Dataset - 2
Training data are manually labelled results from participants of OAEI-2006, Conference track.
- 2146 pairs
- 376 positive examples
- 1770 negative examples
- still true, then the weighted sum (c) is 0.5, for weights a=0.6 and b=0.4
Results of string methods:
Results of Bayesian Networks:
Bayesian Network with learned structure:
Dataset - 3
Training data are taken from track food from OAEI-2006. Because almost all data were positive examples, some negative examples from conference track were added.
- 1973 pairs
- 976 positive examples
- 997 negative examples
- still true, then the weighted sum (c) is 0.69, for weights a=0.6 and b=0.4
Results of string methods:
Results of Bayesian Networks:
Bayesian Network with learned structure:
Dataset - 4
Training data are manually labelled results from participants of OAEI-2006, Conference track. But now there are just class-to-class mappings taken into account.
- 1262 pairs
- 96 positive examples
- 1166 negative examples
- still true, then the weighted sum (c) is 0.44 (precision=0.076 and recall=1.0), for weights a=0.6 and b=0.4
Results of string and graph methods:
Results of Bayesian Networks:
Bayesian Network with learned structure:
Publications
1. váb O., Svátek V. Combining Ontology Mapping Methods Using Bayesian Networks. OM-2006 at ISWC-2006.
2. váb O., Svátek V. Ontology Mapping enhanced using Bayesian Networks. In: Proceedings of Znalosti 2007. 2007.
