OAEI-2006 - Consensus workshop

Ontology Alignment Evaluation Initiative

!NEW! OAEI-2007 - seeking contributions to the 'conference' track collection !NEW!

Content of web:

Motivation and Task
Data sets
Organisation and Presentation
Evaluation Results of participants

Motivation and Task

As a complement to current type of ontology alignment evaluation scenario, this year will also be held a consensus workshop. It will serve for presentation and discussion of results of mining for interesting ontology alignments. There will be no prior reference alignment, and consensus on alignment will be reached via discussion. Compared to the contest scenario, it potentially enables to discuss:

Data sets

The collection of ontologies is dealing with conference organisation. They have been developed within OntoFarm project. 10 ontologies are included. Each of them is shortly described in the following table. Ontologies have been based upon three types of underlying resources:

Name is derived from name of conference or conference organisation tool. For downloading this ontology, just click on its name.
DL expressivity was obtained from SWOOP tool which is Hypermedia-based OWL Ontology Browser and Editor.

NameTypeNumber of ClassesNumber of Datatype Properties Number of Object PropertiesDL expressivityRelated link Notes
EkawInsider77033SHIN(D)http://ekaw.vse.cz the 22nd of July, 2006; ontology has been changed. Inconsistencies have been removed.
SofsemInsider601846ALCHIF(D)http://www.sofsem.cz
SigkddWeb491117ELI(D)http://www.acm.org/sigs/sigkdd/kdd2006
IastedWeb140338ALCIF(D)http://iasted.com/conferences/2005/cancun/ms.htm
ConfiousTool57552SHIN(D)http://www.confious.com
PcsTool231424ELUIF(D)http://precisionconference.com
OpenConfTool622124ALCIO(D)http://www.zakongroup.com/technology/openconf.shtml
ConfToolTool382313SIF(D)http://www.conftool.net the 29th of July, 2006; ontology has been changed. One typo has been corrected (Membet_PC replaced with Member_PC).
CrsTool14215DL-Litehttp://www.conferencereview.com
CmtTool361049ALCIF(D)http://msrcmt.research.microsoft.com/cmt

We deeply acknowledge the effort of contributors to the collection: Július Štuller (and his team), Dušan Rak, Petr Tomášek, Luděk Mácha, Lenka Navrátilová, Pavel Kříž, and David Merka.

Organisation and Presentation

Send any questions, comments, or suggestions to:
Ondřej Šváb ( svabo at vse dot cz ) or Vojtěch Svátek ( svatek at vse dot cz ).

Final results have been presented at Ontology Matching workshop 2006 during so-called 'Consensus building workshop'. During presentation, 'controversial' mappings have been argued. Argumentation process is now under processing.
Presentation from 'Consensus building workshop' is here.

Evaluation

According to the nature of this track, we are mainly interested in some "interesting" mappings ("nuggets"). Although traditional evaulation was not our intention, we made some sort of evaluation as a side-effect of processing results from our six participants. All the statistics as well as precision and recall have been provisionally made by track organisers, who can often be subjective; the focus of the track is on interesting individual alignments and repeated patterns rather than on precision/recall figures.



The abovementioned table encompasses several numerical statistics related to the results of six participants, called according to name of their systems (Automs, Coma++, OWL-CtxMatch, Falcon, HMatch and RiMOM). Finally, there is row with overall statistics. In the following, columns are explained: The following columns are dealing with measure of precision and recall: Additional comments:

Results of participants

Results of all participants and manually labeled data in this track can be downloaded from results page.