OAEI 2007 - Conference track (Consensus workshop)

Ontology Alignment Evaluation Initiative

Content of web:

Motivation and Task
Data sets
Organisation and Presentation
Time-schedule
Evaluation
Results of participants

Motivation and Task
Conference track contains quite real-case ontologies suitable for ontology mappings task because of their heterogenous character of origin. There are no reference alignments, therefore there are employed methods of a posteriori evaluation. Organisers of this track offer a posteriori evaluation of results in part manually and in part by data-mining techniques. Data-mining techniques can answer questions like (you can find more information about this type of evaluation in [1].

Which systems give higher/lower validity than others to the mappings that are deemed ‘in/correct’?
Which systems produce certain types of interwined mappings (so called mapping patterns) more often than others?
Which systems are more successful on certain types of ontologies?

Manual evaluation will produce statistics such as precision and will also serve as input into data-mining based of evaluation. During manual evaluation some interesting mappings will be chosen which can be subject of consensus building discussion. To sum up, special character of this track lies in way of evaluation (manual and data-mining approach, no prior reference alignment) and potential materials for consensus building discussion, which potentially enables to discuss:

alignment types unforeseen by organizers,
alignment types too sophisticated to be evaluated using standard metrics,
negative results as useful feedback.

Previous conference track was held at OAEI-2006.

Data sets

The collection of ontologies is dealing with conference organisation. They have been developed within OntoFarm project. 14 ontologies are included. Each of them is shortly described in the following table. Ontologies have been based upon three types of underlying resources:

actual conference (series) and its web pages (indicated in column Type with mark Web),
actual software tool for conference organisation support (indicated in column Type with mark Tool),
experience of people with personal participation in conference organisation (indicated in column Type with mark Insider).

Name is derived from name of conference or conference organisation tool. For downloading this ontology, just click on its name.
DL expressivity was obtained from SWOOP tool which is Hypermedia-based OWL Ontology Browser and Editor.

Name	Type	Number of Classes	Number of Datatype Properties	Number of Object Properties	DL expressivity	Related link	Notes
Ekaw	Insider	77	0	33	SHIN(D)	http://ekaw.vse.cz
Sofsem	Insider	60	18	46	ALCHIF(D)	http://www.sofsem.cz
Sigkdd	Web	49	11	17	ELI(D)	http://www.acm.org/sigs/sigkdd/kdd2006
Iasted	Web	140	3	38	ALCIF(D)	http://iasted.com/conferences/2005/cancun/ms.htm
Micro	Web	32	9	17	ALCIOF(D)	www.microarch.org	new
Confious	Tool	57	5	52	SHIN(D)	http://www.confious.com
Pcs	Tool	23	14	24	ELUIF(D)	http://precisionconference.com
OpenConf	Tool	62	21	24	ALCIO(D)	http://www.zakongroup.com/technology/openconf.shtml	the 9th of June, 2007. Bad xml:base URI has been repaired.
ConfTool	Tool	38	23	13	SIF(D)	http://www.conftool.net
Crs	Tool	14	2	15	DL-Lite	http://www.conferencereview.com
Cmt	Tool	36	10	49	ALCIF(D)	http://msrcmt.research.microsoft.com/cmt
Cocus	Tool	55	0	35	ALCIF(D)	http://cocus.create-net.it/	new
Paperdyne	Tool	47	21	61	ALCHIOF(D)	http://www.paperdyne.com/	new
Edas	Tool	104	20	30	ALCIF(D)	http://edas.info/	new

We deeply acknowledge the effort of contributors to the collection: Július Štuller (and his team), Du‘an Rak, Petr Tomá‘ek, Luděk Mácha, Lenka Navrátilová, Pavel Kříž, David Měrka, Roman Svoboda, Miroslav Joha, Tomá‘ Zelinka and Jozef Murín.
Organisation and Presentation
Send any questions, comments, or suggestions to:
Ondřej Šváb ( svabo at vse dot cz ) or Vojtěch Svátek ( svatek at vse dot cz ).

Final results have been presented at Ontology Matching workshop 2007. So-called 'Consensus building workshop' was on Monday. During presentation, 'controversial' mappings have been argued. Argumentation process is now under processing.
Presentation from 'Consensus building workshop' is here.

Time-schedule
For more information about time-schedule see main page about OAEI-2007.

Evaluation
Overall statistics are presented at the evaluation page.

References

[1] Svab O., Svatek V., Stuckenschmidt H.: A Study in Empirical and ‘Casuistic’ Analysis of Ontology Mapping Results. In: 4th European Semantic Web Conference (ESWC-2007), Innsbruck 2007. Springer LNCS 4519. Abstract Draft paper (final version available via SpringerLink).

Results of participants

Results of all participants and manually labeled data in this track can be downloaded from results page.