Logo des Repositoriums
 

Measuring the effect of a guideline-based training on ontology design with a competency questions based evaluation approach

Kurzanzeige
dc.contributor.authorBoeker, Martin
dc.contributor.authorGrewe, Niels
dc.contributor.authorRöhl, Johannes
dc.contributor.authorSchober, Daniel
dc.contributor.authorSchulz, Stefan
dc.contributor.authorSeddig-Raufie, Djamila
dc.contributor.authorJansen, Ludger
dc.contributor.editorHorbach, Matthias
dc.date.accessioned2019-03-07T09:31:28Z
dc.date.available2019-03-07T09:31:28Z
dc.date.issued2013
dc.description.abstractOBJECTIVE: (a) To measure the effect of a guideline-based training on the performance of ontology developers compared with the performance after unspecific training by a competency question based evaluation; and (b) to provide empirical evidence for the applicability of competency questions in formal ontology evaluation in general. BACKGROUND: A close connection between ontology development and ontology evaluation as quality management procedure can been attained with the use of competency questions. Competency questions are often used as a semi-formal specification of requirements for an ontology under development. Hence they can also be used as evaluation instruments, in order to check how far an ontology fulfills these requirements. METHODS: A randomized controlled trial was conducted with two groups of 12 students each. The intervention consisted in a differential guideline-based training on ontology development vs. unspecific training. After a group-specific training focusing on three topics per group, performance of students was assessed with 12 exercises (2 exercises per topic), in which the students had to apply their skills. Different types of competency questions were elaborated for the analysis of the students' ontologies. We used the proportion of correct answers as a measure of ontology quality. RESULTS: On single topic level, the performance of ontology developers increased after guideline-based training for two out of the six topics: it increased from a proportion of 0.46 to 0.63 for the topic Process \& Participation and from 0.44 to 0.53 for the topic Collective Material Entity. In regression analysis, a positive correlation was shown between the performance of students on untrained topics and the performance after specific guideline-based training. Moreover, in multiple regression analysis an overall effect of specific training of 0.09 was calculated (p < 0.1). CONCLUSION: The results show an effect of a specific guideline-based training on the performance of ontology developers compared to the performance after unspecific training by an increase of about 10 % on the rate of correct competency questions. In addition, this study has shown the general applicability of competency questions in a formal ontology evaluation scenario. However, the study also shows that the training of ontology developers and their performance evaluation is a tedious task. The resulting performance of ontology developers is more dependent on the a priori individual competencies than on the specific acquired skills after training.en
dc.identifier.isbn978-3-88579-614-5
dc.identifier.pissn1617-5468
dc.identifier.urihttps://dl.gi.de/handle/20.500.12116/20611
dc.language.isoen
dc.publisherGesellschaft für Informatik e.V.
dc.relation.ispartofINFORMATIK 2013 – Informatik angepasst an Mensch, Organisation und Umwelt
dc.relation.ispartofseriesLecture Notes in Informatics (LNI) - Proceedings, Volume P-220
dc.titleMeasuring the effect of a guideline-based training on ontology design with a competency questions based evaluation approachen
dc.typeText/Conference Paper
gi.citation.endPage1795
gi.citation.publisherPlaceBonn
gi.citation.startPage1783
gi.conference.date16.-20. September 2013
gi.conference.locationKoblenz
gi.conference.sessiontitleRegular Research Papers

Dateien

Originalbündel
1 - 1 von 1
Vorschaubild
Name:
1783.pdf
Größe:
204.43 KB
Format:
Adobe Portable Document Format
Herunterladen

Sammlungen

P220 - INFORMATIK 2013 - Informatik angepasst an Mensch, Organisation und Umwelt