Albert Weichselbraun

is a Professor of Information Science at the University of Applied of the Grisons.

Metrics for Evaluation of Ontology-based Information Extraction

1 minute read

by Maynard et al.

This article describes metrics for evaluating ontologies. The article covers the following metrics:

  • precision, recall
  • False positives
  • cost-based evaluation metrics (CBE) such as time x salary-saved
  • learning accuracy (LA) by Hahn, which has been used by Cimiano et. al and yields a similarity between 0 and 1. The metric is computed based on the following data:
    • SP: root - key concept (=gold standard concept)
    • FP: root - predicted concept
    • CP: root - MSCA (=most specific common abstraction; the lowest (most specific) concept common to SP and FP)
    • DP: MSCA - predicted concept

  • correct results => CP = SP => LA=CP/SP = 1
  • incorrect results => LA = CP/(FP+DP)
  • one of the problems of LA is, that it does not consider the depth of the key concept in the hierarchy and therefore yields the same outcome regardless of how far away the key concept is. Maynard et al. show that this behavior sometimes leads to counter-intuitive results.
  • Augmented precision and recall (based on BDM): ¨For computing BDM we require:
    • MSCA
    • CP
    • DPR: MSCA - response (=predicted) concept
    • DPK: MSCA - key concept (=LA's DP)

  • All paths are normalized using (i) the average length of the chains and (ii) a branching factor.
  • BDM is then used to compute the augmented precision and recall

You may also enjoy

Suffix array

1 minute read

The suffix array is a memory-efficient alternative to the suffix tree which provides a sorted list of string indices indicating the string’s suffixes.