Evaluating semantically enriched information systems

Information systems should be designed to allow users to express their information requests in a simple way and to retrieve information that they consider as relevant to these requests. In order to fulfil these requirements, information system designers face many challenges w.r.t. selecting appropriate technologies and deciding on a modelling approach for their system. In this paper we present a conceptual framework that can be used to analyse the “fitness” of (semantically enriched) information systems w.r.t. queriability and information retrieval quality. It can help designers to spot the strengths and weaknesses of various modelling approaches as well as managing tradeoffs between modelling effort and their potential benefits. We evaluate this methodology in a case study on different modelling approaches for annotating medical images.