Evaluating the performance of object-oriented database systems by means of a concrete application

Due to the increasing relevance of object-oriented database systems, special benchmarks have been developed in the past in order to analyze performance. The benchmarks show unanimously that object-oriented systems are excellently suited for applications managing complex structures. In particular, these database systems provide a higher performance than relational ones. This is proven by synthetic schemas and elementary operations such as inserts and traversals, which are supposed to be characteristic of many object-oriented applications. This experience report presents the results of an application-specific investigation, comparing three commercial object-oriented database systems. The tests we performed took the elapsed times of a real application "as a whole". In spite of having similar characteristics to standard benchmarks, we obtained huge differences in performances. To put it in a nutshell, standard benchmarks are not always meaningful and should not let one jump to conclusions.