Knowledge base validation

Completeness and consistency checking tools do seem to identify a significant number of errors in typical rule-bases. One argument put forward by the proponents of such tools is that this shows that they are appropriate and effective. An alternative interpretation, however, is that if fairly crude tools can identify problems-how many deeper problems must remain undetected? Some experimental evidence is provided by Pearce (1988) that the latter argument is true when a model based approach is introduced. A direct comparison was carried out between a knowledge base synthesised from a model and a knowledge base produced through human prototyping. The prototyped system correctly covered some 70-80% of test cases, while the synthesised rules correctly dealt with 100%. In addition, when both systems were integrity checked, further defaults were found with the manual system. Recent work at the Turing Institute has shown that significantly more accurate knowledge bases can be derived from causal, qualitative models of systems, rather than having a human expert provide rules directly. In addition, model-based systems have many other advantages over shallow associational knowledge