Evaluation of the Model

In the evaluation phase, the expert system is tested by comparing the differential diagnosis at each stage of the patient workup with the expert’s opinion and determining where the knowledge base might be revised to improve performance. This process is carried out first. If the system appears to be overconfident about a particular diagnosis (i.e., assigns a higher probability to that diagnosis than the expert thinks appropriate), suspicion is raised that the false positive rate assigned to some data item already entered for the patient being diagnosed is too low. Another explanation might be that highly associated findings are being treated as independent; i.e., they need to be represented as an “or” group or built into a cluster.