Evaluation of diagnostic tests when there is no gold standard. A review of methods

OBJECTIVE: To generate a classification of methods to evaluate medical tests whenthere is no gold standard.METHODS: Multiple search strategies were employed to obtain an overview of thedifferent methods described in the literature, including searches of electronicdatabases, contacting experts for papers in personal archives, exploringdatabases from previous methodological projects and cross-checking of referencelists of useful papers already identified.RESULTS: All methods available were classified into four main groups. The firstmethod group, impute or adjust for missing data on reference standard, needscareful attention to the pattern and fraction of missing values. The secondgroup, correct imperfect reference standard, can be useful if there is reliableinformation about the degree of imperfection of the reference standard and about the correlation of the errors between the index test and the reference standard. The third group of methods, construct reference standard, have in common thatthey combine multiple test results to construct a reference standard outcomeincluding deterministic predefined rules, consensus procedures and statisticalmodelling (latent class analysis). In the final group, validate index testresults, the diagnostic test accuracy paradigm is abandoned and researchexamines, using a number of different methods, whether the results of an indextest are meaningful in practice, for example by relating index test results torelevant other clinical characteristics and future clinical events.CONCLUSIONS: The majority of methods try to impute, adjust or construct areference standard in an effort to obtain the familiar diagnostic accuracystatistics, such as sensitivity and specificity. In situations that deviate only marginally from the classical diagnostic accuracy paradigm, these are valuablemethods. However, in situations where an acceptable reference standard does notexist, applying the concept of clinical test validation can provide a significantmethodological advance. All methods summarised in this report need furtherdevelopment. Some methods, such as the construction of a reference standard usingpanel consensus methods and validation of tests outwith the accuracy paradigm,are particularly promising but are lacking in methodological research. Thesemethods deserve particular attention in future research.