Development and validation of methods for assessing the quality of diagnostic accuracy studies

OBJECTIVES: To develop a quality assessment tool which will be used in systematic reviews to assess the quality of primary studies of diagnostic accuracy.DATA SOURCES: Electronic databases including MEDLINE, EMBASE, BIOSIS and themethodological databases of both CRD and the Cochrane Collaboration.REVIEW METHODS: Three systematic reviews were conducted to provide an evidencebase for the development of the quality assessment tool. A Delphi procedure wasused to develop the quality assessment tool and the information provided by thereviews was incorporated into this. A panel of nine experts in the area ofdiagnostic accuracy studies took part in the Delphi procedure to agree on theitems to be included in the tool. Panel members were also asked to providefeedback on various other items and whether they would like to see thedevelopment of additional topic and design specific items. The Delphi procedureproduced the quality assessment tool, named the QUADAS tool, which consisted of14 items. A background document was produced describing each item included in thetool and how each of the items should be scored.RESULTS: The reviews produced 28 possible items for inclusion in the qualityassessment tool. It was found that the sources of bias supported by the mostempirical evidence were variation by clinical and demographic subgroups, disease prevalence/severity, partial verification bias, clinical review bias andobserver/instrument variation. There was also some evidence of bias for theeffects of distorted selection of participants, absent or inappropriate referencestandard, differential verification bias and review bias. The evidence for theeffects of other sources of bias was insufficient to draw conclusions. The third review found that only one item, the avoidance of review bias, was included inmore than 75% of tools. Spectrum composition, population recruitment, absent orinappropriate reference standard and verification bias were each included in50-75% of tools. Other items were included in less than 50% of tools. The second review found that the quality assessment tool should have the potential to bediscussed narratively, reported in a tabular summary, used as recommendations forfuture research, used to conduct sensitivity or regression analyses and used ascriteria for inclusion in the review or a primary analysis. This suggested thatsome distinction is needed between high- and low-quality studies. Componentanalysis was considered the best approach to incorporate quality into systematic reviews of diagnostic studies and this was taken into consideration whendeveloping the tool.CONCLUSIONS: This project produced an evidence-based quality assessment tool tobe used in systematic reviews of diagnostic accuracy studies. Through the variousstages of the project the current lack of such a tool and the need for asystematically developed validated tool were demonstrated. Further work tovalidate the tool continues beyond the scope of this project. The furtherdevelopment of the tool by the addition of design- and topic-specific criteria isproposed.