QUADAS-3: updated tool to evaluate risk of bias and applicability concerns in diagnostic test accuracy studies [oral presentation]

Background The QUADAS-2 tool, published in 2011, was designed to evaluate the risk of bias and applicability of diagnostic test accuracy (DTA) studies. The publication reporting QUADAS-2 has been cited over 12, 000 times and it is the recommended tool to assess risk of bias and applicability of studies for major HTA organizations. Although feedback on QUADAS-2 has generally been positive, some signaling questions have been identified as problematic and the tool could be improved based on features included in more recently developed tools. Objectives To update QUADAS-2 to develop the new QUADAS-3 tool. Methods We established a core-group of methodological experts to lead the development of the QUADAS-3 tool supported by a wider steering group. We followed the following steps: • Summarised modifications made to QUADAS-2 for the Cochrane Handbook • Web-based survey of reviewers that have used QUADAS-2 • Considered developments from more recent tools in terms of tool structure and implementation • Undertook a review of methodological studies that had evaluated QUADAS-2 • Undertook a review of 50 Cochrane DTA reviews to highlight challenges with the assessment of applicability We have produced a draft tool which is currently undergoing piloting. The results of the piloting, which will also include a comparison of the use of signalling questions with signalling statements, will be used to inform the final version of the tool. Results The new tool follows a similar structure to the QUADAS-2 tool but with some major updates. Key changes include: • An option to define separate synthesis questions rather than just a single review question • A new section on defining the ideal test accuracy trial for each synthesis question • Assessment of risk of bias and applicability at the accuracy estimate level rather than the study level • A change in answers to signaling questions to include options of “probably yes” and “probably no” and to replace “unclear” with “no information” • Replacement of “Flow and Timing domain” with new “Analysis” domain • Changes to some signaling questions • Inclusion of a section for judging overall risk of bias and applicability (across domains) Conclusions QUADAS-3 will be introduced at the conference and the results of piloting discussed.