Piloting QUADAS-3: A Revised Tool for the Quality Assessment of Diagnostic Accuracy Studies

Objective: QUADAS-2 is the most widely used tool for evaluating risk of bias and applicability concerns in diagnostic test accuracy studies within systematic reviews. QUADAS-2 has recently been updated to a new version, named QUADAS-3. This paper outlines the piloting process undertaken as part of the development of QUADAS-3. Study design and Setting: Multi-stage piloting: 1) piloting by the QUADAS-3 steering group on a set of five journal papers, 2) piloting workshop at the Global Evidence Summit attended by 16 participants, 3) think aloud interviews with seven researchers who piloted the tool whilst verbalising their thoughts, 4) piloting in five ongoing or completed systematic reviews by seven review authors who provided feedback in an online survey. Results: Feedback on the tool was generally positive across the four piloting stages. Participants appreciated the structure of the tool, assessment at the estimate level, and the introduction of a framework to define the ideal test accuracy trial. Participants provided suggestions for improvement to the structure and wording of the tool; this led to key changes including the insertion of descriptive prompts within the QUADAS-3 domains, a section at the beginning of the tool to outline the tool’s phases and when they should be completed, and clearer wording throughout the tool. Participants also identified areas where further guidance is required for users, including development of worked examples, which will be covered in the associated QUADAS-3 guidance document. Conclusions: Extensive piloting has ensured that feedback from potential users has been integrated into the development of QUADAS-3.