INTRODUCTION Assessment of the applicability of primary studies to the review questions is an essential but often challenging aspect of systematic reviews of diagnostic test accuracy studies (DTA reviews). OBJECTIVES To explore review authors’ applicability assessments for the QUADAS-2 reference standard domain within Cochrane DTA reviews, by highlighting applicability concerns, identifying potential issues with assessments. Using the findings to develop a framework for assessing the applicability of the target condition as defined by the reference standard. METHODS Methodological review. DTA reviews in the Cochrane Library that used QUADAS-2 and judged applicability for the reference standard domain as “high concern” for at least one study, were eligible. One reviewer extracted the rationale for the “high concern”, this was checked by a second review er. Two reviewers categorized the rationale inductively into themes, a third reviewer verified these. Discussions in the QUADAS development group (steering group and expert panel) regarding the extracted information informed frame work development. RESULTS We identified 50 eligible reviews. We identified potential issues over half (n=28) of the included reviews. In 7, applica bility assessments deviated from QUADAS-2 definitions and in 21 applicability concerns were insufficiently described. Five themes emerged in the remainder: study uses different refer ence standard threshold to define the target condition (6 re views), misclassification by the reference standard in the study such that the target condition in the study does not match the review question (11 reviews), reference standard could not be applied to all participants resulting in a different target condi tion (5 reviews), misunderstanding QUADAS-2 applicability (7 reviews), and insufficient information (21 reviews). The Figure shows the framework that was informed by our findings and discussions. Our framework outlines four potential applicability concerns for the assessment of the tar get condition as defined by the reference standard: different sub-categories of the target condition, different threshold used to define the target condition, reference standard not applied to full study group, and misclassification of the target condition by the reference standard (Figure). CONCLUSION Clear sources of applicability concerns are identifiable, but several Cochrane review authors struggle to adequately identify and report them. We have developed the first applicability framework to guide review authors in their assessment of applicability concerns for the QUADAS reference standard domain. The development of a framework for the QUADAS-2 domains Patient selection and Index Test are ongoing.

Developing a Framework for Assessing the Applicability of the Target Condition in Diagnostic Research

Rutjes Anne W S
;
2025-01-01

Abstract

INTRODUCTION Assessment of the applicability of primary studies to the review questions is an essential but often challenging aspect of systematic reviews of diagnostic test accuracy studies (DTA reviews). OBJECTIVES To explore review authors’ applicability assessments for the QUADAS-2 reference standard domain within Cochrane DTA reviews, by highlighting applicability concerns, identifying potential issues with assessments. Using the findings to develop a framework for assessing the applicability of the target condition as defined by the reference standard. METHODS Methodological review. DTA reviews in the Cochrane Library that used QUADAS-2 and judged applicability for the reference standard domain as “high concern” for at least one study, were eligible. One reviewer extracted the rationale for the “high concern”, this was checked by a second review er. Two reviewers categorized the rationale inductively into themes, a third reviewer verified these. Discussions in the QUADAS development group (steering group and expert panel) regarding the extracted information informed frame work development. RESULTS We identified 50 eligible reviews. We identified potential issues over half (n=28) of the included reviews. In 7, applica bility assessments deviated from QUADAS-2 definitions and in 21 applicability concerns were insufficiently described. Five themes emerged in the remainder: study uses different refer ence standard threshold to define the target condition (6 re views), misclassification by the reference standard in the study such that the target condition in the study does not match the review question (11 reviews), reference standard could not be applied to all participants resulting in a different target condi tion (5 reviews), misunderstanding QUADAS-2 applicability (7 reviews), and insufficient information (21 reviews). The Figure shows the framework that was informed by our findings and discussions. Our framework outlines four potential applicability concerns for the assessment of the tar get condition as defined by the reference standard: different sub-categories of the target condition, different threshold used to define the target condition, reference standard not applied to full study group, and misclassification of the target condition by the reference standard (Figure). CONCLUSION Clear sources of applicability concerns are identifiable, but several Cochrane review authors struggle to adequately identify and report them. We have developed the first applicability framework to guide review authors in their assessment of applicability concerns for the QUADAS reference standard domain. The development of a framework for the QUADAS-2 domains Patient selection and Index Test are ongoing.
2025
Applicability, Systematic review, QUADAS-2, Reference standard, Target condition
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/20.500.14245/14760
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus ND
social impact