TY  - JOUR
AU  - Carrillo, Beltran
AU  - Rubinos-Cuadrado, Marta
AU  - Parellada-Martin, Jazmin
AU  - Palacios-López, Alejandra
AU  - Carrillo-Rubinos, Beltran
AU  - Canillas-Del Rey, Fernando
AU  - Baztán-Cortes, Juan Jose
AU  - Gómez-Pavon, Javier
PY  - 2025
DA  - 2025/4/14
TI  - Effectiveness of The Umbrella Collaboration Versus Traditional Umbrella Reviews for Evidence Synthesis in Health Care: Protocol for a Validation Study
JO  - JMIR Res Protoc
SP  - e67248
VL  - 14
KW  - tertiary evidence synthesis
KW  - The Umbrella Collaboration
KW  - umbrella reviews
KW  - health research methodology
KW  - AI-assisted synthesis
KW  - AI-assisted
KW  - evidence-based decision making
KW  - machine learning
KW  - ML
KW  - artificial intelligence
KW  - AI
KW  - algorithms
KW  - models
KW  - analytics
KW  - digital health
KW  - digital technology
KW  - digital interventions
AB  - Background: The synthesis of evidence in health care is essential for informed decision-making and policy development. This study aims to validate The Umbrella Collaboration (TU), an innovative, semiautomatic tertiary evidence synthesis methodology, by comparing it with Traditional Umbrella Reviews (TUR), which are currently the gold standard. Objective: This study aimed to evaluate whether TU, an artificial intelligence—assisted, software-driven system for tertiary evidence synthesis, can achieve comparable effectiveness to TURs, while offering a more timely, efficient, and comprehensive approach. In addition, as a secondary objective, the study aims to assess the accessibility and comprehensibility of TU’s outputs to ensure its usability and practical applicability for health care professionals. Methods: This protocol outlines a comparative study divided into 2 main parts. The first part involves a quantitative comparison of results obtained using TU and TURs in geriatrics. We will evaluate the identification, size effect, direction, statistical significance, and certainty of outcomes, as well as the time and resources required for each methodology. Data for TURs will be sourced from Medline (via PubMed), while TU will use artificial intelligence—assisted informatics to replicate the research questions of the selected TURs. The second part of the study assesses the ease of use and comprehension of TU through an online survey directed at health professionals, using interactive features and detailed data access. Results: Expected results include the assessment of concordance in identifying outcomes, the size effect, direction and significance of these outcomes, and the certainty of evidence. In addition, we will measure the operational efficiency of each methodology by evaluating the time taken to complete projects. User perceptions of the ease of use and comprehension of TU will be gathered through detailed surveys. The implementation of new methodologies in evidence synthesis requires validation. This study will determine whether TU can match the accuracy and comprehensiveness of TURs while offering benefits in terms of efficiency and user accessibility. The comparative study is designed to address the inherent challenges in validating a new methodology against established standards. Conclusions: If TU proves as effective as TURs but more time-efficient, accessible, and easily updatable, it could significantly enhance the process of evidence synthesis, facilitating informed decision-making and improving health care. This study represents a step toward integrating innovative technologies into routine evidence synthesis practice, potentially transforming health research. International Registered Report Identifier (IRRID): PRR1-10.2196/67248 
SN  - 1929-0748
UR  - https://www.researchprotocols.org/2025/1/e67248
UR  - https://doi.org/10.2196/67248
UR  - http://www.ncbi.nlm.nih.gov/pubmed/40057944
DO  - 10.2196/67248
ID  - info:doi/10.2196/67248
ER  -