This paper is in the following e-collection/theme issue:

Systematic Review Protocols (268) Chatbots and Conversational Agents (531) Generative Language Models Including ChatGPT (311) Mobile Apps for Cardiology (141) Reviews on Cardiovascular Health and Medicine (26) mHealth for Symptom and Disease Monitoring, Chronic Disease Management (1399) Aging with Chronic Disease (110)

Published on in Vol 13 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/52973, first published .
Engagement With Conversational Agent–Enabled Interventions in Cardiometabolic Disease Management: Protocol for a Systematic Review

Engagement With Conversational Agent–Enabled Interventions in Cardiometabolic Disease Management: Protocol for a Systematic Review

Engagement With Conversational Agent–Enabled Interventions in Cardiometabolic Disease Management: Protocol for a Systematic Review

Authors of this article:

Nick Kashyap1, 2 Author Orcid Image ;   Ann Tresa Sebastian2, 3 Author Orcid Image ;   Chris Lynch1, 2, 4, 5 Author Orcid Image ;   Paul Jansons2, 3, 6 Author Orcid Image ;   Ralph Maddison2, 3 Author Orcid Image ;   Tilman Dingler2, 7 Author Orcid Image ;   Brian Oldenburg1, 2, 4, 5 Author Orcid Image
Article Authors Cited by Tweetations (1) Metrics

    Protocol

    1Baker Department of Cardiovascular Research, Translation and Implementation, La Trobe University, Melbourne, Australia

    2Centre for Research Excellence in Digital Technology to Transform Chronic Disease Outcomes, National Health and Medical Research Council, Melbourne, Australia

    3Institute for Physical Activity and Nutrition, School of Exercise and Nutrition Sciences, Deakin University, Geelong, Melbourne, Australia

    4Baker Heart and Diabetes Institute, Melbourne, Australia

    5School of Psychology & Public Health, La Trobe University, Melbourne, Australia

    6Department of Medicine, School of Clinical Sciences at Monash Health, Monash University, Clayton, Melbourne, Australia

    7Delft University of Technology, Delft, Netherlands

    Corresponding Author:

    Nick Kashyap, BSc, MBS

    Baker Department of Cardiovascular Research, Translation and Implementation

    La Trobe University

    Plenty Road and Kingsbury Dr

    Bundoora

    Melbourne, 3086

    Australia

    Phone: 61 422023197

    Email: Nick.Kashyap@baker.edu.au


    Background: Cardiometabolic diseases (CMDs) are a group of interrelated conditions, including heart failure and diabetes, that increase the risk of cardiovascular and metabolic complications. The rising number of Australians with CMDs has necessitated new strategies for those managing these conditions, such as digital health interventions. The effectiveness of digital health interventions in supporting people with CMDs is dependent on the extent to which users engage with the tools. Augmenting digital health interventions with conversational agents, technologies that interact with people using natural language, may enhance engagement because of their human-like attributes. To date, no systematic review has compiled evidence on how design features influence the engagement of conversational agent–enabled interventions supporting people with CMDs. This review seeks to address this gap, thereby guiding developers in creating more engaging and effective tools for CMD management.

    Objective: The aim of this systematic review is to synthesize evidence pertaining to conversational agent–enabled intervention design features and their impacts on the engagement of people managing CMD.

    Methods: The review is conducted in accordance with the Cochrane Handbook for Systematic Reviews of Interventions and reported in accordance with PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. Searches will be conducted in the Ovid (Medline), Web of Science, and Scopus databases, which will be run again prior to manuscript submission. Inclusion criteria will consist of primary research studies reporting on conversational agent–enabled interventions, including measures of engagement, in adults with CMD. Data extraction will seek to capture the perspectives of people with CMD on the use of conversational agent–enabled interventions. Joanna Briggs Institute critical appraisal tools will be used to evaluate the overall quality of evidence collected.

    Results: This review was initiated in May 2023 and was registered with the International Prospective Register of Systematic Reviews (PROSPERO) in June 2023, prior to title and abstract screening. Full-text screening of articles was completed in July 2023 and data extraction began August 2023. Final searches were conducted in April 2024 prior to finalizing the review and the manuscript was submitted for peer review in July 2024.

    Conclusions: This review will synthesize diverse observations pertaining to conversational agent–enabled intervention design features and their impacts on engagement among people with CMDs. These observations can be used to guide the development of more engaging conversational agent–enabled interventions, thereby increasing the likelihood of regular intervention use and improved CMD health outcomes. Additionally, this review will identify gaps in the literature in terms of how engagement is reported, thereby highlighting areas for future exploration and supporting researchers in advancing the understanding of conversational agent–enabled interventions.

    Trial Registration: PROSPERO CRD42023431579; https://tinyurl.com/55cxkm26

    International Registered Report Identifier (IRRID): DERR1-10.2196/52973

    JMIR Res Protoc 2024;13:e52973

    doi:10.2196/52973

    Keywords

    cardiometabolic disease; cardiovascular disease; diabetes; chronic disease; chatbot; acceptability; technology acceptance model; design; natural language processing; adult; heart failure; digital health intervention; Australia; systematic review; meta-analysis; digital health; conversational agent–enabled; health informatics; management 


    Cardiometabolic diseases (CMDs) are a group of interrelated conditions, including heart failure and diabetes, that increase the risk of cardiovascular and metabolic complications [1]. The number of Australians managing CMDs is increasing, necessitating new approaches to better manage these conditions [2,3]. Approximately one-quarter of people globally are estimated to be living with metabolic syndrome and approximately 1 in 13 are living with cardiovascular disease [4,5]. The self-management of CMDs requires individuals to adhere to treatment regimens such as taking prescribed medications, maintaining a healthy diet, and performing regular physical activity to effectively manage their condition [6]. As these lifestyle modifications can be complex and ongoing, digital health interventions have emerged as an integral part of a scalable and accessible strategy to support individuals self-managing CMD [7,8].

    In CMD contexts, the prolonged course of illness, often spanning decades, increases the difficulty of sustained engagement with self-management, where the risk of relapse remains a significant concern over time [9,10]. Therefore, it is essential that interventions aiming to improve self-management must be able to sustain user engagement over long time periods [10]. Indeed, low engagement rates undermine the effectiveness of digital health interventions; a recent review found an average participant dropout rate of 43% in studies assessing digital health interventions applied to the self-management of chronic diseases [11].

    A recent systematic review identified improving personalization and interactivity as effective approaches to improving long-term engagement rates of digital health interventions [12,13]. This is particularly crucial in managing CMDs, where ongoing engagement and sustained behavior change are necessary for the effective attenuation of CMD complications [10]. As components within digital health interventions, conversational agents could greatly enhance the effectiveness of these self-management interventions by contextualizing information, offering constructive feedback, and fostering critical reflection [14-18]. A conversational agent is a technology that interacts with people using natural language, whether text-based or spoken, enabling accessibility to broader populations, including people with motor or cognitive disabilities [19]. Use of conversational agents could also reveal a linguistic dimension to a person’s health status, which could improve the personalization and effectiveness of digital health interventions [16,20].

    Within the emerging research field of conversational agent–enabled interventions, the majority of existing tools are text-based, driven by machine learning algorithms, and delivered through mobile apps [21]. The literature on conversational agent–enabled interventions that support individuals with chronic diseases, including CMDs, to manage their condition is mainly composed of qualitative studies, quasiexperimental studies, pilot tests, and a limited number of randomized controlled trials, which all primarily examine prototypes of these interventions [22-25]. In clinical contexts, mental and physical wellness are the primary domains for the application of conversational agents, where they often provide emotional support [26,27]. In contrast, conversational agents applied to CMD self-management tend to provide a modality for monitoring symptoms and to deliver patient education [26]. However, recent trends indicate a shift from these basic functions to more complex and long-term end points in future applications, such as motivating behavior change [21]. This shift has occurred in parallel with technological advancements in cloud computing and transformer models, which have enabled a new generation of conversational agents termed large language models.

    Large language models use deep neural networks to handle complex language tasks such as summarizing, generating, and translating natural language [28]. These tasks are achieved not by large language models understanding the prompts but rather by repeatedly predicting the word that is statistically the most likely to follow a sequence of words given as a prompt until a full response is offered [29,30]. However, this predictive approach means that large language models are potentially unsafe in clinical contexts, as they are liable to misinterpret prompts or generate unforeseen and inaccurate content [31-33]. Additionally, the opacity of this predictive approach hinders the ability to anticipate all possible outputs, making it challenging to establish safeguards that effectively prevent the dissemination of harmful or misleading information [34,35]. Furthermore, integrating these models into health care systems can involve transmitting sensitive, patient-identifiable data to third-party servers for processing, thereby imposing significant legal and ethical risks on health care providers [36,37].

    Evidence indicates that the design features of conversational agent–enabled interventions impact the engagement of people with mental health issues in managing their condition [38,39]. For example, studies indicate that specific design features such as enhancing the anthropomorphic qualities of conversational agent–enabled interventions can improve engagement by reducing the monotony of repeated interactions [38,40]. Additionally, a recent review explored how design features pertaining to conversational architecture, such as delaying responses, proactive dialogues, and self-disclosures, affected user perceptions of conversational agents [41]. However, to date, there has not been a synthesis of evidence pertaining to conversational agent–enabled intervention design features (eg, personality) and their impacts on the engagement of people managing CMDs. In filling this gap, this review will contribute to the broader developer community making better informed choices when developing conversational agents, leading to more engaging, and therefore effective, conversational agent–enabled interventions for CMD self-management.


    Protocol and Registration

    This protocol is reported in accordance with the Cochrane Handbook for Systematic Reviews of Interventions and the PRISMA-P (Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols) checklist [42,43]. The protocol for this review was registered in the International Prospective Register of Systematic Reviews (PROSPERO; CRD42023431579) in June 2023.

    Eligibility Criteria

    Population

    The review will include studies of adult participants (aged≥18 years) with a CMD diagnosis. CMDs are a group of interrelated conditions, including heart failure and diabetes, that increase the risk of cardiovascular and metabolic complications [1]. Populations reported with comorbidities such as mental health disorders or multiple CMDs will also be included.

    Intervention

    The review will include studies reporting on conversational agent–enabled interventions to assist people with CMD in managing their condition, such as by offering emotional or educational support and assisting individuals to monitor their symptoms.

    Outcomes

    The review will include studies reporting engagement outcomes such as ratings, interviews, analytics, and focus groups.

    Study Design

    The review will include primary research studies, including qualitative studies, quasiexperimental studies, observational studies, and randomized controlled trials. Reviews, editorials, protocols, and non-English publications will be excluded.

    Information Sources

    We systematically searched the Ovid (Medline), Web of Science, and Scopus databases for relevant articles from inception until April 2024. Ovid (Medline) was selected owing to its medical and health science focus, which is useful for capturing research on the topic of CMDs. Web of Science and Scopus were both selected for their broader focus, which is useful for capturing research on conversational agents and engagement topics. To ensure a manageable scope, several databases were ruled out during preliminary searches. For example, Embase was ruled out because it had redundancy with Medline. In addition, the Cochrane Library was too specific, with 4 of the 5 exemplar papers present in all other databases not found in the Cochrane Library database. As this is a rapidly developing field, final searches will be conducted prior to the submission of a manuscript and additional studies that meet the inclusion and exclusion criteria will be incorporated into the review. Reference list searches will be conducted on all articles included in the full-text review.

    Search Strategy

    An extensive set of search terms will be used related to the three central topics of the review: CMD, conversational agents, and engagement. Boolean operators will be used to combine search terms, including the following search string: (“Cardiovascular Diseases”[MeSH Terms] OR metabolic) AND (“conversational agent*” OR chatbot*) AND (accept* OR perceived). This approach is intended to yield a comprehensive collection of literature that explores the intersection of these central topics. An example of a complete search strategy is provided in Multimedia Appendix 1.

    Data Management

    All search results will initially be imported into PaperPile, followed by removal of duplicate entries. The deduplicated library will then be exported from PaperPile and imported into Covidence (Veritas Health Innovation). Covidence (cloud-based software) will be used to store PDF files of articles considered during the full-text review. Additionally, Covidence will be used to store data extraction and quality appraisal forms, study selection results, and reviewer comments. Covidence will be used for data extraction and management and then the data will be exported to Microsoft Excel.

    Study Selection

    During initial screening, two reviewers (NK and ATS) will independently examine the titles and abstracts of all studies collected from the search strategy. Assessments will be based on the defined inclusion and exclusion criteria. If conflicts arise, they will be settled during meetings between the two reviewers. If a conflict is not able to be resolved in this manner, the matter will be arbitrated by a third reviewer to achieve consensus.

    During full-text screening, the remaining studies will be independently assessed by two reviewers to determine inclusion or exclusion using defined inclusion and exclusion criteria. Any conflict between the two reviewers will be settled during meetings, with arbitration by a third reviewer when required.

    Data Extraction

    Data extraction from all included studies, including any supplementary material, will be performed and documented by two independent reviewers. Where more detailed information is required, authors will be contacted once for clarification.

    The data extraction form will be designed to capture a wide array of details necessary to achieve the outcomes described above, primarily by capturing the perspectives of people with CMD on the use of conversational agent–enabled interventions. These details will include bibliographic information, study design, participants and population, intervention, and design features in terms of engagement outcomes.

    Quality Appraisal

    The following Joanna Briggs Institute critical appraisal tools will be used: Checklist for Qualitative Research for qualitative research [44], Checklist for Quasi-Experimental Studies for quasiexperimental studies [45], and Assessment of Risk of Bias for Randomized Controlled Trials for randomized controlled trials [46]. The quality of evidence assessment results will be presented in a summary of findings table.

    Data Synthesis

    An adapted version of the thematic synthesis analysis method developed by Thomas and Harden [47] will be used. This method will focus on data extracted from studies that detail design features and engagement outcomes. During the initial phase, these data will be categorized under multiple domains. Following the extraction, these domains will be iteratively consolidated until further merging would detract from the descriptive accuracy of the domains regarding their respective data sets. Subsequently, within each primary domain, the data will be further sorted into subdomains in a similar manner of iterative consolidation. Tables will be used to detail various aspects of the extracted studies, including study design, population, conversational agent characteristics, and design feature domains. Analysis of these themes will contribute to informing design choices for conversational agent–enabled interventions and to identify research gaps in how engagement tends to be reported in the literature.


    The review was initiated in May 2023 and, prior to title and abstract screening, was registered with PROSPERO in June 2023. Full-text screening of articles was completed in August 2023, followed by data extraction. Final searches were conducted in April 2024 (Figure 1) and the completed review was submitted to Digital Health in July 2024. Data pertaining to study design and sample size have been extracted and are summarized in Table 1.

    ‎
    Figure 1. Flowchart illustrating the study selection process.
    Table 1. Characteristics of included studies.
    Reference, publication yearStudy designConversational agent role
    Apergi et al [22], 2021QuasiexperimentalTo ask the patients the same series of questions related to their heart failure treatment and symptoms and provide feedback
    Balsa et al [48], 2019QualitativeTo assist older people with type 2 diabetes mellitus in medication adherence and lifestyle changes
    Balsa et al [49], 2020QualitativeTo support older people with type 2 diabetes mellitus in medication adherence and lifestyle changes
    Baptista et al [23], 2020QuasiexperimentalTo deliver self-management education and support to adults with type 2 diabetes mellitus
    Cheng et al [24], 2018QualitativeTo provide a less cumbersome way for older patients with type 2 diabetes mellitus to effectively adhere to guidelines
    Echeazarra et al [50], 2021Randomized controlled trialTo help patients with hypertension self-monitor their blood pressure
    Epalte et al [25], 2023QualitativeTo counsel, educate, and train patients and family members with stroke with regard to rehabilitation, care, and other related issues
    Gingele et al [51], 2023QualitativeTo evaluate patients’ health status, provide patient education, and enable communication with heart failure nurses
    Gong et al [52], 2020Randomized controlled trialTo provide more accessible and engaging self-management support, monitoring, and coaching to adults with type 2 diabetes mellitus in Australia
    Guhl et al [53], 2020QuasiexperimentalTo augment patient-centered health care by providing health education, monitoring, and problem-solving for users
    Kimani et al [54], 2016QuasiexperimentalTo provide education on atrial fibrillation and promote adherence to daily heart rhythm monitor readings
    Magnani et al [55], 2017QuasiexperimentalTo promote education, motivation, and monitor patient symptoms and adherence to behaviors
    Roca et al [56], 2021QuasiexperimentalTo improve medication adherence in patients with comorbid type 2 diabetes mellitus and depressive disorder
    Sagstad et al [57], 2022QualitativeTo educate women with gestational diabetes mellitus
    ter Stal et al [58], 2021QualitativeTo support users in the self-management of chronic diseases in a long-term, daily life setting
    Tongpeth et al [59], 2018QualitativeTo improve patients’ knowledge of and response to acute coronary syndrome symptoms
    Tsai et al [60], 2022QualitativeTo support patients with chronic kidney disease manage their condition
    Zhang et al [61], 2015QualitativeTo counsel patients on their diagnoses and medications specified by a clinician, as well as increasing physical activity, improving diet, decreasing stress, and motivating them to be more involved and proactive in their own care

    The effectiveness of digital health interventions in supporting people with CMD is dependent on the extent to which users engage with the digital tool [17,18]. Augmenting digital health interventions with conversational agents, technologies that interact with people using natural language, may enhance engagement because of their human-like attributes [24,25,57]. This protocol outlines a systematic review that aims to synthesize evidence pertaining to conversational agent–enabled intervention design features and their impacts on the engagement of people managing CMDs. The anticipated outcomes of the analysis include the identification of specific design features or themes within various domains that improve engagement with digital health interventions. Additionally, the quality appraisal process is expected to uncover research gaps on conversational agent–enabled interventions, thereby providing a clearer direction for future studies applied to investigating the design of conversational agent–enabled digital health interventions.

    This synthesis will provide guidance on how best to embed engagement strategies within these interventions, thereby facilitating more engaging and effective strategies for supporting people managing CMDs. Additionally, characterizing the literature, in terms of how engagement tends to be reported, will have the benefit of identifying research gaps and highlighting areas for future exploration. These outcomes can in turn support researchers in developing a greater understanding of user engagement with conversational agent–enabled interventions.

    Acknowledgments

    We are grateful to Angela Johns-Hayden, Senior Research Librarian at La Trobe University, for helping in the preparation of the search strategy.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Search strategy.

    DOCX File , 13 KB
    1. Sattar N, Gill JMR, Alazawi W. Improving prevention strategies for cardiometabolic disease. Nat Med. Mar 09, 2020;26(3):320-325. [CrossRef] [Medline]
    2. Australian Burden of Disease Study 2022. Australian Institute of Health and Welfare. URL: https:/​/www.​aihw.gov.au/​reports/​burden-of-disease/​australian-burden-of-disease-study-2022/​contents/​summary [accessed 2024-07-11]
    3. Change of heart: time to end cardiovascular complacency. Baker Heart and Diabetes Institute. 2019. URL: https://baker.edu.au/impact/advocacy/change-of-heart [accessed 2024-07-11]
    4. Saklayen MG. The global epidemic of the metabolic syndrome. Curr Hypertens Rep. Feb 26, 2018;20(2):12. [FREE Full text] [CrossRef] [Medline]
    5. Lindstrom M, DeCleene N, Dorsey H, Fuster V, Johnson CO, LeGrand KE, et al. Global burden of cardiovascular diseases and risks collaboration, 1990-2021. J Am Coll Cardiol. Dec 20, 2022;80(25):2372-2425. [FREE Full text] [CrossRef] [Medline]
    6. Grover A, Joshi A. An overview of chronic disease models: a systematic literature review. Glob J Health Sci. Oct 29, 2014;7(2):210-227. [FREE Full text] [CrossRef] [Medline]
    7. McLean G, Band R, Saunderson K, Hanlon P, Murray E, Little P, et al. DIPSS co-investigators. Digital interventions to promote self-management in adults with hypertension systematic review and meta-analysis. J Hypertens. Apr 2016;34(4):600-612. [FREE Full text] [CrossRef] [Medline]
    8. Wongvibulsin S, Martin SS, Steinhubl SR, Muse ED. Connected health technology for cardiovascular disease prevention and management. Curr Treat Options Cardiovasc Med. May 18, 2019;21(6):29. [FREE Full text] [CrossRef] [Medline]
    9. Lorig KR, Holman HR. Self-management education: history, definition, outcomes, and mechanisms. Ann Behav Med. Aug 2003;26(1):1-7. [CrossRef] [Medline]
    10. Fisher EB, Brownson CA, O'Toole ML, Shetty G, Anwuri VV, Glasgow RE. Ecological approaches to self-management: the case of diabetes. Am J Public Health. Sep 2005;95(9):1523-1135. [CrossRef] [Medline]
    11. Meyerowitz-Katz G, Ravi S, Arnolda L, Feng X, Maberly G, Astell-Burt T. Rates of attrition and dropout in app-based interventions for chronic disease: systematic review and meta-analysis. J Med Internet Res. Sep 29, 2020;22(9):e20283. [FREE Full text] [CrossRef] [Medline]
    12. Chew HSJ. The use of artificial intelligence-based conversational agents (chatbots) for weight loss: scoping review and practical recommendations. JMIR Med Inform. Apr 13, 2022;10(4):e32578. [FREE Full text] [CrossRef] [Medline]
    13. Miner AS, Milstein A, Schueller S, Hegde R, Mangurian C, Linos E. Smartphone-based conversational agents and responses to questions about mental health, interpersonal violence, and physical health. JAMA Intern Med. May 01, 2016;176(5):619-625. [FREE Full text] [CrossRef] [Medline]
    14. Kasneci E, Sessler K, Küchemann S, Bannert M, Dementieva D, Fischer F, et al. ChatGPT for good? On opportunities and challenges of large language models for education. Learn Individ Differ. Apr 2023;103:102274. [CrossRef]
    15. Stein N, Brooks K. A fully automated conversational artificial intelligence for weight loss: longitudinal observational study among overweight and obese adults. JMIR Diabetes. Nov 01, 2017;2(2):e28. [FREE Full text] [CrossRef] [Medline]
    16. Griffin A, Xing Z, Khairat S, Wang Y, Bailey S, Arguello J, et al. Conversational agents for chronic disease self-management: a systematic review. AMIA Annu Symp Proc. 2020;2020:504-513. [FREE Full text] [Medline]
    17. Yeager CM, Benight CC. If we build it, will they come? Issues of engagement with digital health interventions for trauma recovery. Mhealth. Sep 2018;4:37. [FREE Full text] [CrossRef] [Medline]
    18. Couper MP, Alexander GL, Zhang N, Little RJ, Maddy N, Nowak MA, et al. Engagement and retention: measuring breadth and depth of participant use of an online intervention. J Med Internet Res. Nov 18, 2010;12(4):e52. [FREE Full text] [CrossRef] [Medline]
    19. Dingler T, Kwasnicka D, Wei J, Gong E, Oldenburg B. The use and promise of conversational agents in digital health. Yearb Med Inform. Aug 03, 2021;30(1):191-199. [CrossRef] [Medline]
    20. Guy I. Searching by talking: analysis of voice queries on mobile web search. 2016. Presented at: 39th International ACM SIGIR Conference on Research and Development in Information Retrieval; July 17-21, 2016; Pisa, Italy. [CrossRef]
    21. Tudor Car L, Dhinagaran DA, Kyaw BM, Kowatsch T, Joty S, Theng Y, et al. Conversational agents in health care: scoping review and conceptual analysis. J Med Internet Res. Aug 07, 2020;22(8):e17158. [FREE Full text] [CrossRef] [Medline]
    22. Apergi LA, Bjarnadottir MV, Baras JS, Golden BL, Anderson KM, Chou J, et al. Voice interface technology adoption by patients with heart failure: pilot comparison study. JMIR Mhealth Uhealth. Apr 01, 2021;9(4):e24646. [FREE Full text] [CrossRef] [Medline]
    23. Baptista S, Wadley G, Bird D, Oldenburg B, Speight J, My Diabetes Coach Research Group. Acceptability of an embodied conversational agent for type 2 diabetes self-management education and support via a smartphone app: mixed methods study. JMIR Mhealth Uhealth. Jul 22, 2020;8(7):e17038. [FREE Full text] [CrossRef] [Medline]
    24. Cheng A, Raghavaraju V, Kanugo J, Handrianto Y, Shang Y. Development and evaluation of a healthy coping voice interface application using the Google home for elderly patients with type 2 diabetes. 2018. Presented at: 15th IEEE Annual Consumer Communications & Networking Conference (CCNC); January 12-15, 2018; Las Vegas, NV. [CrossRef]
    25. Epalte K, Tomsone S, Vētra A, Bērziņa G. Patient experience using digital therapy "Vigo" for stroke patient recovery: a qualitative descriptive study. Disabil Rehabil Assist Technol. Feb 06, 2023;18(2):175-184. [CrossRef] [Medline]
    26. Pereira J, Díaz Ó. Using health chatbots for behavior change: a mapping study. J Med Syst. Apr 04, 2019;43(5):135. [CrossRef] [Medline]
    27. Lim SM, Shiau CWC, Cheng LJ, Lau Y. Chatbot-delivered psychotherapy for adults with depressive and anxiety symptoms: a systematic review and meta-regression. Behav Ther. Mar 2022;53(2):334-347. [CrossRef] [Medline]
    28. Naveed H, Khan AU, Qiu S, Saqib M, Anwar S, Usman M, et al. A comprehensive overview of large language models. arXiv. Preprint posted online on April 9, 2024. [FREE Full text]
    29. Brown TB, Mann B, Ryder N, Subbiah M, Kaplan J, Dhariwal P, et al. Language models are few-shot learners. arXiv. Preprint posted online on July 22, 2020. [CrossRef]
    30. Loos E, Gröpler J, Goudeau MS. Using ChatGPT in education: human reflection on ChatGPT’s self-feflection. Societies. Aug 21, 2023;13(8):196. [CrossRef]
    31. Alkaissi H, McFarlane S. Artificial hallucinations in ChatGPT: implications in scientific writing. Cureus. Feb 2023;15(2):e35179. [FREE Full text] [CrossRef] [Medline]
    32. Thirunavukarasu AJ, Ting DSJ, Elangovan K, Gutierrez L, Tan TF, Ting DSW. Large language models in medicine. Nat Med. Aug 17, 2023;29(8):1930-1940. [CrossRef] [Medline]
    33. Abd-Alrazaq AA, Rababeh A, Alajlani M, Bewick BM, Househ M. Effectiveness and safety of using chatbots to improve mental health: systematic review and meta-analysis. J Med Internet Res. Jul 13, 2020;22(7):e16021. [FREE Full text] [CrossRef] [Medline]
    34. Touvron H, Lavril T, Izacard G, Martinet X, Lachaux M, Lacroix T, et al. LLaMA: open and efficient foundation language models. arXiv. Preprint posted online on February 27, 2023. [FREE Full text]
    35. Thirunavukarasu AJ. Large language models will not replace healthcare professionals: curbing popular fears and hype. J R Soc Med. May 18, 2023;116(5):181-182. [FREE Full text] [CrossRef] [Medline]
    36. Li X, Zhang T. An exploration on artificial intelligence application: From security, privacy and ethic perspective. 2017. Presented at: IEEE 2nd International Conference on Cloud Computing and Big Data Analysis (ICCCBDA); April 28-30, 2017; Chengdu, China. [CrossRef]
    37. GPT-4 System Card. OpenAI. Mar 2023. URL: https://cdn.openai.com/papers/gpt-4-system-card.pdf [accessed 2024-07-11]
    38. Liu I, Liu F, Xiao Y, Huang Y, Wu S, Ni S. Investigating the key success factors of chatbot-based positive psychology intervention with retrieval- and generative pre-trained transformer (GPT)-based chatbots. Int J Hum Comput Interact. Jan 08, 2024:1-12. [CrossRef]
    39. Liu H, Peng H, Song X, Xu C, Zhang M. Using AI chatbots to provide self-help depression interventions for university students: a randomized trial of effectiveness. Internet Interv. Mar 2022;27:100495. [FREE Full text] [CrossRef] [Medline]
    40. Adam M, Wessel M, Benlian A. AI-based chatbots in customer service and their effects on user compliance. Electron Markets. Mar 17, 2020;31(2):427-445. [CrossRef]
    41. Wei C, Kim Y, Kuzminykh A. The bot on speaking terms: the effects of conversation architecture on perceptions of conversational agents. 2023. Presented at: CUI '23: 5th International Conference on Conversational User Interfaces; July 19-21, 2023; Eindhoven, Netherlands. [CrossRef]
    42. Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.3. Cochrane Training. 2022. URL: https://training.cochrane.org/handbook [accessed 2023-07-01]
    43. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. PRISMA-P Group. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. Jan 02, 2015;350:g7647. [FREE Full text] [CrossRef] [Medline]
    44. Lockwood C, Munn Z, Porritt K. Qualitative research synthesis: methodological guidance for systematic reviewers utilizing meta-aggregation. Int J Evid Based Healthc. Sep 2015;13(3):179-187. [CrossRef] [Medline]
    45. Tufanaru C, Munn Z, Aromataris E. Systematic reviews of effectiveness. JBI Manual for Evidence Synthesis. URL: https://tinyurl.com/mtj3dk36 [accessed 2024-07-11]
    46. Barker T, Stone J, Sears K, Klugar M, Tufanaru C, Leonardi-Bee J, et al. The revised JBI critical appraisal tool for the assessment of risk of bias for randomized controlled trials. JBI Evid Synth. Mar 01, 2023;21(3):494-506. [CrossRef] [Medline]
    47. Thomas J, Harden A. Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. Jul 10, 2008;8(1):45. [FREE Full text] [CrossRef] [Medline]
    48. Balsa J, Neves P, Félix I, Pereira GM, Alves P, Carmo M. Intelligent virtual assistant for promoting behaviour change in older people with T2D. In: Oliveira M, Novais P, Reis L, editors. Progress in Artificial Intelligence. EPIA 2019. Lecture Notes in Computer Science(), vol 11804. Cham. Springer; 2019:372-383.
    49. Balsa J, Félix I, Cláudio AP, Carmo MB, Silva ICE, Guerreiro A, et al. Usability of an intelligent virtual assistant for promoting behavior change and self-care in older people with type 2 diabetes. J Med Syst. Jun 13, 2020;44(7):130. [CrossRef] [Medline]
    50. Echeazarra L, Pereira J, Saracho R. TensioBot: a chatbot assistant for self-managed in-house blood pressure checking. J Med Syst. Mar 15, 2021;45(4):54. [CrossRef] [Medline]
    51. Gingele AJ, Amin H, Vaassen A, Schnur I, Pearl C, Brunner-La Rocca H, et al. Integrating avatar technology into a telemedicine application in heart failure patients: a pilot study. Wien Klin Wochenschr. Dec 02, 2023;135(23-24):680-684. [FREE Full text] [CrossRef] [Medline]
    52. Gong E, Baptista S, Russell A, Scuffham P, Riddell M, Speight J, et al. My Diabetes Coach, a mobile app-based interactive conversational agent to support type 2 diabetes self-management: randomized effectiveness-implementation trial. J Med Internet Res. Nov 05, 2020;22(11):e20322. [FREE Full text] [CrossRef] [Medline]
    53. Guhl E, Althouse A, Pusateri A, Kimani E, Paasche-Orlow M, Bickmore T, et al. The Atrial Fibrillation Health Literacy Information Technology Trial: pilot trial of a mobile health app for atrial fibrillation. JMIR Cardio. Sep 04, 2020;4(1):e17162. [FREE Full text] [CrossRef] [Medline]
    54. Kimani E, Bickmore T, Trinh H, Ring L, Paasche-Orlow M, Magnani J. A smartphone-based virtual agent for atrial fibrillation education and counseling. In: Traum D, Swartout W, Khooshabeh P, Kopp S, Scherer S, Leuski A, editors. Intelligent Virtual Agents. IVA 2016. Lecture Notes in Computer Science(), vol 10011. Cham. Springer; 2016:120-127.
    55. Magnani J, Schlusser C, Kimani E, Rollman B, Paasche-Orlow M, Bickmore T. The Atrial Fibrillation Health Literacy Information Technology System: pilot assessment. JMIR Cardio. 2017;1(2):e7. [FREE Full text] [CrossRef] [Medline]
    56. Roca S, Lozano M, García J, Alesanco A. Validation of a virtual assistant for improving medication adherence in patients with comorbid type 2 diabetes mellitus and depressive disorder. Int J Environ Res Public Health. Nov 17, 2021;18(22):12056. [FREE Full text] [CrossRef] [Medline]
    57. Sagstad MH, Morken N, Lund A, Dingsør LJ, Nilsen ABV, Sorbye LM. Quantitative user data from a chatbot developed for women with gestational diabetes mellitus: observational study. JMIR Form Res. Apr 18, 2022;6(4):e28091. [FREE Full text] [CrossRef] [Medline]
    58. Ter Stal S, Sloots J, Ramlal A, Op den Akker H, Lenferink A, Tabak M. An embodied conversational agent in an eHealth self-management intervention for chronic obstructive pulmonary disease and chronic heart failure: exploratory study in a real-life setting. JMIR Hum Factors. Nov 04, 2021;8(4):e24110. [FREE Full text] [CrossRef] [Medline]
    59. Tongpeth J, Du H, Clark RA. An avatar-based education application to improve patients' knowledge of and response to heart attack symptoms: A pragmatic randomized controlled trial protocol. J Adv Nurs. Nov 25, 2018;74(11):2658-2666. [CrossRef] [Medline]
    60. Tsai W, Li W, Tsai J, Yang J. Improving self-diet management of chronic kidney disease patients through chatbots. In: Gao Q, Zhou J, editors. Human Aspects of IT for the Aged Population. Technology in Everyday Living. HCII 2022. Lecture Notes in Computer Science, vol 13331. Cham. Springer; 2022:268-276.
    61. Zhang Z, Trinh H, Chen Q, Bickmore T. Adapting a geriatrics health counseling virtual agent for the Chinese culture. In: Brinkman WP, Broekens J, Heylen D, editors. Intelligent Virtual Agents. IVA 2015. Lecture Notes in Computer Science(), vol 9238. Cham. Springer; 2015:275-278.

    ‎
    CMD: cardiometabolic disease
    PRISMA: Preferred Reporting Items for Systematic reviews and Meta-Analyses
    PRISMA-P: Preferred Reporting Items for Systematic reviews and Meta-Analyses Protocols
    PROSPERO: International Prospective Register of Systematic Reviews

    Edited by A Mavragani; submitted 20.09.23; peer-reviewed by M Chatzimina, I Liu, A Mihalache; comments to author 02.12.23; revised version received 21.12.23; accepted 04.03.24; published 07.08.24.

    Copyright

    ©Nick Kashyap, Ann Tresa Sebastian, Chris Lynch, Paul Jansons, Ralph Maddison, Tilman Dingler, Brian Oldenburg. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 07.08.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.