This paper is in the following e-collection/theme issue:

Systematic Review Protocols (244) Artificial Intelligence (1197) Chatbots and Conversational Agents (463) Generative Language Models Including ChatGPT (206) Scientometrics, Infometrics, and Altmetrics (83)

Published on in Vol 13 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/54349, first published .
Implementation of Chatbot Technology in Health Care: Protocol for a Bibliometric Analysis

Implementation of Chatbot Technology in Health Care: Protocol for a Bibliometric Analysis

Implementation of Chatbot Technology in Health Care: Protocol for a Bibliometric Analysis

Authors of this article:

Zhao Ni1, 2 Author Orcid Image ;   Mary L Peng3 Author Orcid Image ;   Vimala Balakrishnan4 Author Orcid Image ;   Vincent Tee5 Author Orcid Image ;   Iskandar Azwa5, 6 Author Orcid Image ;   Rumana Saifi5 Author Orcid Image ;   LaRon E Nelson1, 2 Author Orcid Image ;   David Vlahov1, 2 Author Orcid Image ;   Frederick L Altice2, 5, 7, 8 Author Orcid Image
Article Authors Cited by (1) Tweetations (3) Metrics

    Protocol

    1School of Nursing, Yale University, Orange, CT, United States

    2Center for Interdisciplinary Research on AIDS, Yale University, New Haven, CT, United States

    3Department of Global Health and Social Medicine, Harvard Medical School, Harvard University, Boston, MA, United States

    4Department of Information Systems, Faculty of Computer Science and Information Technology, Unversity of Malaya, Kuala Lumpur, Malaysia

    5Centre of Excellence for Research in AIDS, Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia

    6Infectious Disease Unit, Faculty of Medicine, University of Malaya, Kuala Lumpur, Malaysia

    7Section of Infectious Disease, Department of Internal Medicine, Yale School of Medicine, New Haven, CT, United States

    8Division of Epidemiology of Microbial Diseases, Yale School of Public Health, New Haven, CT, United States

    Corresponding Author:

    Zhao Ni, BMed, PhD

    School of Nursing

    Yale University

    400 West Campus Drive

    Orange, CT, 06477

    United States

    Phone: 1 2037373039

    Email: zhao.ni@yale.edu


    Background: Chatbots have the potential to increase people’s access to quality health care. However, the implementation of chatbot technology in the health care system is unclear due to the scarce analysis of publications on the adoption of chatbot in health and medical settings.

    Objective: This paper presents a protocol of a bibliometric analysis aimed at offering the public insights into the current state and emerging trends in research related to the use of chatbot technology for promoting health.

    Methods: In this bibliometric analysis, we will select published papers from the databases of CINAHL, IEEE Xplore, PubMed, Scopus, and Web of Science that pertain to chatbot technology and its applications in health care. Our search strategy includes keywords such as “chatbot,” “virtual agent,” “virtual assistant,” “conversational agent,” “conversational AI,” “interactive agent,” “health,” and “healthcare.” Five researchers who are AI engineers and clinicians will independently review the titles and abstracts of selected papers to determine their eligibility for a full-text review. The corresponding author (ZN) will serve as a mediator to address any discrepancies and disputes among the 5 reviewers. Our analysis will encompass various publication patterns of chatbot research, including the number of annual publications, their geographic or institutional distribution, and the number of annual grants supporting chatbot research, and further summarize the methodologies used in the development of health-related chatbots, along with their features and applications in health care settings. Software tool VOSViewer (version 1.6.19; Leiden University) will be used to construct and visualize bibliometric networks.

    Results: The preparation for the bibliometric analysis began on December 3, 2021, when the research team started the process of familiarizing themselves with the software tools that may be used in this analysis, VOSViewer and CiteSpace, during which they consulted 3 librarians at the Yale University regarding search terms and tentative results. Tentative searches on the aforementioned databases yielded a total of 2340 papers. The official search phase started on July 27, 2023. Our goal is to complete the screening of papers and the analysis by February 15, 2024.

    Conclusions: Artificial intelligence chatbots, such as ChatGPT (OpenAI Inc), have sparked numerous discussions within the health care industry regarding their impact on human health. Chatbot technology holds substantial promise for advancing health care systems worldwide. However, developing a sophisticated chatbot capable of precise interaction with health care consumers, delivering personalized care, and providing accurate health-related information and knowledge remain considerable challenges. This bibliometric analysis seeks to fill the knowledge gap in the existing literature on health-related chatbots, entailing their applications, the software used in their development, and their preferred functionalities among users.

    International Registered Report Identifier (IRRID): PRR1-10.2196/54349

    JMIR Res Protoc 2024;13:e54349

    doi:10.2196/54349

    Keywords

    artificial intelligence; AI; bibliometric analysis; chatbots; health care; health promotion 


    Chatbots are software applications that use computerized algorithms to simulate conversations with human users through text or voice interactions [1,2]. Since their inception in the 1960s, chatbots have found applications in various settings, such as airlines, banks, hotel chains, and information technology companies, serving as digital agents to handle and streamline customers’ queries and needs [3]. Compared to human agents, chatbots can efficiently respond to a large number of users simultaneously, conserving human effort and time while still providing users with a sense of human interaction [4]. Due to this advantageous feature, chatbots have been implemented in health care settings to automatically resolve or deflect repetitive calls, thereby reducing waiting times for health care consumers and enabling health care professionals to focus on more complex cases. Driven by the evolution of Industry 4.0, characterized by the integration of digital technologies, data, and automation to create more efficient and responsive societal systems, the future of high-quality health care hinges on the automation and digitalization of data exchange. Against this social-technological backdrop, artificial intelligence (AI) chatbots, also known as conversational AI, hold substantial promise as innovative tools for advancing our health care systems [5].

    AI chatbots have been developed to automate and streamline various tasks for health care consumers, including retrieving health information, providing digital health support, and offering therapeutic care [6]. The literature reveals that AI chatbots commonly fulfill roles such as assisting individuals in scheduling medical appointments, identifying health clinics, and providing health educational information [7,8]. Research has also shown that health care professionals, patients, and families exhibit favorable attitudes toward the use of chatbot technology to enhance health outcomes [7,9-12].

    While AI chatbots hold considerable potential to drive significant advancements and improvements in health care [13,14], their application in health care is still in its early stages. A significant barrier to the deployment of chatbot technology in health care systems is the lack of sophisticated AI algorithms capable of facilitating precise and personalized human-chatbot interactions to meet the expectations of health care providers or fulfill the needs of health care consumers [11]. For instance, in the United States, health-related chatbots have been developed to monitor the health status of patients with chronic heart failure [15], screen for osteoporosis in menopausal women [16], and detect colorectal cancer in the general population [17]. However, their effectiveness in clinical trials was found to be limited when compared to health professional assessments. To fully realize the potential of chatbot technology in health care systems, more studies are needed to develop more sophisticated AI algorithms that are culturally tailored, theoretically informed, and trained based on clinical needs [18-21]. Creating such sophisticated AI chatbots presents a challenge for both health scientists and chatbot engineers, necessitating iterative collaboration between the 2 [22]. Specifically, after chatbot engineers develop a chatbot prototype, health scientists evaluate it and provide feedback for further refinement. Chatbot engineers then upgrade the chatbot, followed by health scientists testing the updated version, training it, and conducting further assessments. This iterative cycle can impose significant demands in terms of time and funding before a chatbot is equipped with the necessary knowledge and language skills to deliver precise responses to its users.

    Bibliometric analysis is a quantitative research method to discern publication patterns within a specific timeframe [23]. Scholars use this type of analysis to elucidate the intellectual structure of a particular area within the realm of existing literature [24]. Despite the increasing popularity of health-related chatbots, no bibliometric analysis has been conducted to examine their application. Studies on the coverage of health-related chatbot research have predominantly been conducted in the form of scoping or systematic reviews [19,25,26]. The current body of research papers lacks the breadth of a comprehensive scientific performance mapping analysis. Hence, this bibliometric analysis aims to identify the current status and emerging trends in chatbot technology research, serving as an initial stride for researchers worldwide to gain a comprehensive understanding on the landscape of health-related chatbots. This overview will facilitate the identification of areas for improvement and promote the integration of chatbot technology into health care systems.

    Chatbot technology should be promoted in the health care system because many digital health interventions have proven effective but are not implemented in real clinical settings, as they often require high-intensity and sustained human inputs. For example, they often require researchers to regularly and manually send personalized reminders, provide real-time guidance, and initiate referrals [27,28]. To bring population-level effects, digital health intervention needs to be automating personalized messages, modifying them based on responses, and providing new outputs in real time [29]. AI chatbots have the potential to achieve these goals. For example, our previous formative research indicates a high level of acceptance toward the use of chatbot technology among vulnerable populations who are at high risk for HIV [2]. Additionally, we have conducted beta testing for chatbot technology in promoting HIV testing and prevention and found that participants believed chatbot technology provided them with a platform to protect their safety and privacy. This was particularly important in environments where stigma and discrimination toward HIV exist, and where same-sex behaviors are criminalized. Compared with the conventional health care use model, where people need to face stigma and discrimination from health care providers, chatbots can provide them with a safe platform to ask questions and receive consulting services. Therefore, promoting chatbot technology holds significance for enhancing the current health care system and an anonymous user setting in chatbots is necessary to protect health consumers’ privacy [2].


    Bibliographic Search

    We will conduct searches in the following databases: CINAHL (EBSCOHost), IEEE Xplore (IEEE), PubMed, Scopus (Elsevier), and Web of Science (Clarivate) to ensure a comprehensive coverage of research on health-related chatbots. We have identified a set of consensus search keywords related to chatbots and health care through a review of previous systematic reviews [18-21] and consultations with university librarians who possess expertise in informatics and digital health. Our search strategy includes keywords such as “chatbot,” “virtual agent,” “virtual assistant,” “conversational agent,” “conversational AI,” “interactive agent,” “health,” and “healthcare.” A retrieval search string, including the keywords with Boolean operators listed in Multimedia Appendix 1, will be used.

    Selection of Studies

    We will include papers on chatbots that are used to promote health outcomes. All interventional and observational studies published as journal papers or conference proceedings will be included. To offer a holistic view of the evolving usage of chatbots in health care, we will not set restrictions on the year of publication. Moreover, we will not exclude papers published in non–English language to incorporate research findings from low- and middle-income countries [30]. Studies that do not discuss the use of chatbots to promote health or wellness will be excluded. Systematic reviews pertaining only to chatbot designs and development, purposes, or features will be excluded. Papers such as editorials, dissertations, preprints, and letters to the editor will also be excluded.

    Search results from each database will be imported into Covidence (Veritas Health Innovation Ltd), a systematic review management software. Duplicate papers will be identified and removed. Five researchers will independently screen the titles and abstracts of all papers and categorize them as either “include,” “exclude,” or “unsure” based on the following inclusion criteria related to (1) chatbot and (2) health promotion. To provide a comprehensive overview of the current research on health-related chatbots, we will include papers about chatbots designed for various populations, including patients, clinicians, policy makers, or the general population. We will not exclude papers based on their study design. Papers marked as “unsure” will be downloaded and assessed for eligibility. The eligibility assessment will be performed by 2 authors (VB and VT) who are an AI consultant and a clinician. In the event of disagreements, the 2 authors will discuss in team meetings with the corresponding author (ZN) to reach a consensus.

    Data Extraction

    Selected studies will be downloaded from Covidence and imported into VOSViewer (version 1.6.19; Leiden University), a Java-based bibliometric analysis visualization software application. We will use VOSViewer to analyze data related to chatbot publication patterns, encompassing the number of annual publications, the distribution of countries and institutions involved in chatbot research, the number of annual grants that supported chatbot research, the number of funders, the number of journals publishing chatbot research, the number of journal citations, the most prolific authors in the field of chatbots, author network maps, and the most frequently used keywords related to chatbots. Additionally, using Excel (Microsoft Corp), we will manually extract data regarding the number of interventional and observational studies, the methodologies used in creating chatbots, the number of chatbots deployed worldwide, and the usage of chatbots.

    Data Analysis

    Research Characteristics

    In this bibliometric analysis, we will analyze the characteristics of chatbot research based on the topics of the selected studies, identified through their reported keywords, such as primary functions and disease domains. We will report the frequency and percentage of the top keywords and topics by following the framework in previous research to measure the centrality of a keyword using its frequency scores [31].

    Publication Patterns

    We will report the trend of yearly number of publications and showcase the growth rate of publication by computing the monthly publication rate each year from the earliest publication selected to the latest. To control for the “publication noise,” a surge in publications following the releases of ChatGPT, we will stratify the data by publication date before and after the emergence of ChatGPT and conduct sensitivity analyses to distinguish between the direct impact of ChatGPT and other trends in chatbot research. This approach will help to ensure that our findings reflect the broader trends in chatbot technology research and are not disproportionately influenced by the recent increase in publications related to a single event or development. We will calculate the frequencies and percentages of publications for each journal and country in each publication year, whereby we identify the countries and institutions associated with the publication based on the affiliations of the corresponding authors. We will report the trend of research support by identifying the number of grants and funders as reported under each study’s source of funding. Moreover, we will compute the distribution of the following two indicators including (1) the methodology used to create chatbots and (2) the implementation of health chatbots.

    Research Hot Spot and Connectedness

    We will use the number of journal citations to construct bursts, whereby clusters will be sorted by the keywords used by the study. We will further report the most prolific authors based on a combined metric of the number of publications and citation frequency. We will present 2 author network maps using author names and research institutions affiliated with the listed authors. Authors and affiliated institutions will be the nodes within each network connected by edges representing the coauthorship of publications [32]. Edges will be weighted by the number of coauthorships between the same listed authors and their affiliated institutions.

    Ethical Considerations

    This analysis does not involve recruiting human participants or providing interventions; therefore, ethical review and consent forms are not required. We hope that the findings from the manuscript will aid researchers, engineers, health professionals, funders, and policy makers in their future implementation of chatbot technology to facilitate innovative and efficient health care systems.


    The preparation for the bibliometric analysis began on December 3, 2021, with the initial steps involving the research team familiarizing themselves with VOSViewer and CiteSpace, followed by digital consultations with 3 librarians from Yale University. Tentative searches in the databases yielded a total of 2340 papers. The official search phase began on July 27, 2023. Our goal is to complete the screening of papers and perform the analysis by February 15, 2024. We anticipate a significant increase in chatbot research following the emergence of ChatGPT.


    AI chatbots hold strong potential to transform the field of health care. For example, ChatGPT, an AI chatbot developed by OpenAI, has sparked numerous discussions within the health care industry regarding the impact of AI chatbots on human health [13,14,33-38]. Our team has been developing an AI chatbot since June 2019, and we have discovered that developing a sophisticated AI chatbot capable of precise interaction with health care consumers, delivering personalized care, and providing accurate health-related information and knowledge remains a considerable challenge. One of the major obstacles faced by health scientists interested in chatbot research is their lack of familiarity with chatbots’ underlying technologies, such as the computational systems, software platforms, and underlying algorithms that train chatbots and enable their automation and individualization. Similarly, chatbot engineers may have limited insight into the challenges faced by patients and health care providers in real-world clinical settings, making it difficult for them to fully grasp the nuances and directions of chatbot technology. Such information asymmetry in interdisciplinary collaboration hinders health-advancing chatbot technology from reaching its full potential. More literature that can offer diverse stakeholders, ranging from health scientists, health care providers, to chatbot developers, insights into a holistic roadmap to creating effective chatbots to meet health care needs is thus urgently needed to facilitate such interdisciplinary endeavors.

    Chatbot technology holds immense potential to enhance health care quality for both patients and professionals through streamlining administrative processes and assisting with assessment, diagnosis, and treatment. Used for health information acquisition, chatbot-powered search, as we anticipate, will become an important complement to traditional web-based searches. This trend is primarily driven by the convenience of chatbot-powered search for users, as it eliminates the need for users to manually sift through search results as required in traditional web-based searches. However, no recognized standards or guidelines have been established for creating health-related chatbots. We believe that with theory-informed and well-trained algorithms, chatbots can also be used as health care digital assistants to provide consumers and patients with quick, precise, and individualized answers. As demonstrated by the widespread adoption of ChatGPT in recent times, it is reasonable to anticipate that the development and implementation of AI chatbots will substantially enhance the efficiency and convenience of health care consumers’ information search. Existing studies have also shown that AI chatbots can increase health appointment volume by automating the scheduling process for health care consumers, achieved through a user-friendly platform that enables appointment scheduling, verification, and cancellation with ease, further leading to reduced no-show rates. For example, Weill Cornell Medicine reported a 47% increase in appointments booked digitally through the use of AI chatbots [39]. Moreover, chatbot technology can enhance the work efficiency of nurses, physicians, and other health care professionals by delivering prompt responses to inquiries related to clinical practice standards, eliminating the need to navigate prolonged websites or extensive clinical guidelines for specific information.

    Despite the obvious benefits of chatbot technology in health care, several potential risks of using chatbots exist, including breaching privacy, providing misinformation, and generating systematically biased responses [2,7-9]. These risks are relevant to the nature of chatbot technology, in which chatbot developers need to maximize a personalized experience and enable chatbots to provide users with precision answers through training chatbots [12]. However, training chatbots requires chatbot technology to have access to a wealth of users’ personal data. This poses a major threat to using chatbots in health care. To address privacy issues, chatbot developers and researchers must ensure that users’ data are protected using encryption during human-chatbot interactions or when a chatbot needs to retrieve backend data [2]. Second, misinformation originates from the immature or flaws of the chatbot algorithms. Training a chatbot is an iterative process that demands a large data set and vetting of the outputs by researchers. During a chatbot creation, the earlier versions of the chatbot often provide redundant and impersonalized information that may prevent users from using the chatbot. To increase chatbot usability, a chatbot must be precise enough in its communications with users or can connect users to a human agent if necessary [11,12]. Third, even well-trained chatbots can provide biased responses or solutions to users [13]. This is particularly true if the data sets used to train chatbots are biased. To minimize these risks of using chatbots in health care, it is necessary for researchers to validate chatbot outputs and reduce biases in the data sets used to train a chatbot. Only by adopting this approach, quality chatbots with high usability can be used to promote health care.

    To fully realize the potential of chatbot technology in improving health outcomes for everyone, sustained collaborative efforts from an interdisciplinary research team comprising chatbot engineers and health scientists are essential. To the best of our knowledge, this is the first study aimed at summarizing the current status and future trends of chatbots in the health care field. This study will provide a broad overview of publications on health-related chatbot research and bridge the knowledge gap in the existing literature, including software used for chatbot development, popular functionalities, and chatbots’ applications in health care. This study includes papers published since the inception of the chatbot and is not confined by the language of publication. Consequently, it offers a global perspective on the evolution of chatbots within the health care domain. One limitation of this study is its nature as a bibliometric analysis, which does not explore topics in the same depth as a systematic review.

    Acknowledgments

    This work was supported by grants from the National Institutes of Health (R21TW011663 and R33TW011663 for ZN and FLA; and D43TW011324 for FLA and RS) and faculty research funds from Yale University School of Nursing for ZN.

    We extend our sincere thanks to Janene Batten and other librarians at Yale University for their invaluable assistance and guidance in preparing for this bibliometric analysis

    Data Availability

    Data sharing is not applicable to this article as no data sets were generated or analyzed during this study.

    Authors' Contributions

    ZN, VT, MLP, and VB conceived the study and wrote the first draft of the manuscript. MLP and VB helped to develop the bibliographic search and bibliometric analysis. All authors contributed to the development of the study protocol, revised the subsequent version of the manuscript, and approved the submitted version.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    A retrieval search string.

    DOCX File , 15 KB
    1. Kate B, Scardina J. What is a chatbot? Techtarget. 2019. URL: https://searchcustomerexperience.techtarget.com/definition/chatbot [accessed 2021-09-01]
    2. Peng ML, Wickersham JA, Altice FL, Shrestha R, Azwa I, Zhou X, et al. Formative evaluation of the acceptance of HIV prevention artificial intelligence Chatbots by men who have sex with men in Malaysia: focus group study. JMIR Form Res. 2022;6(10):e42055. [https://formative.jmir.org/2022/10/e42055/] [CrossRef] [Medline]
    3. Coheur L. From Eliza to Siri and beyond. In: Lesot MJ, Vieira S, Reformat MZ, Carvalho JP, Wilbik A, Bouchon-Meunier B, et al, editors. Information Processing and Management of Uncertainty in Knowledge-Based Systems: 18th International Conference, IPMU 2020, Lisbon, Portugal, June 15–19, 2020, Proceedings, Part I, Vol 1237. Cham. Springer International Publishing; 2020;29-41.
    4. Luxton DD. Ethical implications of conversational agents in global public health. Bull World Health Organ. 2020;98(4):285-287. [https://europepmc.org/abstract/MED/32284654] [CrossRef] [Medline]
    5. Milne-Ives M, de Cock C, Lim E, Shehadeh MH, de Pennington N, Mole G, et al. The effectiveness of artificial intelligence conversational agents in health care: systematic review. J Med Internet Res. 2020;22(10):e20346. [https://www.jmir.org/2020/10/e20346/] [CrossRef] [Medline]
    6. Mesko B. The top 12 health Chatbots. The Medical Futurist. 2021. URL: https://medicalfuturist.com/top-12-health-chatbots/ [accessed 2021-11-18]
    7. Palanica A, Flaschner P, Thommandram A, Li M, Fossat Y. Physicians' perceptions of Chatbots in health care: cross-sectional web-based survey. J Med Internet Res. 2019;21(4):e12887. [https://www.jmir.org/2019/4/e12887/] [CrossRef] [Medline]
    8. Koman J, Fauvelle K, Schuck S, Texier N, Mebarki A. Physicians' perceptions of the use of a Chatbot for information seeking: qualitative study. J Med Internet Res. 2020;22(11):e15185. [https://www.jmir.org/2020/11/e15185/] [CrossRef] [Medline]
    9. Abd-Alrazaq AA, Alajlani M, Ali N, Denecke K, Bewick BM, Househ M. Perceptions and opinions of patients about mental health chatbots: scoping review. J Med Internet Res. 2021;23(1):e17828. [https://www.jmir.org/2021/1/e17828/] [CrossRef] [Medline]
    10. Miles O, West R, Nadarzynski T. Health chatbots acceptability moderated by perceived stigma and severity: a cross-sectional survey. Digit Health. 2021;7:1-7. [https://journals.sagepub.com/doi/10.1177/20552076211063012?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub 0pubmed] [CrossRef] [Medline]
    11. Nadarzynski T, Miles O, Cowie A, Ridge D. Acceptability of artificial intelligence (AI)-led chatbot services in healthcare: a mixed-methods study. Digit Health. 2019;5:1-12. [https://journals.sagepub.com/doi/10.1177/2055207619871808?url_ver=Z39.88-2003&rfr_id=ori:rid:crossref.org&rfr_dat=cr_pub 0pubmed] [CrossRef] [Medline]
    12. Oh YJ, Zhang J, Fang ML, Fukuoka Y. A systematic review of artificial intelligence chatbots for promoting physical activity, healthy diet, and weight loss. Int J Behav Nutr Phys Act. 2021;18(1):160. [https://ijbnpa.biomedcentral.com/articles/10.1186/s12966-021-01224-6] [CrossRef] [Medline]
    13. Wang X, Sanders HM, Liu Y, Seang K, Tran BX, Atanasov AG, et al. ChatGPT: promise and challenges for deployment in low- and middle-income countries. Lancet Reg Health West Pac. 2023;41:100905. [https://linkinghub.elsevier.com/retrieve/pii/S2666-6065(23)00223-7] [CrossRef] [Medline]
    14. No Authors. Will ChatGPT transform healthcare? Nat Med. 2023;29(3):505-506. [https://www.nature.com/articles/s41591-023-02289-5] [CrossRef] [Medline]
    15. Galescu L, Allen J, Ferguson G, Quinn J, Swift M. Speech recognition in a dialog system for patient health monitoring. Presented at: 2009 IEEE International Conference on Bioinformatics and Biomedicine Workshop; November 1-4, 2009, 2009; Washington, DC, USA. [CrossRef]
    16. Heyworth L, Kleinman K, Oddleifson S, Bernstein L, Frampton J, Lehrer M, et al. Comparison of interactive voice response, patient mailing, and mailed registry to encourage screening for osteoporosis: a randomized controlled trial. Osteoporos Int. 2014;25(5):1519-1526. [CrossRef] [Medline]
    17. Simon SR, Zhang F, Soumerai SB, Ensroth A, Bernstein L, Fletcher RH, et al. Failure of automated telephone outreach with speech recognition to improve colorectal cancer screening: a randomized controlled trial. Arch Intern Med. 2010;170(3):264-270. [https://jamanetwork.com/journals/jamainternalmedicine/fullarticle/415583] [CrossRef] [Medline]
    18. Hoermann S, McCabe KL, Milne DN, Calvo RA. Application of synchronous text-based dialogue systems in mental health interventions: systematic review. J Med Internet Res. 2017;19(8):e267. [https://www.jmir.org/2017/8/e267/] [CrossRef] [Medline]
    19. de Cock C, Milne-Ives M, van Velthoven MH, Alturkistani A, Lam C, Meinert E. Effectiveness of conversational agents (virtual assistants) in health care: protocol for a systematic review. JMIR Res Protoc. 2020;9(3):e16934. [https://www.researchprotocols.org/2020/3/e16934/] [CrossRef] [Medline]
    20. Geoghegan L, Scarborough A, Wormald JCR, Harrison CJ, Collins D, Gardiner M, et al. Automated conversational agents for post-intervention follow-up: a systematic review. BJS Open. 2021;5(4):zrab070. [https://europepmc.org/abstract/MED/34323916] [CrossRef] [Medline]
    21. Luo TC, Aguilera A, Lyles CR, Figueroa CA. Promoting physical activity through conversational agents: mixed methods systematic review. J Med Internet Res. 2021;23(9):e25486. [https://www.jmir.org/2021/9/e25486/] [CrossRef] [Medline]
    22. Corti K, Gillespie A. A truly human interface: interacting face-to-face with someone whose words are determined by a computer program. Front Psychol. 2015;6:634. [https://europepmc.org/abstract/MED/26042066] [CrossRef] [Medline]
    23. Linnenluecke MK, Marrone M, Singh AK. Conducting systematic literature reviews and bibliometric analyses. Aust J Manag. 2019;45(2):175-194. [https://journals.sagepub.com/doi/10.1177/0312896219877678] [CrossRef]
    24. Naveen D, Satish K, Debmalya M, Nitesh P, Lim WM. How to conduct a bibliometric analysis: an overview and guidelines. J Bus Res. 2021;133:285-296. [https://doi.org/10.1016/j.jbusres.2021.04.070] [CrossRef]
    25. Car LT, Dhinagaran DA, Kyaw BM, Kowatsch T, Joty S, Theng YL, et al. Conversational agents in health care: scoping review and conceptual analysis. J Med Internet Res. 2020;22(8):e17158. [https://www.jmir.org/2020/8/e17158/] [CrossRef] [Medline]
    26. Gaffney H, Mansell W, Tai S. Conversational agents in the treatment of mental health problems: mixed-method systematic review. JMIR Ment Health. 2019;6(10):e14166. [https://mental.jmir.org/2019/10/e14166/] [CrossRef] [Medline]
    27. Ni Z, Liu C, Wu B, Yang Q, Douglas C, Shaw RJ. An mHealth intervention to improve medication adherence among patients with coronary heart disease in China: development of an intervention. Int J Nurs Sci. 2018;5(4):322-330. [https://linkinghub.elsevier.com/retrieve/pii/S2352-0132(18)30083-8] [CrossRef] [Medline]
    28. Ni Z, Wu B, Yang Q, Yan LL, Liu C, Shaw RJ. An mHealth intervention to improve medication adherence and health outcomes among patients with coronary heart disease: randomized controlled trial. J Med Internet Res. 2022;24(3):e27202. [https://www.jmir.org/2022/3/e27202/] [CrossRef] [Medline]
    29. Holloway IW, Winder TJ, Lea CH, Tan D, Boyd D, Novak D. Technology use and preferences for mobile phone-based HIV prevention and treatment among Black young men who have sex with men: exploratory research. JMIR Mhealth Uhealth. 2017;5(4):e46. [https://mhealth.jmir.org/2017/4/e46/] [CrossRef] [Medline]
    30. Walpole SC. Including papers in languages other than english in systematic reviews: important, feasible, yet often omitted. J Clin Epidemiol. 2019;111:127-134. [CrossRef] [Medline]
    31. Vega-Oliveros DA, Gomes PS, Milios EE, Berton L. A multi-centrality index for graph-based keyword extraction. Inform Process Manag. 2019;56(6):102063. [CrossRef]
    32. Bender ME, Edwards S, von Philipsborn P, Steinbeis F, Keil T, Tinnemann P. Using co-authorship networks to map and analyse global neglected tropical disease research with an affiliation to Germany. PLoS Negl Trop Dis. 2015;9(12):e0004182. [https://journals.plos.org/plosntds/article?id=10.1371/journal.pntd.0004182] [CrossRef] [Medline]
    33. Sallam M. ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns. Healthcare (Basel). 2023;11(6):887. [https://www.mdpi.com/resolver?pii=healthcare11060887] [CrossRef] [Medline]
    34. Cascella M, Montomoli J, Bellini V, Bignami E. Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios. J Med Syst. 2023;47(1):33. [https://europepmc.org/abstract/MED/36869927] [CrossRef] [Medline]
    35. Sajjad M, Saleem R. Evolution of healthcare with ChatGPT: a word of caution. Ann Biomed Eng. 2023;51(8):1663-1664. [CrossRef] [Medline]
    36. Zheng Y, Wang L, Feng B, Zhao A, Wu Y. Innovating healthcare: the role of ChatGPT in streamlining hospital workflow in the future. Ann Biomed Eng. Preprint posted online on July 18, 2023. [https://link.springer.com/article/10.1007/s10439-023-03323-w#citeas] [CrossRef] [Medline]
    37. Howard A, Hope W, Gerada A. ChatGPT and antimicrobial advice: the end of the consulting infection doctor? Lancet Infect Dis. 2023;23(4):405-406. [https://www.thelancet.com/journals/laninf/article/PIIS1473-3099(23)00113-5/fulltext] [CrossRef] [Medline]
    38. Li H, Moon JT, Purkayastha S, Celi LA, Trivedi H, Gichoya JW. Ethics of large language models in medicine and medical research. Lancet Digit Health. 2023;5(6):e333-e335. [https://linkinghub.elsevier.com/retrieve/pii/S2589-7500(23)00083-3] [CrossRef] [Medline]
    39. Feng H, Zou Z, Zhang Q, Wang L, Ouyang YQ, Chen Z, et al. The effect of the group-based Otago exercise program on frailty among nursing home older adults with cognitive impairment. Geriatr Nurs. 2021;42(2):479-483. [CrossRef] [Medline]

    ‎
    AI: artificial intelligence

    Edited by A Mavragani; submitted 07.11.23; peer-reviewed by J Wang, A Gerada; comments to author 01.12.23; revised version received 07.12.23; accepted 16.01.24; published 15.02.24

    Copyright

    ©Zhao Ni, Mary L Peng, Vimala Balakrishnan, Vincent Tee, Iskandar Azwa, Rumana Saifi, LaRon E Nelson, David Vlahov, Frederick L Altice. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 15.02.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.