Evaluating an Innovative HIV Self-Testing Service With Web-Based, Real-Time Counseling Provided by an Artificial Intelligence Chatbot (HIVST-Chatbot) in Increasing HIV Self-Testing Use Among Chinese Men Who Have Sex With Men: Protocol for a Noninferiority Randomized Controlled Trial

Introduction

Background

Globally, the HIV epidemic among men who have sex with men (MSM) remains out of control [1]. High coverage of HIV testing among the at-risk population is the first and crucial step to achieving the 90-90-90 targets set by the joint United Nations Program on HIV/AIDS [2]. Therefore, international health authorities recommend that MSM take up HIV testing every 6 months [3,4]. However, despite the high risk of HIV infection among MSM in the Hong Kong Special Administrative Region of China (HIV prevalence: 6.54% in 2017), their HIV testing rate remained inadequate (65% in the past year) [5].

HIV self-testing (HIVST) could remove the obstacles of HIV testing faced by MSM, such as inconvenience and perceived stigma from service providers [6]. A systematic review and meta-analysis suggested that HIVST could significantly increase the frequency and coverage of HIV testing among MSM [7]. Therefore, the World Health Organization (WHO) strongly recommends that HIVST should be offered as an additional approach to existing HIV testing services [8]. MSM in Hong Kong mainly relied on facility-based HIV testing and counseling provided by community-based organizations (CBO) and governmental clinics [9]. During the COVID-19 pandemic, facility-based HIV testing services provided by CBOs and governmental clinics were suspended or closed in Hong Kong [10]. However, the availability of HIVST kits in Hong Kong was not affected by the pandemic. An increasing demand for HIVST among MSM was observed in Hong Kong [11].

Pretest and posttest counseling is essential for HIVST, facilitating linkage to care, and behavioral change among HIVST users. In standard-of-care HIV testing and counseling, the pretest counseling involves assessing the individual’s risks, providing information about HIV testing, asserting the user’s right to refuse the test, and informing the user about the possibility of beneficial disclosure of serostatus status, as well as providing preventive information and material [12]. Posttest counseling entails interpreting testing results, offering psychological support to individuals with positive results, facilitating beneficial disclosure of positive serostatus, and referring individuals for further care, treatment, and support services. Additionally, preventive information and material are provided for those with negative test results [12]. Despite the advantages of HIVST, HIVST users may skip pretest or posttest counseling. There are concerns about incorrect procedures, inadequate support for users, and potential issues related to the linkage of care [13]. A meta-analysis study examined that HIVST reduced linkage to care by 47% comparing facility-based HIV testing due to the lack of counseling support [14].

In previous projects, an evidence-based HIVST service was developed for MSM in Hong Kong, which included web-based promotion of HIVST, delivery of free HIVST kits, and provision of web-based, real-time instruction and standard-of-care pretest and posttest counseling through live chat application (HIVST-OIC) [15]. The HIVST-OIC is highly effective in increasing HIVST uptake and the proportion of HIVST users receiving counseling along with testing. A randomized controlled trial showed that after being offered HIVST-OIC, 87.9% of participants took up HIVST, and all HIVST users received counseling. Moreover, the HIVST-OIC was effective in reducing sexual risk behaviors among users. The Centers for Disease Control and Prevention listed HIVST-OIC as an evidence-based intervention and best practice [16]. However, there are issues during the implementation, as HIVST-OIC requires intensive resources. Therefore, it is difficult to increase the service capacity of HIVST-OIC, resulting in a significant gap in implementation.

A chatbot is a computerized program that can automatically select and provide different paths of intervention according to participants’ responses. The chatbot can offer personalized, engaging, and on-demand health communication. With the advancement in artificial intelligence, a chatbot can learn from previous human-machine interactions to increase the accuracy and quality of future interactions [17]. A recent systematic review demonstrated the feasibility and effectiveness of chatbot in promoting healthy lifestyles, smoking cessation, treatment adherence, and substance misuse [18]. The chatbot is potentially useful for delivering real-time counseling to support HIVST users. To our knowledge, this approach is novel.

Objectives

This study aims to demonstrate whether an innovative HIVST service with web-based real-time instruction and counseling provided by a fully automated chatbot (HIVST-chatbot) is as efficacious, if not more efficacious, as HIVST-OIC in increasing HIVST uptake and the proportion of HIVST users receiving counseling alongside testing among MSM in Hong Kong.

The primary objective is to establish whether HIVST-chatbot would produce effects similar to HIVST-OIC in increasing HIVST uptake and the proportion of HIVST users receiving counseling alongside testing within a 6-month follow-up period. We hypothesize that HIVST-chatbot would be statistically noninferior to HIVST-OIC in increasing HIVST uptake and the proportion of HIVST users receiving counseling alongside testing. The noninferiority margin is set as 10% (HIVST-chatbot is no worse than 10% less than that of the HIVST-OIC). The secondary objectives are to compare HIVST-chatbot with HIVST-OIC on the prevalence of condomless anal intercourse, multiple male sex partnerships, and uptake of HIV testing other than HIVST at month 6.

Methods

Study Design

A parallel-group, noninferiority randomized controlled trial will be conducted. A total of 528 participants are randomized 1:1 to either the intervention group (n=264) or the control group (n=264). We will promote and implement HIVST-chatbot and HIVST-OIC in the intervention group and control group, respectively. A telephone follow-up evaluation will be conducted 6 months after the baseline survey by blinded interviewers. The study was registered at ClinicalTrials.gov (NCT05796622). A flowchart diagram is shown in Figure 1.

‎

Participants and Recruitments

Inclusion criteria are (1) Hong Kong–based Chinese-speaking men aged at least 18 years; (2) anal intercourse with at least 1 man within the past 6 months; (3) willingness to provide contact information for follow-up at month 6; (4) having access to WhatsApp; and (5) not having the intention to leave Hong Kong for 1 month consecutively in the next 6 months. Those who have been diagnosed as HIV-positive will be excluded.

Participants will be recruited through multiple sources. Upon obtaining the approval of the owners, trained and experienced fieldworkers will approach prospective MSM participants in gay bars and saunas at different time slots during weekdays and weekends. They will brief prospective participants about the study details and give them an information sheet. The research team will also conduct web-based outreach by posting study information periodically as discussion topics on gay websites with the highest traffic in Hong Kong. If prospective participants are interested in this study, they can contact the research team through private messaging, WhatsApp, telephone, or email. Recruitment will be supplemented by peer referrals.

Ethics Approval

Participants will be guaranteed anonymity during the study, the right to quit at any time, and that refusal will have no consequences. Verbal instead of written informed consent will be obtained to allow participants to maintain anonymity. Field-workers will sign a form pledging that the participants have been fully informed about the study. Similar methods to obtain informed consent have been commonly used in interventional studies targeting MSM in Hong Kong [15,19-21]. Multiple forms of contact information will be obtained to make an appointment for a baseline telephone survey, which takes about 20 minutes to complete. Another telephone follow-up survey will be conducted 6 months after the completion of the baseline survey. A supermarket coupon (HK $50 [US $6.38]) will be mailed to an address provided by the participants upon completion of each survey. Ethics approval was obtained from the Survey and Behavioral Research Ethics Committee of the Chinese University of Hong Kong (SBRE-22-0488).

Sample Size Planning

The target sample size is 528 (n=264 per group) participants. Our previous study showed that about 90% of MSM used HIVST after exposure to HIVST-OIC promotion and implementation [15]. We conservatively assume that 80% of participants in the control group will take up HIVST during the project period. A noninferiority margin of between-group difference of this primary outcome is set at 10% (no worse than 10% less than that of the control group). A sample size of 198 per group will be sufficient to confer 80% power at the 1-sided significance level of .05 (PASS 11.0). Given the 25% dropout rate during the follow-up period [22], 264 participants per group (528 total) will be needed for the study.

The Baseline Survey and Random Allocation Process

Participants will be randomized evenly to either the intervention or control groups. Computer-generated random allocation codes will be produced and sealed in opaque envelopes by a research staff member without involvement in recruitment or baseline survey. One envelope will be drawn and opened by the fieldworkers. They will then inform the participant which group he is assigned to. Block randomization with a block size of 8 will be used.

The Control Group

Web-Based Promotion of HIVST-OIC

Participants will watch a 7-minute web-based video promoting HIV testing in general and HIVST-OIC developed by previous projects [15,20]. The web-based video is based on the constructs of perceived benefits and perceived barriers of the Health Belief Model [23]. The video has 2 parts. The first part is to promote HIV testing in general. A local MSM introduces the potential benefits of taking up HIV testing in general (eg, help to detect HIV infection earlier, increase trust between sex partners, and reduce psychological burdens) and the WHO recommendation for MSM with risk behaviors to take up HIV testing every 6 months. In the second part, the same MSM narratively discusses the benefits and barriers of HIVST-OIC, demonstrates its procedures, and emphasizes the availability of real-time web-based support.

Signing up for HIVST-OIC

After completing watching the web-based video, participants are asked to select their preferable HIVST kits (oral fluid-based or blood-based), choose a service model (comprehensive version with real-time instruction and pretest and posttest counseling or a simplified version with real-time posttest counseling only), and fill in their contact information and address to receive the HIVST kits. Participants can receive a free HIVST kit through rapid courier service, mail it in a plain envelope, or pick it up at collaborative CBO or the research office. Participants can make an appointment with trained HIV testing administrators to implement HIVST-OIC after receiving the HIVST kit. OraQuick in-home test kit (OraSure Technologies, Inc.; sensitivity: 91.7%; specificity: 99.9%) and BioSure HIV self-test kit (BioSure, Inc.; sensitivity: 99.7%; specificity: 99.9%) will be used in this study.

Implementation of HIVST-OIC

The implementation will follow the same procedures reported in previous studies [15,20]. Through live-chat applications (eg, Line, WhatsApp, and Skype), an experienced HIV testing administrator will guide the participants through self-test procedures on-screen and in real-time. Participants are guaranteed anonymity, and no taping will be made. Users may elect to refrain from displaying their faces on the screen. Standard-of-care pretest and posttest counseling will be provided if the user chooses the comprehensive version. The administrator will also explain how to use the HIVST kits and send the user a demonstration video if needed. The entire procedure of the comprehensive version takes about 60 minutes to complete, which is comparable to the time required for facility-based HIV testing and counseling. If the user prefers a simplified version, pretest counseling will be replaced by web-based pretest information (eg, window period of HIVST and types of sexual behaviors with a high risk of HIV transmission). Users can access the demonstration video and standard-of-care, real-time counseling. The simplified version will take about 20 minutes to complete. Users screened to be positive will be given immediate psychological support and explained about the need to take up free confirmatory HIV antibody testing offered by the Department of Health. If necessary, the collaborative CBO staff member will accompany them to the CBO or Department of Health.

The Intervention Group

Web-Based Promotion of HIVST-Chatbot

Participants in the intervention group will watch a web-based video promoting HIV testing in general and HIVST-chatbot. The first part of the web-based video promoting HIV testing generally is the same as the one for the control group. In the second part, the same MSM narratively discusses the advantages of HIVST-chatbot (no need to wait for an appointment, operates 24 hours a day, and artificial intelligence) and the availability of real-time support from chatbot and supporting staff (if users receive positive screening results). The video will also demonstrate the procedures of the HIVST-chatbot. Project staff will help the participants connect to the chatbot. Participants are guaranteed anonymity during the implementation.

Signing up for HIVST-Chatbot

Participants will receive a free HIVST kit in the same manner. The chatbot will contact the participants to confirm receipt of the kits. Participants can get the chatbot to implement HIVST without a previous appointment.

Implementation of HIVST-Chatbot

Pretest Counseling

When users contact the chatbot for implementation, the chatbot will provide standard-of-care pretest and posttest counseling covering key components listed in the Quality Assurance Guideline on HIV Testing and Counseling Services in the format of text messages or videos [24]. The workflow of the HIVST-chatbot is presented in Figure 2.

‎

1. Risk assessment: The chatbot will ask some standard questions to assess users’ HIV risk (eg, injective drug use, use of condoms and alcohol or drug during sexual behaviors, and use of preexposure prophylaxis) and show the response categories. After users complete the questions, the chatbot will interpret their responses and define their risk level. The questions and definition of risk level are identical to those used by facility-based HIV testing and counseling services in Hong Kong (Multimedia Appendix 1).
2. Providing health education advice tailored to risk level: The chatbot will inform users about their risk level and explain the rationale. For high-risk users, the chatbot will prepare them for potential positive results. It will explain the implication of positive screening results and follow-up procedures and emphasize that the staff of collaborative CBO will provide immediate support for users receiving positive testing results from 10 AM to 9 PM (Monday to Friday). These users are advised to perform HIVST within such a time frame. For users without high risk, advice on maintaining safe sex practices is provided. Users can skip parts (1) and (2). If they prefer to skip these parts, standard pretest information will be sent to them (eg, window period of HIVST or types of sexual behaviors with a high risk of HIV transmission).
3. Addressing concerns and obtaining consent for HIVST: Users can choose one or more references from a list of common concerns identified by literature review and interviews of local MSM and service providers. In case their problems are not included in the list, the participant can raise them verbally or in written text. The chatbot will interpret users’ input, extract keywords, retrieve relevant information from the knowledge graph, and prepare a response in the text to the participants. Ultimately, the chatbot will ask whether users are ready to perform HIVST.

Instruction on How to Use the HIVST Kit

The chatbot will play a prerecorded web-based demonstration video showing the procedures of using an oral fluid-based or blood-based HIVST kit. Users can watch the demonstration video repeatedly. They can also skip it if they are familiar with the procedures.

Reporting Testing Results

The chatbot will show examples of positive, negative, and invalid results to help the users interpret them. Users will report their results by answering a multiple-choice question.

The chatbot will ask the users to take a photo of their testing result and upload it to a secure web page for verification.

Posttest Counseling

For users receiving negative testing results, the chatbot will explain that the user is not infected unless the test was taken within the window period, remind them about the risk of HIV infection, and provide advice for safe sex practices. Ultimately, the chatbot will emphasize the need for regular HIV testing.

For users receiving positive testing results, the chatbot will automatically notify a designated supporting staff from the collaborative CBO after obtaining users’ consent. The supporting staff will follow up with the user as soon as possible. The chatbot will explain (1) this is a preliminary positive test result, (2) further confirmatory testing in the Department of Health is required, (3) a staff member from collaborative CBO is already notified and will follow up with them as soon as possible, and (4) emphasize that HIV treatment is very effective. Hotline numbers for 24-hour emotional support and the collaborative CBO will also be provided. Our project staff may accompany them to visit the collaborative CBO or the Department of Health if desired.

Development of the HIVST-Chatbot

The Architecture of the Chatbot System

We adopted the WhatsApp platform to implement the chatbot. The chatbot is integrated with WhatsApp through its public Web API services. Participants’ messages are sent to WhatsApp’s instant messaging server and a separately constructed chatbot system (an administrative system and the chatbot). The chatbot system processes and sends a message back to the WhatsApp instant messaging server. Finally, the users can view the message sent by the chatbot. The entire process spans less than a second without a sluggish feel for users. The architecture of the chatbot system is presented in Figure 3.

‎

The chatbot system contains 3 modules:

1. Dialogue management module: The dialogue system records all conversations between users and the chatbot. The system can extract context information, such as basic statistics of users’ activities and previous interactions between a user and the chatbot. The natural language processing module analyzes the text content of each message. Then, the message is forwarded to trigger specific actions based on preprogrammed rules. For example, when a user sends a message containing a specific keyword, the chatbot immediately responds with information that corresponds to that keyword. In addition, the module initiates new conversations based on the preprogrammed intervention plan.
2. User management module: This module records detailed information about user conversations. Administrators can link users’ WhatsApp numbers with the chatbot and review the progress of the intervention delivery (eg, number of completed sessions or disconnection between users and the chatbot).
3. Multimedia management module: The chatbot system supports image uploading, sending, and updating through this module to allow image and video exchange.

Preparation of Knowledge Graph

To build a system that addresses various personal and relevant concerns related to HIV, HIV testing, and HIVST, we created a knowledge graph linking questions and responses. We conducted in-depth interviews with 10 local MSM to crowdsource concerns related to HIV, HIV testing, and HIVST. We also interviewed experienced HIV testing administrators, CBO workers, and experts in HIV prevention to obtain responses addressing these concerns. The knowledge graph will be used to set rules for the chatbot and train it to get familiar with and identify commonly encountered concerns or misinformation and how to address them through machine learning. The knowledge graph will evolve with more previous interactions during the intervention period.

Conversation Mechanism

1. The chatbot uses a mixture of techniques so that a dialogue is not merely rule-based, but it can also learn from a big pool of previous conversation patterns through machine learning. The chatbot resolves the recent conversation to create a plan for a suitable response. Relevant information is retrieved and used for the preparation of a reply. A persuasion template is selected in accordance with the current stance of the conversation and the perceived user characteristics by the chatbot. The pool of previous conversation patterns will grow bigger after extended usage, and the variety of responses will increase accordingly.
2. In case there is no relevant response to participants’ input, the chatbot will respond by saying, “I am not confident about an answer to your question.” The chatbot will suggest a new conversation topic after 3 off-topic inputs from the user.
3. In case the participants want to know more about 1 topic. The chatbot will respond differently within the knowledge graph to the same topic. Upon request, the chatbot can also repeat the response.
4. In case the conversation between the chatbot and the participants is interrupted. The chatbot will cancel participants’ incomplete input and invite them to input again.
5. The chatbot will document previous conversations with the users and avoid responding repetitively to users’ questions.

Pilot Testing and Refinement

We purposely recruited 30 MSM who were HIV-negative or unknown serostatus to use the HIVST-chatbot. With informed consent, the research team retrieved and reviewed users’ interactions with the chatbot. Feedback was also collected from these users. The pilot study showed that the chatbot was running smoothly. Users were satisfied with the performance and interface of the chatbot.

Measurements

Primary Outcomes

The primary outcomes of this study are self-reported uptake of any HIVST at month 6 and the proportion of HIVST users self-reported having received counseling along with HIVST during the follow-up period. The chatbot will accurately document the number of HIVST-chatbot use, while HIV testing administrators will record the number of HIVST-OIC performed.

Secondary Outcomes

The secondary outcomes of this study are as follows:

1. Condomless anal intercourse with men and multiple male sex partnerships in the past 6 months measured at month 6.
2. Uptake of other types of HIV testing in the past 6 months measured at month 6. They are (1) HIV testing at CBOs in Hong Kong, (2) HIV testing at public hospitals or clinics in Hong Kong, (3) HIV testing at private hospitals or clinics in Hong Kong, (4) HIV testing at other organizations in Hong Kong, and (5) HIV testing in places other than Hong Kong.

Baseline Background Characteristics

Information on social demographics (eg, age, current relationship status, education level, and income), sexual orientation, history of HIV testing, COVID-19, and other sexually transmitted infections will be collected. In addition, participants are asked to report the use of preexposure prophylaxis and different HIV or sexually transmitted infection prevention services (receiving free condoms, peer education and pamphlets, and attending workshops).

Process Evaluation

At month 6, participants will be asked about their level of satisfaction with the promotion and implementation of HIVST-chatbot and HIVST-OIC.

Statistical Analysis

Intention-to-treat analysis will be performed. Multiple imputations will handle missing values at month 6. Markov Chain Monte Carlo method will be used to impute data with an arbitrary pattern of missing values, while monotone methods will be used to impute data having a monotone pattern of missing values. Predictors included baseline background characteristics and baseline values of these outcomes. The relative risk reduction, absolute risk reduction, and number needed to treat, and their 95% CIs will be calculated using Excel. The chi-square tests will inspect between-group balances of potential confounders at baseline. Adjustments will be made if any potential confounders show P<.05 in the comparisons. SPSS 26.0 for Windows (SPSS Inc) will be used for all analyses; statistical significance is set at the P<.05 level (1-sided).

Results

Study recruitment started in April 2023, and data collection is expected to be concluded in January 2024. No results are available as of manuscript preparation.

Discussion

The COVID-19 pandemic and its control measures (eg, lockdown, closure of business, and physical distancing) have direct and negative impacts on HIV testing service use among MSM, especially for facility-based HIV testing services [25-28]. Across countries, health authorities have been actively promoting HIVST to mitigate such negative impacts [29-31]. As a result, there was an increase in HIVST use among MSM [11]. Such growth may continue even in the postpandemic era. Therefore, ensuring sufficient support and linkage to care for the increasing number of MSM, HIVST users becomes more important. Although evidence-based HIVST services existed (eg, HIVST-OIC), most required intensive resources to implement and maintain. Such constraints limited their public health impacts. To our knowledge, this is the first attempt to apply an artificial intelligence chatbot to deliver real-time instruction and counseling support to HIVST users. This randomized controlled trial will compare the efficacy of the HIVST-chatbot with an evidence-based HIVST service, the HIVST-OIC, in increasing HIVST uptake and the proportion of HIVST users receiving counseling along with the testing. If HIVST-chatbot is proven noninferior to HIVST-OIC in improving HIVST uptake and coverage of counseling among users, it may address the existing HIVST service gaps. Since HIVST-chatbot is fully-automated and requires fewer resources to implement and maintain, it can be promoted and implemented by governmental organizations and CBOs in Hong Kong as an alternative approach to HIVST-OIC. The HIVST-chatbot has the potential to overcome the barriers to using HIVST (eg, concerns about lack of support and missing linkage to care), improve overall HIV testing coverage, and reduce the HIV burden among MSM.

Although the trial will yield essential insights for HIVST program planning and policy making, it has some limitations. First, the use of HIV testing other than HIVST-chatbot or HIVST-OIC is self-reported. Such responses may be over-reported due to social desirability. Second, similar to most interventional studies [15,19-21], participants will be recruited through convenient sampling. Cautions should be taken when generalizing the findings to MSM in Hong Kong. Third, collecting characteristics of MSM who refuse to participate in the study is difficult. Selection bias may exist.

In conclusion, this study will generate important research and policy implications regarding using chatbot in HIVST services. If HIVST-chatbot is proven noninferior to HIVST-OIC, it can be easily integrated into existing HIVST services in Hong Kong, given its relatively low resource requirement for implementation and maintenance. HIVST-chatbot holds the potential to overcome the barriers to using HIVST. Therefore, the coverage of HIV testing, the level of support, and the linkage to care for MSM HIVST users will be increased.