This paper is in the following e-collection/theme issue:

Non-Randomized Studies (funded, eHealth) (101) Web-based and Mobile Health Interventions (3092) Smoking Cessation (514) Prevention and Health Promotion (879) Substance Abuse (379) Generative Language Models Including ChatGPT (201) Chatbots and Conversational Agents (458)

Published on in Vol 12 (2023)

This is a member publication of University of Toronto

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/53556, first published .
AI Conversational Agent to Improve Varenicline Adherence: Protocol for a Mixed Methods Feasibility Study

AI Conversational Agent to Improve Varenicline Adherence: Protocol for a Mixed Methods Feasibility Study

AI Conversational Agent to Improve Varenicline Adherence: Protocol for a Mixed Methods Feasibility Study

Authors of this article:

Nadia Minian1, 2, 3, 4, 5 Author Orcid Image ;   Kamna Mehra1 Author Orcid Image ;   Mackenzie Earle1 Author Orcid Image ;   Sowsan Hafuth1 Author Orcid Image ;   Ryan Ting-A-Kee1 Author Orcid Image ;   Jonathan Rose1, 6 Author Orcid Image ;   Scott Veldhuizen1, 2 Author Orcid Image ;   Laurie Zawertailo1, 3, 4 Author Orcid Image ;   Matt Ratto7, 8 Author Orcid Image ;   Osnat C Melamed1, 2, 3, 5 Author Orcid Image ;   Peter Selby1, 2, 3, 9, 10 Author Orcid Image
Article Authors Cited by (1) Tweetations (9) Metrics

    Protocol

    1INTREPID Lab, Centre for Addiction and Mental Health, Toronto, ON, Canada

    2Department of Family and Community Medicine, University of Toronto, Toronto, ON, Canada

    3Campbell Family Mental Health Research Institute, Centre for Addiction and Mental Health, Toronto, ON, Canada

    4Department of Pharmacology and Toxicology, University of Toronto, Toronto, ON, Canada

    5Institute of Medical Sciences, University of Toronto, Toronto, ON, Canada

    6Edward S Rogers Sr Department of Electrical and Computer Engineering, University of Toronto, Toronto, ON, Canada

    7Faculty of Information, University of Toronto, Toronto, ON, Canada

    8Schwartz Reisman Institute for Technology and Society, University of Toronto, Toronto, ON, Canada

    9Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada

    10Department of Psychiatry, University of Toronto, Toronto, ON, Canada

    Corresponding Author:

    Nadia Minian, PhD

    INTREPID Lab

    Centre for Addiction and Mental Health

    1025 Queen Street West

    Toronto, ON, M6J1H1

    Canada

    Phone: 1 4165358501 ext 77420

    Email: nadia.minian2@camh.ca


    Background: Varenicline is a pharmacological intervention for tobacco dependence that is safe and effective in facilitating smoking cessation. Enhanced adherence to varenicline augments the probability of prolonged smoking abstinence. However, research has shown that one-third of people who use varenicline are nonadherent by the second week. There is evidence showing that behavioral support helps with medication adherence. We have designed an artificial intelligence (AI) conversational agent or health bot, called “ChatV,” based on evidence of what works as well as what varenicline is, that can provide these supports. ChatV is an evidence-based, patient- and health care provider–informed health bot to improve adherence to varenicline. ChatV has been programmed to provide medication reminders, answer questions about varenicline and smoking cessation, and track medication intake and the number of cigarettes.

    Objective: This study aims to explore the feasibility of the ChatV health bot, to examine if it is used as intended, and to determine the appropriateness of proceeding with a randomized controlled trial.

    Methods: We will conduct a mixed methods feasibility study where we will pilot-test ChatV with 40 participants. Participants will be provided with a standard 12-week varenicline regimen and access to ChatV. Passive data collection will include adoption measures (how often participants use the chatbot, what features they used, when did they use it, etc). In addition, participants will complete questionnaires (at 1, 4, 8, and 12 weeks) assessing self-reported smoking status and varenicline adherence, as well as questions regarding the acceptability, appropriateness, and usability of the chatbot, and participate in an interview assessing acceptability, appropriateness, fidelity, and adoption. We will use “stop, amend, and go” progression criteria for pilot studies to decide if a randomized controlled trial is a reasonable next step and what modifications are required. A health equity lens will be adopted during participant recruitment and data analysis to understand and address the differences in uptake and use of this digital health solution among diverse sociodemographic groups. The taxonomy of implementation outcomes will be used to assess feasibility, that is, acceptability, appropriateness, fidelity, adoption, and usability. In addition, medication adherence and smoking cessation will be measured to assess the preliminary treatment effect. Interview data will be analyzed using the framework analysis method.

    Results: Participant enrollment for the study will begin in January 2024.

    Conclusions: By using predetermined progression criteria, the results of this preliminary study will inform the determination of whether to advance toward a larger randomized controlled trial to test the effectiveness of the health bot. Additionally, this study will explore the acceptability, appropriateness, fidelity, adoption, and usability of the health bot. These insights will be instrumental in refining the intervention and the health bot.

    Trial Registration: ClinicalTrials.gov NCT05997901; https://classic.clinicaltrials.gov/ct2/show/NCT05997901

    International Registered Report Identifier (IRRID): PRR1-10.2196/53556

    JMIR Res Protoc 2023;12:e53556

    doi:10.2196/53556

    Keywords

    evaluation; health bot; medication adherence; smoking cessation; varenicline; artificial intelligence; AI 


    Medication adherence presents a significant and pressing challenge worldwide across health care systems [1]. This problem spans across various medical conditions, from chronic illnesses such as hypertension, diabetes, and cardiovascular disease to acute infections [2]. Nonadherence to medication regimens can lead to a cascade of problems, including decreased treatment effectiveness, a higher risk of disease progression or complications, medication resistance, increased health care costs, and a reduced overall quality of life for patients [3]. It also places an additional burden on health care providers, who must constantly navigate the complexities of patient adherence to optimize treatment outcomes. Addressing medication adherence is not only a matter of improving individual health but also a vital step toward more efficient health care systems and improving population health.

    One notable area where medication adherence could have a significant effect on public health is with smoking cessation medications. Tobacco smoking is not only extremely addictive [4], but it also damages almost every organ in the body [5] and is a leading cause of health inequalities [6]. Despite decades of efforts to reduce the smoking prevalence in Canada [7], over 4 million (15%) Canadians smoked in 2020 [8]. Tobacco use remains the leading cause of preventable death in Canada [9], and following smoking cessation, the risk of developing tobacco-related diseases gradually decreases over time [10].

    Varenicline, a partial agonist of the α-4 β-2 nicotinic receptor, is one of the most effective smoking cessation medications associated with a doubling of long-term quitting with 12 weeks of continuous treatment [11]; however, adherence is a major barrier to achieving and maintaining abstinence [12-14]. Among those who take varenicline for smoking cessation, over one-third stop taking the medication by the second week of treatment, and these patients are significantly less likely to quit smoking [15-18]. In addition, several studies that have examined the correlation between sociodemographics and adherence to varenicline have found male sex [15,19], older age [15,19-21], White race [21,22], and higher education to be associated with better adherence [21]. Conversely, 1 study found that greater age was associated with poorer adherence [21].

    One of the most effective strategies to improve adherence to medication is tailoring behavioral interventions [23]. Research has also shown that the use of technological solutions is crucial to providing personalized, cost-effective, and evidence-based interventions [24]. One such digital health solution is a web-based program integrating artificial intelligence (AI) conversational agents, or “health bots,” that can be helpful in improving adherence to medication [25]. Health bots have greater functionality compared with applications providing simple text message reminders since they can track medication intake, provide information about side effects, and answer medication-related questions [26,27].

    The “ChatV” health bot is an example of such a digital health solution that we developed to improve adherence to the smoking cessation medication varenicline [28]. We used a theory-based, patient-centered approach through the Discover Design Build and Test framework [29] to design the health bot; we conducted a rapid review to explore the barriers and facilitators to varenicline adherence and interviewed 20 patients and 19 health care providers to understand similar barriers and facilitators as well as features to be included in a health bot that may improve varenicline adherence. The data from these sources were analyzed using the capability, opportunity, motivation-behavior change (COM-B) [30] model of behavior change and its associated framework (theoretical domain framework) [31] and taxonomy (behavior change techniques taxonomy) [32] to identify the features to be included in the health bot. We used the Wizard of Oz methodology [33] to build a library of responses that were programmed in the health bot to answer users’ questions. We are currently beta testing the minimal viable product that has the following features: (1) scheduling and receiving reminders for varenicline dosing; (2) setting a smoking quit date; (3) answering questions related to varenicline (eg, guidance for missed doses) and smoking cessation (eg, strategies to reduce cravings); (4) providing information and strategies to manage side effects [12,15,18]; (5) tracking medication use and smoking; and (6) encouraging participants to increase their motivation to maintain medication intake and continue their quit attempts. Throughout the development of the health bot, a health equity lens was adopted to address any disparities in varenicline use and health bot features between different demographic characteristics.

    In this protocol, we detail our plans to examine the uptake, usability, acceptability, and appropriateness of ChatV (ie, feasibility) [34] and to determine whether progression to a fully powered randomized controlled trial (RCT) is warranted.


    Design

    We will conduct a nonrandomized, single-arm feasibility study.

    Participant Recruitment

    We will recruit 40 participants. In order to ensure diversity with respect to age, gender, race, and socioeconomic status, we will purposefully recruit participants through a variety of methods, including the research boards of Centre for Addiction and Mental Health (CAMH), where recruitment flyers are posted for patients to explore research opportunities; health care providers at CAMH programs (eg, Smoking Treatment for Ontario Patients [STOP] program, Aboriginal services, and Rainbow services) and partnering organizations (Rainbow Health Ontario); the STOP program’s list of people who are seeking treatment for smoking and who have consented to be contacted for future research studies; and social media and community boards. Research staff members will conduct a prescreening for those who are interested, and if eligible, participants will be enrolled in the study.

    Inclusion Criteria

    The inclusion criteria include individuals who are seeking treatment for their tobacco use who smoke 10 or more cigarettes a day, are willing to set a quit date in the next 30 days, are willing to take varenicline for 12 weeks, are aged 18 years, speak and read English, have a smartphone with a data plan, report being committed to answering questions during follow-up, and live in Ontario.

    Exclusion Criteria

    The exclusion criteria include having contraindications to varenicline use, being pregnant, planning to become pregnant, breastfeeding, or participating in the co-design phase.

    Ethical Considerations

    The CAMH Research Ethics Board (REB #050/2022) has provided approval to conduct the study, and all participants will provide written informed consent before participation. It has been registered at ClinicalTrials.gov (NCT05997901). Any amendments to the protocol are noted, and substantial changes will be reported to the ethics committee. All interested participants will receive an information sheet and a consent form describing the study and providing sufficient information for the participant to make a decision regarding study participation. A research staff member will arrange a personal meeting with the prospective participants and inform them again about all study procedures, benefits, risks, and their rights regarding study participation. Afterward, the consent form is signed by both the participant and the research staff before the participant undertakes any study procedure. The informed consent process is documented in Research Electronic Data Capture (REDCap; Vanderbilt University).

    All study-related documents and data will be held in strict confidence and stored on CAMH servers that will follow CAMH policies and procedures to ensure participant privacy and confidentiality. Any data collected as part of the participants’ completion of the electronic consent form will be securely stored in REDCap, a CAMH-approved data collection tool.

    In addition to free medication (for 12 weeks), participants will receive an honorarium in the form of a CAD $30 (US $22) gift card after completion of the 4-week follow-up survey and a CAD $40 (US $30) gift card after completion of the 12-week interview.

    Procedures

    The research team will inform potential participants about the study and evaluate if they meet the inclusion criteria. If they qualify, participants will receive a consent form and arrange a phone-based discussion to obtain consent. After obtaining consent, participants (n=40) will complete an internet-based questionnaire covering sociodemographic characteristics, nicotine dependence [35], and an adapted scale to measure varenicline adherence self-efficacy [36-38]. After completing the questionnaire, participants will have an internet-based or in-person visit with a health care provider for eligibility confirmation and to obtain a prescription for varenicline for the first 4 weeks (1 tablet [0.5 mg] daily for the first 3 days, 1 tablet [0.5 mg] twice daily for the next 4 days, and 1 tablet [1 mg] twice daily for 3 weeks). In addition, research staff will show participants how to use the health bot on their phones. Medication will be provided in person or, in the case of internet-based contacts, mailed to the participants. A follow-up visit will be scheduled at 2 weeks, where health care providers will assess the participant’s tolerance of varenicline and provide a prescription for the remaining 8 weeks (1 tablet [1 mg] twice daily or a different dose depending on the tolerance of each participant). Any unanticipated problems (ie, adverse events that are unexpected in terms of severity, nature, or frequency; are related or possibly related to participation in the research; and suggest that the research places other research participants at greater risk of harm) will be documented and reported to the research ethics board, and adverse drug reactions will be reported to the market authorization holder (Apotex) as applicable.

    Participants will be required to complete surveys at 1, 4, 8, and 12 weeks. When participants join the study, they will be able to opt to respond to these questions using either the health bot or an internet-based platform. Those who do not complete the internet-based or health bot follow-up within the designated time frame will receive a phone call from our research staff. Additionally, after 12 weeks of joining the study, participants will be invited to participate in a 1-hour semistructured phone interview.

    Measures

    Overview

    The taxonomy of implementation outcomes of Proctor et al [39] guides our selection of implementation outcomes (Table 1).

    Table 1. Summary of measures.
    MeasuresSurvey timelineInterviewAnalytics

    		BaselineWeek 1Week 4Week 8Week 12Week 12Weeks 1-12
    Demographics✓





    Adoption




    		✓✓
    Acceptability
    		✓a✓a✓a✓a✓
    Appropriateness
    		✓b✓b✓b✓b✓
    Fidelity




    		✓
    Usability
    		✓c✓c✓c✓c✓
    Medication Adherence
    		✓d✓d✓d✓d✓
    Smoking Status
    		✓✓✓✓✓

    aAcceptability of Intervention Measure.

    bIntervention Appropriateness Measure.

    cSystem Usability Scale.

    dTimeline Followback.

    Acceptability

    Acceptability is the extent to which an innovation is agreeable, palatable, or satisfactory to a stakeholder [39]. In order to measure acceptability, we will use the Acceptability of Intervention Measure (AIM) in the follow-up surveys as well as in the semistructured phone interviews.

    Appropriateness

    Appropriateness is the perceived fit or compatibility of an innovation with a practice setting or context [39]. In this study, appropriateness will be assessed on an individual basis, including factors such as alignment with users’ attitudes, needs, and background, as measured in the 12-week survey with the Intervention Appropriateness Measure (IAM), a validated 4-item intervention appropriateness scale [40], and explored in depth in the semistructured interviews.

    Fidelity

    Fidelity is the extent to which an intervention is used as intended [39]. For this study, in the semistructured interviews, we will investigate the functions of the health bot that participants used; the purposes for which they used it (such as answering questions or seeking support); the frequency of usage; and the quality, specifically whether the health bot provided accurate information.

    Adoption

    Adoption is the intention, decision, or initiation of use for an evidence-based practice, characterizing it at the level of the provider or organization [39]. Since the concept of adoption is closely related to the actual use of systems, researchers investigating behavioral intervention technologies, such as the health bot, have extended this level of analysis to include consumer behavior [41]. In this study, we will measure the adoption of the health bot by analyzing analytics data throughout the 12 weeks that the participant is scheduled to take varenicline. These data will allow us to evaluate the timing (day and time) of user interactions with the health bot, the specific features used, and the duration of engagement. Thus, we will have objective data on the actual adoption of the health bot by each participant. Additionally, during the semistructured interviews, we will probe for participants’ subjective perceptions of their use of the health bot.

    Usability

    Usability will be assessed using the System Usability Scale (SUS) [42]. The SUS is a 10-item measure assessing usability and user satisfaction with technology.

    The AIM, IAM, and SUS all have items that are rated on a 5-point Likert scale (1=completely or strongly disagree to 5=completely or strongly agree).

    Medication Adherence and Smoking Status

    Participants will be asked about their smoking status [43] and adherence to varenicline using the Timeline Followback (TLFB) [44] in the surveys collected at 1, 4, 8, and 12 weeks. Participants will report the number of pills taken since the previous assessment. Participants who log their varenicline use in the health bot will be able to refer to these data when filling out the TLFB. We chose the TLFB method for assessing adherence because the 12-week TLFB measure demonstrates a moderate correlation with saliva varenicline levels and is widely regarded as the most practical measure of varenicline adherence [16]. In order to assess the feasibility and accuracy of measuring adherence using TLFB, participants will also be asked to send pictures of their varenicline blister packs at 1, 4, 8, and 12 weeks. This additional measure will also help explore which method to assess adherence should be used for any future research studies (eg, RCT). Smoking abstinence will be defined by a negative response to the dichotomous 7-day point prevalence question, “Have you had a cigarette, even a puff, in the last 7 days?”

    Analysis of Survey Data

    We will calculate descriptive statistics, including means, medians, ranges, and SDs, for the demographic data as well as for the AIM, IAM, and analytics data captured by the health bot internal logs. All scores will be analyzed using the appropriate guidelines provided by Weiner et al [40] (acceptability and appropriateness) and Brooke [42] (usability). Consistent with previous studies, adherence will be defined as taking ≥80% of the prescribed varenicline [45-48]. We will not treat adherence as a continuous variable because it is likely to be strongly bimodal and is not necessarily linearly associated with the cessation outcome. We will calculate the percentage of participants who were adherent at week 12 and who reported not having smoked in the last 7 days. These results will serve as progression criteria to determine whether a full-scale RCT is warranted. Although the sample size will be insufficient for meaningful regression analysis, especially given the potentially nonlinear effects of age, we will also test for differences across age and gender groups using 2-tailed t tests and chi-square tests. The power for these comparisons will be low; therefore, this analysis will be an exploratory evaluation of large adherence or outcome differences that may be important to future study design, and interpretation will focus on observed differences and CIs.

    Analysis of Semistructured Interviews

    We will use framework analysis [49] to analyze the data. Specifically, we will use the implementation outcomes of Proctor et al [39] with an intersectional lens as the a priori framework. Framework analysis is a method used to draw descriptive and explanatory conclusions from qualitative data and includes the following six steps: (1) verbatim transcription of interviews, (2) familiarization of interviews, (3) coding and labeling the data, (4) developing a framework by grouping and categorizing codes, (5) indexing transcripts using the developed framework, and (6) reducing and charting data into a matrix. Interview findings will help us contextualize the findings from the surveys and the information collected from the use of the health bot. For example, we will understand why the health bot was or was not used (adoption); if it was used as intended (fidelity), what about the health bot the participants found agreeable and what they did not (acceptability); what features, if any, aligned with participants’ attitudes, needs, and background (appropriate); and how other features would need to change in order to align with their needs. In addition, we will be able to understand how social identity (eg, age and gender) and power structures (eg, ageism and sexism) played a role in these results. This will provide us with important information to improve the health bot through an equity lens.

    Triangulation of Findings

    Once the survey data and interviews are analyzed, we will triangulate the results by listing all the findings on the same page to identify instances where findings from each method align (convergence), provide additional insights on the same topic (complementarity), or seem to conflict with one another (discrepancy or dissonance) [50,51]. Triangulating the data will allow us to contextualize the quantitative findings and explore “intermethod discrepancy” that might lead to a better understanding of how the health bot works and the modifications needed.

    Exploring the Feasibility of the Approaches on Recruitment, Retention, and Data Collection

    This study, using “stop, amend, and go” progression criteria for pilot studies [51], will help determine if proceeding with an RCT is a logical next step and whether additional adjustments are necessary. We will only conduct a full-scale RCT if none of the criteria outlined in Table 2 meet the “stop” criterion (unless we see that there are contextual issues that can be modified to overcome the identified problems). If 1 or more of the concepts in Table 2 meet the “amend” criterion, the trial will proceed to an RCT only if the issues can be addressed. If all concepts in Table 2 satisfy the “go” criterion, we will proceed with planning an RCT.

    Table 2. Criteria to decide if a randomized controlled trial is warranted.
    CriteriaStopAmendGo
    Patient recruitmentIf <30 patients are recruited within 12 monthsIf 30-40 patients are recruited within 12 monthsIf 40 patients are recruited within 12 months
    Patient retentionIf <25% of patients answer the 12-week surveyIf 25.1%-79.9% of patients answer the 12-week surveyIf ≥80% of patients answer the 12-week survey
    Missing data, medication adherence, or smoking statusIf >60% of adherence data and smoking status are missingIf 21%-59% of adherence and smoking status data are missingIf ≤20% of adherence and smoking status data are missing
    Acceptability and appropriatenessIf  mean score for the AIMa or IAMb is ≤8If  mean score for the AIM or IAM is 9-12If  mean score for the AIM or IAM is ≥13
    UsabilityIf mean SUSc score is ≤40If mean SUS score is 41-67If mean SUS score is ≥68
    AdoptionIf mean use is ≤ 20 timesIf mean use is 21-79 timesIf mean use is ≥80 times

    aAIM: Acceptability of Intervention Measure.

    bIAM: Intervention Appropriateness Measure.

    cSUS: System Usability Scale.


    Participant enrollment for the study will begin in January 2024.


    Despite the proven effectiveness of varenicline in aiding smoking cessation [52], a significant number of individuals attempting to quit continue to smoke because they struggle with adhering to their varenicline regimen [12-14]. While there are proven strategies to help people adhere to their medications, they have been hard to implement. Digital technology for medication adherence has started to emerge [53,54]; whether these tools can be feasibly implemented and adopted, especially within diverse and disadvantaged populations, remains largely unknown.

    In this protocol, we outline the methodology to assess the uptake, usability, acceptability, and appropriateness of ChatV and assist in determining whether it is appropriate to advance to an RCT to gauge the effectiveness of the intervention. This study will address many limitations of previous mobile health trials [55,56], including using an intervention that clearly outlines the behavior-change theory used and the intervention content. In addition, this study provides well-outlined a priori criteria for progressing to an RCT. This will enhance its replicability. Furthermore, this study will provide insights into potential mechanisms underlying the intervention’s impact, shed light on the effectiveness, or ineffectiveness, of its various components, and offer possible explanations.

    In summary, the results of this feasibility study will provide much-needed data and insight into the potential implementation of ChatV to augment adherence to smoking cessation medications. In addition, this project has the potential to increase our understanding of how digital tools can improve adherence to other medications in a cost-effective manner.

    Acknowledgments

    This research is funded by a Proof of Concept Intervention Grant in Primary Prevention of Cancer (action grant) of the Canadian Cancer Society and the Canadian Institutes of Health Research-Institute for Cancer Research (grant 707218) and a Canadian Institutes of Health Research Project grant (funding reference PJT 180405; Multimedia Appendix 1). The authors would like to thank MEMOTEXT for their contribution to the development of the ChatV health bot.

    Data Availability

    The data sets generated during this study will be available from the corresponding author upon reasonable request.

    Authors' Contributions

    NM conceived the study with the support of JR, PS, SV, LZ, and MR. NM (lead), KM, ME, SH, RTAK, and PS have shaped the intervention (health bot). NM (lead), KM, and ME were involved in gaining ethical approval. NM and KM wrote the first draft of the manuscript. All authors reviewed and edited the manuscript and approved the final version of the manuscript.

    Conflicts of Interest

    The authors declare that they have no conflicting interests with respect to this manuscript; however, some authors have general disclosures to report. NM reports receipt of funding from the Canadian Institutes of Health Research, the Canadian Cancer Society, and the discovery fund of the Centre for Addiction and Mental Health (CAMH). PS reports receipt of funding from the Canadian Institutes of Health Research, the Canadian Cancer Society, Pfizer Inc, Pfizer Canada, and the Ontario Lung Association. PS also reports that, through an open-tender process, Johnson & Johnson, Novartis, and Pfizer Inc are vendors of record for having provided smoking cessation pharmacotherapy for research studies at free or discounted rates. PS holds the Vice-Chair, Research, and Giblon Professor in Family Medicine Research, a university-named professorship at the University of Toronto. OCM reports receiving grant and salary support from the Centre for Addiction and Mental Health and the Department of Family and Community Medicine at the University of Toronto through the New Investigator Award.

    Multimedia Appendix 1

    Peer-review report by the Canadian Institutes of Health Research.

    PDF File (Adobe PDF File), 78 KB
    1. Sabaté E. Adherence to Long-Term Therapies: Evidence for Action. Geneva, Switzerland. World Health Organization; 2003.
    2. Mongkhon P, Ashcroft DM, Scholfield CN, Kongkaew C. Hospital admissions associated with medication non-adherence: a systematic review of prospective observational studies. BMJ Qual Saf. 2018;27(11):902-914. [CrossRef] [Medline]
    3. Osterberg L, Blaschke T. Adherence to medication. N Engl J Med. 2005;353(5):487-497. [CrossRef]
    4. National Center for Chronic Disease Prevention and Health Promotion (US) Office on Smoking and Health. The Health Consequences of Smoking—50 Years of Progress: A Report of the Surgeon General. Atlanta, GA. US Centers for Disease Control and Prevention; 2014.
    5. Office of the Surgeon General OoS, Health. Reports of the surgeon general. In: The Health Consequences of Smoking: A Report of the Surgeon General. Atlanta (GA). Centers for Disease Control and Prevention (US); 2004.
    6. Briefing: health inequalities and smoking 2019. Actions on Smoking and Health (ASH). 2022. URL: https://ash.org.uk/wp-content/uploads/2019/09/ASH-Briefing_Health-Inequalities.pdf [accessed 2023-11-25]
    7. Reid JHT, Burkhalter R, Rynard VL, Douglas O. Tobacco use in Canada: patterns and trends. Propel Centre for Population Health Impact, University of Waterloo. 2019. URL: https:/​/uwaterloo.​ca/​tobacco-use-canada/​sites/​default/​files/​uploads/​files/​tobacco_use_in_canada_2019.​pdf [accessed 2023-11-25]
    8. Statistics Canada. Smokers, by age group. Table 13-10-0096-10. 2022. URL: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=1310009610 [accessed 2023-11-25]
    9. Janz T. Current smoking trends. Statistics Canada. 2012. URL: https://www150.statcan.gc.ca/n1/en/pub/82-624-x/2012001/article/11676-eng.pdf?st=ENM5C-fZ [accessed 2023-11-25]
    10. US Public Health Service, Office of the Surgeon General; US Office on Smoking and Health. The health benefits of smoking cessation: a report of the surgeon general. US Office on Smoking and Health. 1990. URL: https://profiles.nlm.nih.gov/spotlight/nn/catalog/nlm:nlmuid-101584932X37-doc [accessed 2023-11-25]
    11. Cahill K, Stevens S, Perera R, Lancaster T. Pharmacological interventions for smoking cessation: an overview and network meta-analysis. Cochrane Database Syst Rev. 2013;2013(5):CD009329. [FREE Full text] [CrossRef] [Medline]
    12. Kaur K, Kaushal S, Chopra SC. Varenicline for smoking cessation: a review of the literature. Curr Ther Res. Mar 2009;70(1):35-54. [FREE Full text] [CrossRef] [Medline]
    13. Pacek LR, McClernon FJ, Bosworth HB. Adherence to pharmacological smoking cessation interventions: a literature review and synthesis of correlates and barriers. Nicotine Tob Res. 2018;20(10):1163-1172. [FREE Full text] [CrossRef] [Medline]
    14. Liberman JN, Lichtenfeld MJ, Galaznik A, Mastey V, Harnett J, Zou KH, et al. Adherence to varenicline and associated smoking cessation in a community-based patient setting. J Manag Care Pharm. 2013;19(2):125-131. [FREE Full text] [CrossRef] [Medline]
    15. Catz SL, Jack LM, McClure JB, Javitz HS, Deprey M, Zbikowski SM, et al. Adherence to varenicline in the COMPASS smoking cessation intervention trial. Nicotine Tob Res. 2011;13(5):361-368. [FREE Full text] [CrossRef] [Medline]
    16. Peng AR, Morales M, Wileyto EP, Hawk LW, Cinciripini P, George TP, et al. Measures and predictors of varenicline adherence in the treatment of nicotine dependence. Addict Behav. 2017;75:122-129. [FREE Full text] [CrossRef] [Medline]
    17. Peng AR, Schnoll R, Hawk LW, Cinciripini P, George TP, Lerman C, et al. Predicting smoking abstinence with biological and self-report measures of adherence to varenicline: impact on pharmacogenetic trial outcomes. Drug Alcohol Depend. 2018;190:72-81. [FREE Full text] [CrossRef] [Medline]
    18. Peng AR, Swardfager W, Benowitz NL, Ahluwalia JS, Lerman C, Nollen NL, et al. Impact of early nausea on varenicline adherence and smoking cessation. Addiction. 2020;115(1):134-144. [FREE Full text] [CrossRef] [Medline]
    19. Zeng F, Chen CI, Mastey V, Zou KH, Harnett J, Patel BV. Effects of copayment on initiation of smoking cessation pharmacotherapy: an analysis of varenicline reversed claims. Clin Ther. 2011;33(2):225-234. [CrossRef] [Medline]
    20. Crawford G, Weisbrot J, Bastian J, Flitter A, Jao NC, Carroll A, et al. Predictors of varenicline adherence among cancer patients treated for tobacco dependence and its association with smoking cessation. Nicotine Tob Res. 2019;21(8):1135-1139. [FREE Full text] [CrossRef] [Medline]
    21. Suehs BT, Davis C, Galaznik A, Joshi AV, Zou KH, Patel NC. Association of out-of-pocket pharmacy costs with adherence to varenicline. J Manag Care Pharm. 2014;20(6):592-600. [FREE Full text] [CrossRef] [Medline]
    22. Shelley D, Tseng TY, Gonzalez M, Krebs P, Wong S, Furberg R, et al. Correlates of adherence to varenicline among HIV+ smokers. Nicotine Tob Res. 2015;17(8):968-974. [FREE Full text] [CrossRef] [Medline]
    23. Hollands GJ, Naughton F, Farley A, Lindson N, Aveyard P. Interventions to increase adherence to medications for tobacco dependence. Cochrane Database Syst Rev. 2019;8(8):CD009164. [FREE Full text] [CrossRef] [Medline]
    24. Pavel M, Jimison HB, Wactlar HD, Hayes TL, Barkis W, Skapik J, et al. The role of technology and engineering models in transforming healthcare. IEEE Rev Biomed Eng. 2013;6:156-177. [CrossRef] [Medline]
    25. Aggarwal A, Tam CC, Wu D, Li X, Qiao S. Artificial intelligence-based chatbots for promoting health behavioral changes: systematic review. J Med Internet Res. 2023;25:e40789. [FREE Full text] [CrossRef] [Medline]
    26. Bates M. Health care chatbots are here to help. IEEE Pulse. 2019;10(3):12-14. [CrossRef] [Medline]
    27. Olano-Espinosa E, Avila-Tomas JF, Minue-Lorenzo C, Matilla-Pardo B, Serrano MES, Martinez-Suberviola FJ, et al. Effectiveness of a conversational chatbot (Dejal@bot) for the adult population to quit smoking: pragmatic, multicenter, controlled, randomized clinical trial in primary care. JMIR Mhealth Uhealth. 2022;10(6):e34273. [FREE Full text] [CrossRef] [Medline]
    28. Minian N, Mehra K, Rose J, Veldhuizen S, Zawertailo L, Ratto M, et al. Cocreation of a conversational agent to help patients adhere to their varenicline treatment: a study protocol. Digit Health. 2023;9:20552076231182807. [FREE Full text] [CrossRef] [Medline]
    29. Lyon AR, Munson SA, Renn BN, Atkins DC, Pullmann MD, Friedman E, et al. Use of human-centered design to improve implementation of evidence-based psychotherapies in low-resource communities: protocol for studies applying a framework to assess usability. JMIR Res Protoc. 2019;8(10):e14990. [FREE Full text] [CrossRef] [Medline]
    30. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42. [FREE Full text] [CrossRef] [Medline]
    31. Cane J, O'Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37. [FREE Full text] [CrossRef] [Medline]
    32. Michie S, Richardson M, Johnston M, Abraham C, Francis J, Hardeman W, et al. The behavior change technique taxonomy (v1) of 93 hierarchically clustered techniques: building an international consensus for the reporting of behavior change interventions. Ann Behav Med. 2013;46(1):81-95. [FREE Full text] [CrossRef] [Medline]
    33. Kelley JF. An iterative design methodology for user-friendly natural language office information applications. ACM Trans Inf Syst. 1984;2(1):26-41. [FREE Full text] [CrossRef]
    34. Whitehead AL, Sully BGO, Campbell MJ. Pilot and feasibility studies: is there a difference from each other and from a randomised controlled trial? Contemp Clin Trials. 2014;38(1):130-133. [FREE Full text] [CrossRef] [Medline]
    35. Fagerström K. Determinants of tobacco use and renaming the FTND to the Fagerstrom Test for Cigarette Dependence. Nicotine Tob Res. 2012;14(1):75-78. [CrossRef] [Medline]
    36. Fucito LM, Toll BA, Salovey P, O'Malley SS. Beliefs and attitudes about bupropion: implications for medication adherence and smoking cessation treatment. Psychol Addict Behav. 2009;23(2):373-379. [CrossRef] [Medline]
    37. Fernandez S, Chaplin W, Schoenthaler AM, Ogedegbe G. Revision and validation of the Medication Adherence Self-Efficacy Scale (MASES) in hypertensive African Americans. J Behav Med. 2008;31(6):453-462. [FREE Full text] [CrossRef] [Medline]
    38. Tseng TY, Krebs P, Schoenthaler A, Wong S, Sherman S, Gonzalez M, et al. Combining text messaging and telephone counseling to increase varenicline adherence and smoking abstinence among cigarette smokers living with HIV: a randomized controlled study. AIDS Behav. 2017;21(7):1964-1974. [FREE Full text] [CrossRef] [Medline]
    39. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65-76. [FREE Full text] [CrossRef] [Medline]
    40. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108. [FREE Full text] [CrossRef] [Medline]
    41. Hermes EDA, Lyon AR, Schueller SM, Glass JE. Measuring the implementation of behavioral intervention technologies: recharacterization of established outcomes. J Med Internet Res. 2019;21(1):e11752. [FREE Full text] [CrossRef] [Medline]
    42. Brooke J. SUS: a 'quick and dirty' usability. In: Jordan PW, Thomas B, Weerdmeester B, McClelland I, editors. Usability Evaluation in Industry. Boca Raton, Florida. CRC Press; 1996;189-194.
    43. Heatherton TF, Kozlowski LT, Frecker RC, Rickert W, Robinson J. Measuring the heaviness of smoking: using self-reported time to the first cigarette of the day and number of cigarettes smoked per day. Br J Addict. 1989;84(7):791-799. [CrossRef] [Medline]
    44. Brown RA, Burgess ES, Sales SD, Whiteley JA, Evans DM, Miller IW. Reliability and validity of a smoking timeline follow-back interview. Psychol Addict Behav. 1998;12(2):101-112. [CrossRef]
    45. Hays JT, Leischow SJ, Lawrence D, Lee TC. Adherence to treatment for tobacco dependence: association with smoking abstinence and predictors of adherence. Nicotine Tob Res. 2010;12(6):574-581. [CrossRef] [Medline]
    46. Nollen NL, Mayo MS, Ahluwalia JS, Tyndale RF, Benowitz NL, Faseru B, et al. Factors associated with discontinuation of bupropion and counseling among African American light smokers in a randomized clinical trial. Ann Behav Med. 2013;46(3):336-348. [FREE Full text] [CrossRef] [Medline]
    47. Buchanan TS, Berg CJ, Cox LS, Nazir N, Benowitz NL, Yu L, et al. Adherence to varenicline among African American smokers: an exploratory analysis comparing plasma concentration, pill count, and self-report. Nicotine Tob Res. 2012;14(9):1083-1091. [FREE Full text] [CrossRef] [Medline]
    48. Nollen NL, Cox LS, Nazir N, Ellerbeck EF, Owen A, Pankey S, et al. A pilot clinical trial of varenicline for smoking cessation in black smokers. Nicotine Tob Res. 2011;13(9):868-873. [FREE Full text] [CrossRef] [Medline]
    49. Gale NK, Heath G, Cameron E, Rashid S, Redwood S. Using the framework method for the analysis of qualitative data in multi-disciplinary health research. BMC Med Res Methodol. 2013;13:117. [FREE Full text] [CrossRef] [Medline]
    50. Farmer T, Robinson K, Elliott SJ, Eyles J. Developing and implementing a triangulation protocol for qualitative health research. Qual Health Res. 2006;16(3):377-394. [CrossRef] [Medline]
    51. O'Cathain A, Murphy E, Nicholl J. Three techniques for integrating data in mixed methods studies. BMJ. 2010;341:c4587. [FREE Full text] [CrossRef] [Medline]
    52. Garrison GD, Dugan SE. Varenicline: a first-line treatment option for smoking cessation. Clin Ther. 2009;31(3):463-491. [CrossRef] [Medline]
    53. Thomson K, Brouwers C, Damman OC, de Bruijne MC, Timmermans DR, Melles M. How health care professionals evaluate a digital intervention to improve medication adherence: qualitative exploratory study. JMIR Hum Factors. 2018;5(1):e7. [FREE Full text] [CrossRef] [Medline]
    54. Musiimenta A, Tumuhimbise W, Mugaba AT, Muzoora C, Armstrong-Hough M, Bangsberg D, et al. Digital monitoring technologies could enhance tuberculosis medication adherence in Uganda: mixed methods study. J Clin Tuberc Other Mycobact Dis. 2019;17:100119. [FREE Full text] [CrossRef] [Medline]
    55. McClure JB, Anderson ML, Bradley K, An LC, Catz SL. Evaluating an adaptive and interactive mHealth smoking cessation and medication adherence program: a randomized pilot feasibility study. JMIR Mhealth Uhealth. 2016;4(3):e94. [FREE Full text] [CrossRef] [Medline]
    56. Gordon JS, Armin JS, Cunningham JK, Muramoto ML, Christiansen SM, Jacobs TA. Lessons learned in the development and evaluation of RxCoach™, an mHealth app to increase tobacco cessation medication adherence. Patient Educ Couns. 2017;100(4):720-727. [FREE Full text] [CrossRef] [Medline]

    ‎
    AI: artificial intelligence
    AIM: Acceptability of Intervention Measure
    CAMH: Centre for Addiction and Mental Health
    COM-B: capability, opportunity, motivation-behavior change
    IAM: Intervention Appropriateness Measure
    RCT: randomized controlled trial
    REDCap: Research Electronic Data Capture
    STOP: Smoking Treatment for Ontario Patients
    SUS: System Usability Scale
    TLFB: Timeline Followback

    Edited by A Mavragani; The proposal for this study was peer reviewed by the Canadian Institutes of Health Research. See the Multimedia Appendix for the peer-review report; submitted 10.10.23; accepted 23.11.23; published 11.12.23.

    Copyright

    ©Nadia Minian, Kamna Mehra, Mackenzie Earle, Sowsan Hafuth, Ryan Ting-A-Kee, Jonathan Rose, Scott Veldhuizen, Laurie Zawertailo, Matt Ratto, Osnat C Melamed, Peter Selby. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 11.12.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.