This paper is in the following e-collection/theme issue:

Usage Data and Process Outcomes in eHealth Trials (56) Health Coaching for Diabetes Patients (142) Web-based and Mobile Health Interventions (3102) Diabetes Self-Management (463)

Published on in Vol 12 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/46115, first published .
Understanding Whether and How a Digital Health Intervention Improves Transition Care for Emerging Adults Living With Type 1 Diabetes: Protocol for a Mixed Methods Realist Evaluation

Understanding Whether and How a Digital Health Intervention Improves Transition Care for Emerging Adults Living With Type 1 Diabetes: Protocol for a Mixed Methods Realist Evaluation

Understanding Whether and How a Digital Health Intervention Improves Transition Care for Emerging Adults Living With Type 1 Diabetes: Protocol for a Mixed Methods Realist Evaluation

Authors of this article:

Ruoxi Wang1 Author Orcid Image ;   Geneviève Rouleau2, 3, 4 Author Orcid Image ;   Gillian Lynn Booth5, 6, 7 Author Orcid Image ;   Anne-Sophie Brazeau8 Author Orcid Image ;   Noor El-Dassouki9 Author Orcid Image ;   Madison Taylor9 Author Orcid Image ;   Joseph A Cafazzo7, 9, 10, 11 Author Orcid Image ;   Marley Greenberg12, 13 Author Orcid Image ;   Meranda Nakhla14, 15 Author Orcid Image ;   Rayzel Shulman7, 16, 17, 18 Author Orcid Image ;   Laura Desveaux1, 7 Author Orcid Image
Article Authors Cited by Tweetations (4) Metrics

    Protocol

    1Institute for Better Health, Trillium Health Partners, Mississauga, ON, Canada

    2Institute for Health System Solutions and Virtual Care, Women’s College Hospital, Toronto, ON, Canada

    3Département des Sciences Infirmières, Université du Québec en Outaouais, St-Jérôme, QC, Canada

    4Faculté des sciences infirmières, l'Université de Montréal, Montreal, QC, Canada

    5MAP Centre for Urban Health Solutions, Unity Health Toronto, Toronto, ON, Canada

    6Department of Medicine, University of Toronto, Toronto, ON, Canada

    7Institute of Health Policy, Management and Evaluation, University of Toronto, Toronto, ON, Canada

    8School of Human Nutrition, McGill University, Montréal, QC, Canada

    9Centre for Digital Therapeutics, Toronto General Hospital, University Health Network, Toronto, ON, Canada

    10Institute of Biomedical Engineering, University of Toronto, Toronto, ON, Canada

    11Department of Computer Science, University of Toronto, Toronto, ON, Canada

    12Department of Philosophy, Joint Centre for Bioethics, University of Toronto, Toronto, ON, Canada

    13Diabetes Action Canada, Toronto, ON, Canada

    14Division of Endocrinology, Montreal Children’s Hospital, McGill University, Montréal, QC, Canada

    15Research Institute of the McGill University Health Centre, Montréal, QC, Canada

    16Child Health Evaluative Sciences, SickKids Research Institute, Toronto, ON, Canada

    17Division of Endocrinology, The Hospital for Sick Children, Toronto, ON, Canada

    18Department of Pediatrics, University of Toronto, Toronto, ON, Canada

    *these authors contributed equally

    Corresponding Author:

    Laura Desveaux, PT, PhD

    Institute for Better Health

    Trillium Health Partners

    100 Queensway W

    Mississauga, ON, L5B 1B8

    Canada

    Phone: 1 437 772 6836

    Email: laura.desveaux@thp.ca


    Background: Emerging adults living with type 1 diabetes (T1D) face a series of challenges with self-management and decreased health system engagement, leading to an increased risk of acute complications and hospital admissions. Effective and scalable strategies are needed to support this population to transfer seamlessly from pediatric to adult care with sufficient self-management capability. While digital health interventions for T1D self-management are a promising strategy, it remains unclear which elements work, how, and for which groups of individuals.

    Objective: This study aims to evaluate the design and implementation of a multicomponent SMS text message–based digital health intervention to support emerging adults living with T1D in real-world settings. The objectives are to identify the intervention components and associated mechanisms that support user engagement and T1D health care transition experiences and determine the individual characteristics that influence the implementation process.

    Methods: We used a realist evaluation embedded alongside a randomized controlled trial, which uses a sequential mixed methods design to analyze data from multiple sources, including intervention usage data, patient-reported outcomes, and realist interviews. In step 1, we conducted a document analysis to develop a program theory that outlines the hypothesized relationships among “individual-level contextual factors, intervention components and features, mechanisms, and outcomes,” with special attention paid to user engagement. Among them, intervention components and features depict 10 core characteristics such as transition support information, problem-solving information, and real-time interactivity. The proximal outcomes of interest include user engagement, self-efficacy, and negative emotions, whereas the distal outcomes of interest include transition readiness, self-blood glucose monitoring behaviors, and blood glucose. In step 2, we plan to conduct semistructured realist interviews with the randomized controlled trial’s intervention-arm participants to test the hypothesized “context-intervention-mechanism-outcome” configurations. In step 3, we plan to triangulate all sources of data using a coincidence analysis to identify the necessary combinations of factors that determine whether and how the desired outcomes are achieved and use these insights to consolidate the program theory.

    Results: For step 1 analysis, we have developed the initial program theory and the corresponding data collection plan. For step 2 analysis, participant enrollment for the randomized controlled trial started in January 2023. Participant enrollment for this realist evaluation was anticipated to start in July 2023 and continue until we reached thematic saturation or achieved informational power.

    Conclusions: Beyond contributing to knowledge on the multiple pathways that lead to successful engagement with a digital health intervention as well as target outcomes in T1D care transitions, embedding the realist evaluation alongside the trial may inform real-time intervention refinement to improve user engagement and transition experiences. The knowledge gained from this study may inform the design, implementation, and evaluation of future digital health interventions that aim to improve transition experiences.

    International Registered Report Identifier (IRRID): PRR1-10.2196/46115

    JMIR Res Protoc 2023;12:e46115

    doi:10.2196/46115

    Keywords

    digital health; emerging adults; realist evaluation; self-management; transition to adult care; type 1 diabetes 


    Type 1 diabetes (T1D) is a common chronic condition that affects 9 million people globally [1], including over 650,000 children (aged 0-14 years) and 560,000 teenagers (aged 15-19 years) [2]. T1D management requires knowledge, skills, and motivation to perform daily self-management as well as routine health care management and monitoring throughout the lifespan [3-5]. A total of 42% of individuals with T1D onset during childhood often experience deterioration in glycemic management, which confers an increased risk of chronic complications and acute diabetes complications such as life-threatening ketoacidosis during adolescence and early adulthood [1,6]. Effective health care management and a smooth transition from pediatric to adult care can help mitigate this risk [5,7,8]. Unfortunately, the transition period is fraught with challenges in part because it coincides with emerging adulthood, when individuals are facing a series of changes in their independence and responsibilities for disease management [9]. Current estimates suggest that over 20% of emerging adults experience a gap of over 6 months between pediatric and adult care medical visits during transition [10,11]. Effective transition support must attend to medical, psychological, educational, and vocational needs in order to support sufficient self-management skills [12-14]. Failure to address these needs may lead to adverse health outcomes [14], decreased health system engagement [10], and increased hospital admissions [15].

    Strategies to improve the transition process have been implemented at the patient, provider, and service levels [16,17], but are often institution-specific or resource-intensive, thereby limiting their accessibility, scalability, and generalizability [18-20]. Digital health interventions are increasingly used to address these limitations while also leveraging the high prevalence of smartphone use among youth [21]. The hypothesized benefits are also compelling, including the flexibility to access and exchange information instantly, irrespective of geographic constraints, at low cost, and the potential to provide more personalized support [22]. Digital health T1D management interventions, through mobile apps [23-25], websites [26], and text messages, emails, or telephone calls [25,27-29], have become increasingly common [23-29] and acceptable [23,26,27] to young adults, with demonstrated impacts on clinic attendance [28] and glycemic control [29]. A recent paper by Cafazzo et al [23] piloting a T1D management mobile app in adolescents found 88% of participants were satisfied with the app. Moreover, they found a significant increase in the frequency of blood glucose measurement among their participant sample. In a pragmatic clinical trial, Butalia et al [28] observed significantly greater outpatient appointment attendance in transitioning youth living with T1D who received a communication technology-based (SMS text message, email, or telephone) transition coordinator intervention as compared to those who received care as usual. Also, through a crossover trial with adolescents, Rami et al [29] found SMS text messaging-based telemedical support feasible and helpful in improving hemoglobin A1c (HbA1c).

    However, potential impact can be undermined by the often-reported rapid drop in intervention engagement, which occurs sometime between 2 weeks and 6 months after initial use [30,31]. Among adolescents living with T1D exposed to a mobile self-management tool, 1 study reported that only 35% of participants were either moderately or highly engaged over the 12-month intervention [24]. Despite this well-known risk, comparatively little is known about how to overcome this common challenge. Relatedly, many multicomponent digital health interventions report aggregate effects [27-29], limiting our ability to understand which elements contribute to achieving the distinct but related goals of ensuring sustained intervention engagement while also effectively addressing the known barriers to desired outcomes.

    Realist evaluation, a theory-driven evaluation framework [32], seeks to unpack the black box of interventions by establishing the causal links between intervention resources and associated outcomes while identifying the circumstances needed to facilitate the change in a Context-Mechanism-Outcome (CMO) framework [33-36]. Given the complex and context-specific nature of health interventions [37], they have been increasingly adopted in the health field [34,38-40] to solve the question of what works, for whom, and in what circumstances [41-43].

    By acknowledging the potential of realist evaluation in addressing the abovementioned knowledge gaps, we have developed a protocol for a sequential mixed methods realist evaluation embedded alongside a randomized controlled trial (RCT) evaluating the effect of a digital health intervention on the transition experiences of emerging adults living with T1D. Using a realist evaluation framework, the specific objectives of the embedded realist evaluation will be to (1) develop and test the intervention program theory that identifies what elements of the intervention contribute to successful engagement and improve transition experiences, how they do it, and for which groups of individuals; and (2) consolidate the intervention program theory by identifying the crucial combinations of factors that are minimally sufficient for an effective implementation of the digital health intervention and their coexistent causal pathways.


    Study Design

    Following Mirzoev et al [40], we plan to conduct this embedded realist evaluation in 3 steps (Figure 1), which include developing (step 1; completed), testing (step 2; to be conducted), and consolidating the program theory (step 3; to be conducted).

    ‎
    Figure 1. Study design and methods.

    As a theory-based evaluation and in line with best practices [32,44], this study is guided by a program theory outlining the hypothesized mechanisms through which the digital health intervention is expected to work and the corresponding context, with specific attention paid to user engagement (step 1). We have added an explanatory factor “intervention” (I) to the conventional “context-mechanism-outcome” (CMO) framework [45] following Breton et al [46] and Shams et al [47] to disaggregate the 2 major components of mechanisms (M) in the conventional CMO framework (ie, intervention resources and recipients’ reasoning) [36]. This will enable us to specify which elements of this multicomponent intervention contribute to the desired outcomes and how they are influenced by context [44]. Specifically, we have developed the initial program theory in the framework of “context-intervention-mechanism-outcome” (CIMO) to understand for individuals with which contextual characteristics (C), which intervention components and features (I), trigger what mechanisms (M), and therefore yield what target outcomes (O). This initial program theory informs the following data collection and analysis [39].

    Since realist evaluation is method neutral [48], this study uses mixed methods and integrates multisource data to test and consolidate the program theory. In step 2, we plan to conduct semistructured realist interviews with participants in the RCT intervention arm in a “teacher-learner cycle” [49]. The interviews will aim to test each of the hypothesized CIMO configurations by exploring individual perspectives and experiences using “Keeping in Touch” (KiT), including how participants interact with KiT and under what circumstances it facilitates (or fails to facilitate) mechanisms that lead to engagement and improved transition experiences.

    Multicomponent interventions often achieve desired outcomes through distinct combinations of factors that interact to produce the outcome, as well as through multiple mechanisms [38]. For instance, the combination of a high level of positive outcome expectancies (ie, belief about the consequence of performing a specific behavior) and a high level of self-efficacy (ie, confidence about the ability to perform a specific behavior) may lead to improved diabetes self-care [50,51], whereas the combination of a high level of outcome expectancies and a low level of self-efficacy may lead to poorer diabetes self-care [51]. This highlights the need to identify difference-making factors as well as the settings in which they make a difference. Coincidence analysis (CNA) is a novel configurational approach underpinned by Boolean algebra to identify the crucial difference-making conditions, which include combinations of factors (including contextual factors and intervention components) necessary to achieve target outcomes and the causal pathways (mechanisms) that lead to positive effects [52]. Therefore, in step 3, we will leverage intervention usage data, patient-reported outcomes, and interview data and perform CNA to gain such an in-depth understanding, according to which we will consolidate the program theory.

    Step 1: Developing Program Theory

    Document Analysis

    We conducted a document analysis to develop the program theory using 2 data sources. First, we reviewed the program documentation, including the intervention design documents, RCT protocol, and data collection materials, to identify its intervention components, extract the corresponding features, understand the assumptions of the designers on how this multicomponent digital health intervention would lead to desired outcomes, and select the corresponding outcome indicators. Second, we reviewed published research articles on behavioral science theories, with special attention paid to digital behavior change interventions (DBCIs) and chronic disease self-management. The objective for reviewing pre-existing theories was twofold: (1) to inform the coding of intervention features, determine the key mechanisms, and identify the individual characteristics that may influence the implementation process; and (2) to select appropriate preexisting conceptual frameworks to inform the development of CIMO configurations, that is, build causal relationships among the constructs [53]. We focused on microlevel theories according to the study objective, that is, to understand the implementation process of a digital health intervention among individuals.

    Intervention Components and Features

    The KiT intervention [20] was developed using a user-centered design approach that engaged adolescents and emerging adults living with T1D as well as adult and pediatric diabetes providers. Specific intervention content was informed by clinician consultation and an environmental scan of diabetes transition resources mapped to domains of the “readiness of emerging adults with diabetes diagnosed in youth” (READDY) tool with the aim of improving the transition experiences among emerging adults living with T1D (Textbox 1). In addition to providing care coordination support (eg, appointment note-keeping and reminders), KiT is designed to provide personalized T1D informational support (eg, educational content and a question and answer feature) based on an individual’s interests and self-reported confidence about their diabetes knowledge and skills using a set of “if-then” rules (Multimedia Appendix 1).

    Textbox 1. Brief description of the digital intervention.

    Intervention name: Keeping in Touch (KiT)

    Delivery method: SMS chatbot

    Study population: Emerging adults living with type 1 diabetes (T1D) residing in Ontario or Quebec who are within 4 months of their planned transfer to adult care

    Intervention length: 12 months

    Intervention development methods: Based on user-centered design approaches (detailed information can be found [54])

    Intervention components:

    • Informational content (one topic per month):
      • Topics received by all users: 4 topics provided to all participants at months 1, 4, 7, and 10 respectively. Topics include coping with T1D, care navigation, sick day and ketone management, medical insurance and financial support.
      • Topics based on user needs and interests: 8 topics from a pool of 10 candidate topics provided at the rest of 8 months. Topics are determined by the participant’s baseline transition readiness status (self-reported confidence about diabetes knowledge and skills measured by the “readiness of emerging adults with diabetes diagnosed in youth” READDY tool) and their interests. Topics include hypoglycemia, pumps and pump programming, insulin adjustments, drugs and alcohol, travel, driving, school and work accommodations, exercise, nutrition and carbohydrates, and sexual health.
    • Question and Answer: KiT recognizes key words in participants’ T1D-related questions and automatically pulls resources from the KiT database to answer their question. Interaction with participants by periodically asking for feedback 5 minutes after providing question and answer responses.
    • Transition care coordination:
      • Reminder: KiT sends reminders for participants to book appointments and requests them to input their appointment information in the chatbot. KiT then sends appointment and bloodwork or urine test reminders at the participant’s preferred times.
      • Note-keeping: KiT allows participants to save a list of items to discuss in their appointments, which is sent back to them 1 hour before their scheduled appointments.
      • Care coordination support: KiT sends prompts to help participants prepare for appointments and reflect on their care experiences.
      • Clinic information: KiT sends information on the adult clinic that the participant will be attending, for clinics included within the chatbot database.

    We have identified intervention components and features (Table 1). By performing deductive coding according to existing literature, we extracted several common components and features of DBCIs, including personalization [55,56], problem-solving support [57-59], reminders [55], real-time interactivity [55,60], credible sources [57], user-friendly SMS text message tone [55], and diverse forms of information (enhanced media) [56]. We classified the remaining KiT components (eg, coping with T1D and care navigation) through inductive coding.

    Table 1. Keeping in Touch (KiT) intervention components and features.
    Intervention strategy and content

    		Timeline (month)Component or feature
    Informational content

    		Topics received by all users


    		Coping with T1D1
    • Stress management strategiesa


    		Care navigation4
    • Transition support informationa


    		Sick day and ketone management7
    • T1D self-management information and suggestionsa



    		Medical insurance and financial support10
    • T1D self-management information and suggestionsa

    		Topics based on user needs and interests (selected by baseline readiness status and personal preferences [once every 3 months])


    		Hypoglycemia2, 3, 5, 6, 8, 9, 11, 12
    • T1D self-management information and suggestionsa
    • Personalizationb [55,56]


    		Pumps and pump programming2, 3, 5, 6, 8, 9, 11, 12
    • T1D self-management information and suggestionsa


    		Insulin adjustments2, 3, 5, 6, 8, 9, 11, 12
    • Personalizationb [55,56]


    		Drugs and alcohol2, 3, 5, 6, 8, 9, 11, 12
    • T1D self-management information and suggestionsa


    		Travel2, 3, 5, 6, 8, 9, 11, 12
    • Personalizationb [55,56]


    		Driving2, 3, 5, 6, 8, 9, 11, 12
    • T1D self-management information and suggestionsa


    		School and work accommodations2, 3, 5, 6, 8, 9, 11, 12
    • Personalizationb [55,56]


    		Exercise2, 3, 5, 6, 8, 9, 11, 12
    • T1D self-management information and suggestionsa


    		Nutrition and carbohydrates2, 3, 5, 6, 8, 9, 11, 12
    • Personalizationb [55,56]


    		Sexual health2, 3, 5, 6, 8, 9, 11, 12
    • T1D self-management information and suggestionsa
    Question and answer

    		T1D self-management knowledge chatbot1-12
    • T1D self-management information and suggestionsa
    • Problem-solving supporta [57-59]
    • Real-time interactivityb [55,60]
    Transition care coordination

    		Reminders1-12
    • Transition remindersa [55]

    		Note-keeping1-12

    		Care coordination support1-12
    • Transition support informationa
    • Personalizationb [55,56]

    		Clinic information1-12
    • Transition support informationa
    • Personalizationb [55,56]
    Message content

    		Information topics and resources were selected based on credibility—c
    • Credible sourcesb [57]

    		Young-adult friendly educational text messages—
    • User-friendly message toneb [55]
    Message format

    		Educational information in forms including but not limited to text, graphics, images, and videos—
    • Diverse forms of information (enhanced media)b [56]
    Settings

    		Frequency of receiving education content—
    • Personalizationb [55,56]

    		Time of receiving messages—
    • Personalizationb [55,56]

    		Times of receiving appointment reminders—
    • Personalizationb [55,56]

    		Option to take a break from educational messages for 2 weeks (maximum 2 times throughout the intervention)—
    • Personalizationa [55,56]

    aIntervention component.

    bIntervention feature.

    cNot available.

    Initial Program Theory

    We have developed the initial program theory (Figure 2) by integrating 3 empirically validated individual-level behavioral change theories, including “capability, opportunity, motivation, and behavior” (COM-B) [61], “health action process approach” (HAPA) [62], and “technology acceptance model” (TAM) [63].

    ‎
    Figure 2. Initial program theory of the Keeping in Touch (KiT) intervention. COM-B: capability, opportunity, motivation, and behavior; DBCI: digital behavior change interventions; HAPA: health action process approach; T1D: type 1 diabetes; TAM: technology acceptance model.

    Informed by COM-B [61], we hypothesize that KiT may improve participants’ transition outcomes (ie, transition readiness, self-management behavior, and health outcomes) by triggering changes in individual motivation (ie, self-efficacy, negative emotions, outcome expectancies, behavioral cueing) and capability (ie, knowledge, skills). T1D self-management information and suggestions, transition support information, and problem-solving support may increase participants’ T1D and transition-related knowledge and skills [64], and therefore, capability [61]. T1D self-management information and suggestions [65] and problem-solving support [59] may improve one’s outcome expectancies and self-efficacy, respectively, which are the key motivational factors of an individual’s behavioral change [66]. One’s self-efficacy may also be improved by receiving stress management strategies [67]. Meanwhile, stress management strategies may lead to reduced negative emotions [58,67], which have been recognized as one of the most prominent motivational challenges to a successful T1D health care transition [14,68]. Transition reminders may function as behavior cueing to plan for and attend clinic appointments [69]. Well-designed intervention functions, such as personalization, credible sources, and diverse forms of information, may amplify the impact of KiT’s intervention content.

    The extent to which KiT can trigger changes in health care transition capability and motivation is hypothesized to be dependent on the level of user engagement [38,55,70,71]. Positive intervention outcomes (eg, improved transition readiness) may in turn incentivize increased user engagement [55], suggesting a positive feedback loop. According to TAM [63], user engagement is influenced by 2 core factors, that is, perceived usefulness and perceived ease of use [72], which may be influenced by intervention content and technological functions [73].

    Intervention effectiveness depends heavily on the interaction of the intervention with users and their context [55], yet there is a paucity of contextual insights in the DBCI literature, creating a gap in our understanding of which contextual factors influence the efficacy of intervention mechanisms in T1D and how [74]. According to the HAPA framework [62], behavioral determinants, such as self-efficacy [75], outcome expectancies [66,75], risk perception [65,76], and social support [66,77], impact both outcome and the intervention implementation process itself. Contextual factors, such as the complementary or substitute relationships between KiT and other conventional self-management tools (eg, handwritten methods) [38], may also predict user engagement alongside more traditional variables (ie, participant demographics [55,70]).

    Step 2: Testing Program Theory

    Participants

    Eligible participants will include English- or French-speaking emerging adults living with T1D enrolled in the RCT intervention arm who provided informed consent to be contacted for the embedded process evaluation at the time of enrollment in the RCT. We are targeting a conservative sample size of 25-30 participants for the interviews based on previous realist interviews and CNA studies [38,42,78,79].

    To increase the representativeness of the interview sample, we will purposively select participants with desired and undesired proximal outcomes of user engagement. This will be achieved by identifying “high-engagers” and “low- to medium-engagers” according to the intervention usage data at month 3 of the RCT. As there is no standard classification or threshold of DBCI engagement, we have used early KiT intervention usage data to identify a feasible definition of engagement. Considering the data availability, we conservatively define “high-engagers” as those with a 100% response rate to question prompts, with all other participants classified as “low- to medium-engagers.” All low- to medium-engagers will be invited for an interview. Should resource constraints not allow us to interview all high engagers, we will purposively sample to balance representation by gender and site of recruitment. The iterative sampling will continue until we reach thematic saturation (ie, no new themes are identified in the multisource data analysis), or all interested participants have been interviewed.

    Data Collection

    To test the initial program theory developed in step 1, we will perform semistructured realist interviews in a “teacher-learner cycle” [49]. The interviews will start with some general questions about the participants’ experiences managing T1D and engaging with KiT [80,81]. Based on their level of engagement and responses, the interviewer will act as a teacher to introduce the candidate CIMO configurations to the participant for their comments. After learning about the CIMO configurations, the participant will then act as teacher to confirm, extend, or refute the hypothesized pathways with their own examples of how they reacted to specific intervention components or features, what they saw as influencing their decision-making, and how this was perceived to affect their outcomes [80,82]. For low- to medium-engagers, the interview topics will focus on CIMO configurations regarding user engagement. For high engagers, the interview topics will focus on CIMO configurations regarding user engagement as well as distal outcomes (ie, transition readiness, self-blood glucose monitoring, and blood glucose). The interview guides (Multimedia Appendix 2) will first be piloted to ensure a comprehensive assessment of all relevant CIMO constructs. Interviews will be conducted by PhD-trained research coordinators with qualitative interview training, previous experience conducting semistructured interviews, and in-depth knowledge of the initial program theory. They will also receive study-specific training from the senior study lead, who is an experienced qualitative researcher. Research coordinators will have no previous relationship with study participants or the T1D community. Interviews will be transcribed verbatim by an independent third party.

    Data Analysis

    Transcripts will be coded using MAXQDA, a software for interview data analysis, and analyzed using the principles of thematic analysis strategies [82,83]. Data will be deductively coded and mapped to the predefined CIMO constructs to reflect triads (eg, context + intervention outcome, intervention mechanism outcome) and tetrads (context + intervention mechanism outcome) that would confirm, extend, or refute the current CIMO configurations [84]. Open coding will be applied when themes are identified that do not fit within the definitions of predefined CIMO constructs. The findings will be compared against the hypotheses developed in step 1, with a label of “supported,” “refined,” or “rejected” given to each hypothesized pathway based on the evidence.

    Several strategies will ensure the fidelity and credibility of the interview data, such as using multiple sources of data; creating a chain of evidence that documents all elements of the study database; having both broader research team members and interview participants participate in the triangulation analysis and the return of findings (construct and external validity); examining points of convergence and divergence within and among the data set (internal validity through cross-comparative analyses); and having a stepped analysis process with an initial independent review of the data by 2 reviewers who then meet to reach consensus around the common themes.

    Step 3: Consolidating Program Theory

    Data Integration and Factor Calibration

    We plan to triangulate information from realist interviews, intervention usage data, patient-reported outcomes, and a demographic survey as outlined in Table 2. Patient reported outcome and demographic information will be collected through RCT baseline and follow-up surveys. Intervention usage data will be collected by our third-party collaborator, Memotext, which delivers and manages the RCT intervention. Memotext’s system logs all incoming and outgoing messages, and they will be sharing them with the research team periodically.

    We will create a data set in which each interview participant is treated as a unique case and each CIMO element is included as a variable (as defined in the initial program theory outlined in Figure 2). Where variables are continuous (eg, self-efficacy, negative emotions, transition readiness, and blood glucose), we will include them directly. Where variables are qualitative (eg, knowledge and skills, outcome expectancies, behavioral cueing), we will convert them to data-driven categorical variables that will be defined by the research team and reported in the final manuscript.

    Table 2. Data collection plan for the realist evaluation (new variables may derive from the realist interviews).
    Categories and constructData sourceMeasureTime of measurement (months)
    Context (individual level)

    		Behavior determinants and contextual factors except demographicsInterviewSelf-reported influences on behavior and engagementTime of interview

    		DemographicsDemographic questionnaireGender, ethnicity, insurance type, etc.0
    Intervention componentsIntervention usage data and interviewNumber of messages sent by KiT chatbot and how the participant perceives each component

    		T1Da self-management information and suggestions

    		2, 3, 5, 6

    		Stress management strategies

    		1

    		Transition support information

    		4

    		Problem-solving support

    		1, 2, 3, 4, 5, 6

    		Transition reminders

    		1, 2, 3, 4, 5, 6
    Intervention featuresInterviewHow the participant perceives each featureTime of interview

    		Credible sources



    		Personalization



    		Real-time interactivity



    		User-friendly message tone



    		Diverse forms of information


    Mechanisms of action targeted by interventionInterviewSelf-reported behavior changes and perceived impact on targeted mechanisms of actionTime of interview

    		Knowledge and skills



    		Outcome expectancies



    		Behavioral cueing



    		Perceived ease of use



    		Perceived usefulness


    Proximal outcomes 

    		User engagementbIntervention usage dataResponse rate to question prompts1, 2, 3, 4, 5, 6

    		Self-efficacyb and negative emotionsbPatient-reported outcomesSEDMc and BDA Stigma Subscaled0, 6
    Distal outcomesPatient-reported outcomes
    		0, 6

    		Transition readiness
    		READDYe

    		Blood glucose
    		Self-reported HbA1cf

    		Self-blood glucose monitoring
    		Sensor use and additional measures of glycemia for glucose sensor user: whether the participant has been using a sensor for over 70% of the time during the past 14 days at the time of the survey.

    aT1D: type 1 diabetes.

    bFactors will be cross validated using interview data.

    cStanford Self-Efficacy for Diabetes Management.

    dBarriers to Diabetes Adherence in Adolescence Questionnaire Stigma Subscale.

    eReadiness of Emerging Adults With Diabetes Diagnosed in Youth.

    fHbA1c: hemoglobin A1c.

    Data Analysis

    We will conduct CNA using the R package “cna” to consolidate the initial program theory using a bottom-up, data-driven approach [85]. In order to identify the crucial difference-makers of outcomes from a large number of candidate-influencing factors (ie, context, intervention, and mechanism), we will screen the exogenous factors of each outcome before developing CNA models to demonstrate the causal relationships among them.

    To achieve factor reduction, we will begin with exploratory data analysis using a routine that operates within the same regularity framework as CNA. Specifically, we will apply the “minimally sufficient condition” (msc) function in the “cna” package to search across the entire data set (ie, all variables and all cases) at once to identify factors with the strongest connections to target outcomes, following the process outlined in Miech et al [86]. We will perform separate analyses in order to identify the minimally sufficient conditions for the presence of desired outcomes (ie, engagement and transition readiness) and those for the absence of desired outcomes, respectively [52,78,79]. For each outcome condition, we will run the “msc” routine 5 times at consistency thresholds of 0.95, 0.90, 0.85, 0.80, and 0.75, examining all 1-, 2-, and 3-condition configurations that meet consistency requirements; have the highest coverage score for their complexity level; and align with theory, background knowledge, case familiarity, and logic [78,87]. Through this process, we will identify a subset of factors to use in the subsequent modeling phase.

    In the modeling phase, we will iteratively develop preliminary and final models based on the following criteria: overall model consistency of 0.80 or greater; overall model coverage of 0.80 or greater; and aligns with theory, experience, background knowledge, and logic. Since CNA analyzes data in a bottom-up manner, such analysis will provide empirical findings that can be used to consolidate the initial program theory.

    Ethics Approval

    Ethics approval for this study was obtained from the Trillium Health Partners Research Ethics Board (ID: 1086). Ethics for the RCT were obtained from Clinical Trials Ontario through the Hospital for Sick Children Research Ethics Board (Project ID: 3986). The registration number for the larger RCT is NCT05434754.

    The participants for this realist evaluation will provide their written informed consent at the time of RCT enrollment. The realist evaluation research team will reach out to consenting RCT participants with a web-based survey link and letter of information with the intention of recruiting and scheduling an interview.

    This letter informs participants of their privacy and confidentiality protections, specifically that transcripts will be anonymized and that direct quotes used in reports or publications will not contain any information that could be used to identify them. A deidentified file with the study ID and associated participant information (ie, demographic information, transcript number, intervention usage data, and RCT patient reported outcome data) will be used to organize participant information and will only be accessible by the study coordinator. Only deidentified data will be used for subsequent analyses.

    Participants will receive a CAD $25 (US $18.50) electronic gift card for completing the interview.


    The KiT RCT commenced in January 2023. As of May 17, 2023, a total of 36 participants had been enrolled in the RCT. Of the 15 participants who have been assigned to the intervention arm, 13 consented to realist evaluation. Participant enrollment for this realist evaluation was anticipated to start in July 2023 and continue until we reach thematic saturation, or all interested participants have been interviewed. The study findings are planned to be disseminated through peer-reviewed publications and conference presentations in 2024.


    Project Findings and Significance

    It is widely agreed that digital health technologies have considerable potential to facilitate diabetes self-management and that user engagement is central to whether or not there is an impact [56]. This study will address several gaps in existing literature, including the identification of which specific features of a digital health intervention facilitate sustained engagement, which threshold level of user engagement leads to desired outcomes, and what individual-level contextual factors facilitate or hinder the implementation process [56,88]. This is a novel application of a realist evaluation to explore nuanced relationships in the context of digital health interventions and T1D health care transitions. It may contribute to knowledge from practical and methodological perspectives, respectively.

    While implementation strategies are increasingly informed by implementation science, evidence-based selection of combinations of strategies is often lacking [89]. This realist evaluation is conducted in parallel alongside the RCT implementation, which will enable us to inform real-time refinement of KiT by identifying the features that contribute to desired outcomes and demonstrating the mechanisms of action in real-world settings. Further, this contextualized understanding of whether and how KiT leads to change prioritizes the usefulness of information for decision-making by identifying enablers of and constraints on its delivery across a range of settings [37]. The findings may also shed light on the design, implementation, and evaluation of future digital health interventions that aim to improve transition experiences.

    This study will synthesize multiple data sources (ie, qualitative interviews, patient-reported outcomes, and intervention usage data) to enable rich complementary insights [90]. Moreover, this study uses CNA, a novel analytical approach underpinned by Boolean algebra. This approach systematically identifies a “minimal theory,” that is, the crucial combinations of factors linked to target outcomes [89]. By incorporating CNA, we will be able to evaluate empirically the theory-driven configurational models and refine the initial program theory as needed.

    Limitations

    First, we are not able to quantify user’s engagement relating to informational content (eg, active days of engagement for a specific module [30]) due to intervention constraints, limiting our ability to understand the value of specific educational elements. Moreover, due to the limited number of interactive SMS text messages, we are not able to measure the change in user engagement frequency using indicators such as the daily SMS text message response rate [91]. As an exploratory mitigation, we will use self-report data gathered by semistructured interviews. We will also use the available intervention usage data to create a composite variable (ie, response rate at 3 months) as a proxy to measure the general level of user engagement. Future studies may benefit from collecting user engagement data at the level of each intervention component and increasing the number of web-based texts to facilitate a more accurate measurement of user engagement. Second, we only consider individual-level contextual factors due to the study objective (ie, identifying target populations for the digital health intervention) and data availability. The omission of setting-level factors (eg, access to health care system, social norms) may limit our ability to comprehensively understand the complex conditions that influence emerging adults’ sustained engagement and health care transition experiences and therefore limit the transferability of our findings under some circumstances [55]. Third, data collection and analysis will occur in parallel alongside the RCT implementation in order to inform real-time intervention refinement. However, timing the study ahead of RCT end point outcome analysis will limit our ability to understand the impact on end point trial outcomes [90]. This was a conscious decision, as many RCTs show suboptimal results [71,92,93], suggesting a need to focus on upstream outcomes such as engagement as a first step.

    Conclusions

    Digital health interventions have emerged as a promising resource to support diabetes self-management capacity among emerging adults living with T1D. However, little is known about what components of these interventions are effective, how they are effective, and for whom they are most effective. Taking KiT as an example, our embedded realist evaluation will address this knowledge gap by using a mixed methods design and focusing on an important but often overlooked upstream outcome—user engagement. Besides informing real-time intervention refinements, the knowledge gained from this study may shed light on the design, implementation, and evaluation of future digital health interventions that aim to improve transition experiences.

    Acknowledgments

    This study is funded by Canadian Institutes of Health Research. The authors wish to thank Dr Edward J Miech for his professional support on the application of CNA in this study.

    Data Availability

    The data set of this study will be available from the corresponding author on reasonable request.

    Authors' Contributions

    LD and RS originated the idea for the project. LD and RW were primary drafters and editors of the manuscript. GR, GLB, A-SB, NE-D, MT, JAC, MG, MN, and RS contributed to manuscript revision. All authors have read and approved the final version of this manuscript.

    Conflicts of Interest

    RS has received speakers and advisory board fees from Dexcom. A-SB has received speaker fees from Dexcom. A-SB is a Fonds de recherche du Québec en Santé (FRQS) research scholar. MN is supported by an FRQS Senior Salary Award. The other authors declare that they have no competing interests.

    Multimedia Appendix 1

    KiT if-then logic.

    DOCX File , 40 KB
    Multimedia Appendix 2

    Interview guides.

    DOCX File , 27 KB
    1. Green A, Hede SM, Patterson CC, Wild SH, Imperatore G, Roglic G, et al. Type 1 diabetes in 2017: global estimates of incident and prevalent cases in children and adults. Diabetologia. 2021;64(12):2741-2750. [FREE Full text] [CrossRef] [Medline]
    2. International Diabetes Federation. IDF Diabetes Atlas—10th edition. International Diabetes Federation; 2021. URL: https://diabetesatlas.org/atlas/tenth-edition/ [accessed 2023-08-04]
    3. Reidy C, Foster C, Rogers A. A facilitated web-based self-management tool for people with type 1 diabetes using an insulin pump: intervention development using the behavior change wheel and theoretical domains framework. J Med Internet Res. 2020;22(5):e13980. [FREE Full text] [CrossRef] [Medline]
    4. Karlsson A, Arman M, Wikblad K. Teenagers with type 1 diabetes--a phenomenological study of the transition towards autonomy in self-management. Int J Nurs Stud. 2008;45(4):562-570. [CrossRef] [Medline]
    5. Brorsson AL, Bratt EL, Moons P, Ek A, Jelleryd E, Torbjörnsdotter T, et al. Randomised controlled trial of a person-centred transition programme for adolescents with type 1 diabetes (STEPSTONES-DIAB): a study protocol. BMJ Open. 2020;10(4):e036496. [FREE Full text] [CrossRef] [Medline]
    6. Foster NC, Beck RW, Miller KM, Clements MA, Rickels MR, DiMeglio LA, et al. State of type 1 diabetes management and outcomes from the T1D exchange in 2016-2018. Diabetes Technol Ther. 2019;21(2):66-72. [CrossRef] [Medline]
    7. Garvey KC, Telo GH, Needleman JS, Forbes P, Finkelstein JA, Laffel LM. Health care transition in young adults with type 1 diabetes: perspectives of adult endocrinologists in the U.S. Diabetes Care. 2016;39(2):190-197. [FREE Full text] [CrossRef] [Medline]
    8. Huang JS, Terrones L, Tompane T, Dillon L, Pian M, Gottschalk M, et al. Preparing adolescents with chronic disease for transition to adult care: a technology program. Pediatrics. 2014;133(6):e1639-e1646. [FREE Full text] [CrossRef] [Medline]
    9. Agarwal S, Raymond JK, Schutta MH, Cardillo S, Miller VA, Long JA. An adult health care-based pediatric to adult transition program for emerging adults with type 1 diabetes. Diabetes Educ. 2017;43(1):87-96. [CrossRef] [Medline]
    10. Garvey KC, Wolpert HA, Rhodes ET, Laffel LM, Kleinman K, Beste MG, et al. Health care transition in patients with type 1 diabetes: young adult experiences and relationship to glycemic control. Diabetes Care. 2012;35(8):1716-1722. [FREE Full text] [CrossRef] [Medline]
    11. Garvey KC, Foster NC, Agarwal S, DiMeglio LA, Anderson BJ, Corathers SD, et al. Health care transition preparation and experiences in a U.S. national sample of young adults with type 1 diabetes. Diabetes Care. 2017;40(3):317-324. [FREE Full text] [CrossRef] [Medline]
    12. Dovey-Pearce G, Christie D. Transition in diabetes: young people move on—we should too. Paediatr Child Health. 2013;23(4):174-179. [CrossRef]
    13. Los E, Ulrich J, Guttmann-Bauman I. Technology use in transition-age patients with type 1 diabetes: reality and promises. J Diabetes Sci Technol. 2016;10(3):662-668. [FREE Full text] [CrossRef] [Medline]
    14. Peters A, Laffel L, American Diabetes Association Transitions Working Group. Diabetes care for emerging adults: recommendations for transition from pediatric to adult diabetes care systems: a position statement of the American Diabetes Association, with representation by the American College of Osteopathic Family Physicians, the American Academy of Pediatrics, the American Association of Clinical Endocrinologists, the American Osteopathic Association, the Centers for Disease Control and Prevention, Children with Diabetes, The Endocrine Society, the International Society for Pediatric and Adolescent Diabetes, Juvenile Diabetes Research Foundation International, the National Diabetes Education Program, and the Pediatric Endocrine Society (formerly Lawson Wilkins Pediatric Endocrine Society). Diabetes Care. 2011;34(11):2477-2485. [FREE Full text] [CrossRef] [Medline]
    15. Holmes-Walker DJ, Llewellyn AC, Farrell K. A transition care programme which improves diabetes control and reduces hospital admission rates in young adults with type 1 diabetes aged 15-25 years. Diabet Med. 2007;24(7):764-769. [CrossRef] [Medline]
    16. Crowley R, Wolfe I, Lock K, McKee M. Improving the transition between paediatric and adult healthcare: a systematic review. Arch Dis Child. 2011;96(6):548-553. [CrossRef] [Medline]
    17. Schultz AT, Smaldone A. Components of interventions that improve transitions to adult care for adolescents with type 1 diabetes. J Adolesc Health. 2017;60(2):133-146. [CrossRef] [Medline]
    18. Lane JT, Ferguson A, Hall J, McElligott M, Miller M, Lane PH, et al. Glycemic control over 3 years in a young adult clinic for patients with type 1 diabetes. Diabetes Res Clin Pract. 2007;78(3):385-391. [CrossRef] [Medline]
    19. Vidal M, Jansa M, Anguita C, Torres M, Giménez M, Esmatjes E, et al. Impact of a special therapeutic education programme in patients transferred from a paediatric to an adult diabetes unit. European Diabetes Nursing. 2015;1(1):23-27. [FREE Full text] [CrossRef]
    20. Sanmugalingham G, Mok E, Cafazzo JA, Desveaux L, Brazeau AS, Booth GL, et al. Text message-based intervention, Keeping in Touch (KiT), to support youth as they transition to adult type 1 diabetes care: a protocol for a multisite randomised controlled superiority trial. BMJ Open. 2023;13(5):e071396. [FREE Full text] [CrossRef] [Medline]
    21. Rideout V, Robb MB. The common sense census: media use by tweens and teens, 2019. San Francisco, CA. Common Sense Media; 2019. URL: https:/​/www.​commonsensemedia.org/​sites/​default/​files/​research/​report/​2019-census-8-to-18-full-report-updated.​pdf [accessed 2023-08-04]
    22. World Health Organization. mHealth: New Horizons for Health through Mobile Technologies: Second Global Survey on eHealth. Geneva. World Health Organization; 2011.
    23. Cafazzo JA, Casselman M, Hamming N, Katzman DK, Palmert MR. Design of an mHealth app for the self-management of adolescent type 1 diabetes: a pilot study. J Med Internet Res. 2012;14(3):e70. [FREE Full text] [CrossRef] [Medline]
    24. Goyal S, Nunn CA, Rotondi M, Couperthwaite AB, Reiser S, Simone A, et al. A mobile app for the self-management of type 1 diabetes among adolescents: a randomized controlled trial. JMIR Mhealth Uhealth. 2017;5(6):e82. [FREE Full text] [CrossRef] [Medline]
    25. Frøisland DH, Arsand E, Skårderud F. Improving diabetes care for young people with type 1 diabetes through visual learning on mobile phones: mixed-methods study. J Med Internet Res. 2012;14(4):e111. [FREE Full text] [CrossRef] [Medline]
    26. Ng AH, Crowe TC, Ball K, Rasmussen B. A mHealth support program for Australian young adults with type 1 diabetes: a mixed methods study. Digit Health. 2019;5:2055207619882179. [FREE Full text] [CrossRef] [Medline]
    27. Han Y, Faulkner MS, Fritz H, Fadoju D, Muir A, Abowd GD, et al. A pilot randomized trial of text-messaging for symptom awareness and diabetes knowledge in adolescents with type 1 diabetes. J Pediatr Nurs. 2015;30(6):850-861. [FREE Full text] [CrossRef] [Medline]
    28. Butalia S, Crawford SG, McGuire KA, Dyjur DK, Mercer JR, Pacaud D. Improved transition to adult care in youth with type 1 diabetes: a pragmatic clinical trial. Diabetologia. 2021;64(4):758-766. [FREE Full text] [CrossRef] [Medline]
    29. Rami B, Popow C, Horn W, Waldhoer T, Schober E. Telemedical support to improve glycemic control in adolescents with type 1 diabetes mellitus. Eur J Pediatr. 2006;165(10):701-705. [CrossRef] [Medline]
    30. Böhm AK, Jensen ML, Sørensen MR, Stargardt T. Real-world evidence of user engagement with mobile health for diabetes management: longitudinal observational study. JMIR Mhealth Uhealth. 2020;8(11):e22212. [FREE Full text] [CrossRef] [Medline]
    31. Glasgow RE, Christiansen SM, Kurz D, King DK, Woolley T, Faber AJ, et al. Engagement in a diabetes self-management website: usage patterns and generalizability of program use. J Med Internet Res. 2011;13(1):e9. [FREE Full text] [CrossRef] [Medline]
    32. McHugh S, Tracey ML, Riordan F, O'Neill K, Mays N, Kearney PM. Evaluating the implementation of a national clinical programme for diabetes to standardise and improve services: a realist evaluation protocol. Implement Sci. 2016;11:107. [FREE Full text] [CrossRef] [Medline]
    33. Jackson SF, Kolla G. A new realistic evaluation analysis method. Am J Eval. 2012;33(3):339-349. [CrossRef]
    34. Gilmore B, McAuliffe E, Power J, Vallières F. Data analysis and synthesis within a realist evaluation: toward more transparent methodological approaches. Int J Qual Methods. 2019;18:1-11. [FREE Full text] [CrossRef]
    35. Ray P. The Science of Evaluation: A Realist Manifesto. London. SAGE Publications Ltd; 2013.
    36. Dalkin SM, Greenhalgh J, Jones D, Cunningham B, Lhussier M. What's in a mechanism? Development of a key concept in realist evaluation. Implement Sci. 2015;10:49. [FREE Full text] [CrossRef] [Medline]
    37. Moore GF, Audrey S, Barker M, Bond L, Bonell C, Hardeman W, et al. Process evaluation of complex interventions: medical research council guidance. BMJ. 2015;350:h1258. [FREE Full text] [CrossRef] [Medline]
    38. Desveaux L, Shaw J, Saragosa M, Soobiah C, Marani H, Hensel J, et al. A mobile app to improve self-management of individuals with type 2 diabetes: qualitative realist evaluation. J Med Internet Res. 2018;20(3):e81. [FREE Full text] [CrossRef] [Medline]
    39. Giralt AN, Nöstlinger C, Lee J, Salami O, Lallemant M, Ouma O, et al. Understanding the acceptability and adherence to paediatric antiretroviral treatment in the new formulation of pellets (LPV/r): the protocol of a realist evaluation. BMJ Open. 2017;7(3):e014528. [FREE Full text] [CrossRef] [Medline]
    40. Mirzoev T, Etiaba E, Ebenso B, Uzochukwu B, Manzano A, Onwujekwe O, et al. Study protocol: realist evaluation of effectiveness and sustainability of a community health workers programme in improving maternal and child health in Nigeria. Implement Sci. 2016;11(1):83. [FREE Full text] [CrossRef] [Medline]
    41. Greenhalgh T, Humphrey C, Hughes J, Macfarlane F, Butler C, Pawson R. How do you modernize a health service? A realist evaluation of whole-scale transformation in London. Milbank Q. 2009;87(2):391-416. [FREE Full text] [CrossRef] [Medline]
    42. Pham Q, Cafazzo JA, Feifer A. Adoption, acceptability, and effectiveness of a mobile health app for personalized prostate cancer survivorship care: protocol for a realist case study of the ned app. JMIR Res Protoc. 2017;6(10):e197. [FREE Full text] [CrossRef] [Medline]
    43. Desveaux L, Agarwal P, Shaw J, Hensel JM, Mukerji G, Onabajo N, et al. A randomized wait-list control trial to evaluate the impact of a mobile application to improve self-management of individuals with type 2 diabetes: a study protocol. BMC Med Inform Decis Mak. 2016;16(1):144. [FREE Full text] [CrossRef] [Medline]
    44. Skivington K, Matthews L, Simpson SA, Craig P, Baird J, Blazeby JM, et al. A new framework for developing and evaluating complex interventions: update of medical research council guidance. BMJ. 2021;374:n2061. [FREE Full text] [CrossRef] [Medline]
    45. De Weger E, Van Vooren NJE, Wong G, Dalkin S, Marchal B, Drewes HW, et al. What’s in a realist configuration? Deciding which causal configurations to use, how, and why. Int J Qual Methods. 2020;19:1-8. [FREE Full text] [CrossRef]
    46. Breton M, Smithman MA, Sasseville M, Kreindler SA, Sutherland JM, Beauséjour M, et al. How the design and implementation of centralized waiting lists influence their use and effect on access to healthcare—a realist review. Health Policy. 2020;124(8):787-795. [FREE Full text] [CrossRef] [Medline]
    47. Shams F, Wong JSH, Nikoo M, Outadi A, Moazen-Zadeh E, Kamel MM, et al. Understanding eHealth cognitive behavioral therapy targeting substance use: realist review. J Med Internet Res. 2021;23(1):e20557. [FREE Full text] [CrossRef] [Medline]
    48. Goicolea I, Vives-Cases C, Sebastian MS, Marchal B, Kegels G, Hurtig AK. How do primary health care teams learn to integrate Intimate Partner Violence (IPV) management? A realist evaluation protocol. Implement Sci. 2013;8:36. [FREE Full text] [CrossRef] [Medline]
    49. Manzano A. The craft of interviewing in realist evaluation. Evaluation. 2016;22(3):342-360. [CrossRef]
    50. Iannotti RJ, Schneider S, Nansel TR, Haynie DL, Plotnick LP, Clark LM, et al. Self-efficacy, outcome expectations, and diabetes self-management in adolescents with type 1 diabetes. J Dev Behav Pediatr. 2006;27(2):98-105. [CrossRef] [Medline]
    51. Williams KE, Bond MJ. The roles of self-efficacy, outcome expectancies and social support in the self-care behaviours of diabetics. Psychol Health Med. 2010;7(2):127-141. [CrossRef]
    52. Whitaker RG, Sperber N, Baumgartner M, Thiem A, Cragun D, Damschroder L, et al. Coincidence analysis: a new method for causal inference in implementation science. Implement Sci. Dec 11, 2020;15(1):108. [FREE Full text] [CrossRef] [Medline]
    53. Shearn K, Allmark P, Piercy H, Hirst J. Building realist program theory for large complex and messy interventions. Int J Qual Methods. 2017;16(1):1-11. [FREE Full text] [CrossRef]
    54. McCurdie T, Taneva S, Casselman M, Yeung M, McDaniel C, Ho W, et al. mHealth consumer apps: the case for user-centered design. Biomed Instrum Technol. 2012;46(S2):49-56. [FREE Full text] [CrossRef] [Medline]
    55. Perski O, Blandford A, West R, Michie S. Conceptualising engagement with digital behaviour change interventions: a systematic review using principles from critical interpretive synthesis. Transl Behav Med. 2017;7(2):254-267. [FREE Full text] [CrossRef] [Medline]
    56. Shan R, Sarkar S, Martin SS. Digital health technology and mobile devices for the management of diabetes mellitus: state of the art. Diabetologia. 2019;62(6):877-887. [FREE Full text] [CrossRef] [Medline]
    57. Garnett CV, Crane D, Brown J, Kaner EFS, Beyer FR, Muirhead CR, et al. Behavior change techniques used in digital behavior change interventions to reduce excessive alcohol consumption: a meta-regression. Ann Behav Med. 2018;52(6):530-543. [FREE Full text] [CrossRef] [Medline]
    58. Kebede MM, Liedtke TP, Möllers T, Pischke CR. Characterizing active ingredients of eHealth interventions targeting persons with poorly controlled type 2 diabetes mellitus using the behavior change techniques taxonomy: scoping review. J Med Internet Res. 2017;19(10):e348. [FREE Full text] [CrossRef] [Medline]
    59. Carey RN, Connell LE, Johnston M, Rothman AJ, de Bruin M, Kelly MP, et al. Behavior change techniques and their mechanisms of action: a synthesis of links described in published intervention literature. Ann Behav Med. 2019;53(8):693-707. [FREE Full text] [CrossRef] [Medline]
    60. Evans WD, Abroms LC, Poropatich R, Nielsen PE, Wallace JL. Mobile health evaluation methods: the Text4baby case study. J Health Commun. 2012;17(Suppl 1):22-29. [CrossRef] [Medline]
    61. Michie S, van Stralen MM, West R. The behaviour change wheel: a new method for characterising and designing behaviour change interventions. Implement Sci. 2011;6:42. [FREE Full text] [CrossRef] [Medline]
    62. Schwarzer R. Modeling health behavior change: how to predict and modify the adoption and maintenance of health behaviors. Appl Psychol. 2008;57(1):1-29. [CrossRef]
    63. Davis FD. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q. 1989;13(3):319-340. [CrossRef]
    64. Connell LE, Carey RN, de Bruin M, Rothman AJ, Johnston M, Kelly MP, et al. Links between behavior change techniques and mechanisms of action: an expert consensus study. Ann Behav Med. 2019;53(8):708-720. [FREE Full text] [CrossRef] [Medline]
    65. Scheerman JFM, van Empelen P, van Loveren C, van Meijel B. A mobile app (WhiteTeeth) to promote good oral health behavior among dutch adolescents with fixed orthodontic appliances: intervention mapping approach. JMIR Mhealth Uhealth. 2018;6(8):e163. [FREE Full text] [CrossRef] [Medline]
    66. Joseph RP, Ainsworth BE, Vega-López S, Adams MA, Hollingshead K, Hooker SP, et al. Rationale and design of smart walk: a randomized controlled pilot trial of a smartphone-delivered physical activity and cardiometabolic risk reduction intervention for African American women. Contemp Clin Trials. 2019;77:46-60. [FREE Full text] [CrossRef] [Medline]
    67. Grey M, Whittemore R, Jeon S, Murphy K, Faulkner MS, Delamater A, et al. TeenCope Study Group. Internet psycho-education programs improve outcomes in youth with type 1 diabetes. Diabetes Care. 2013;36(9):2475-2482. [FREE Full text] [CrossRef] [Medline]
    68. Iyengar J, Thomas IH, Soleimanpour SA. Transition from pediatric to adult care in emerging adults with type 1 diabetes: a blueprint for effective receivership. Clin Diabetes Endocrinol. 2019;5(1):3. [FREE Full text] [CrossRef] [Medline]
    69. Cronin RM, Hankins JS, Byrd J, Pernell BM, Kassim A, Adams-Graves P, et al. Modifying factors of the health belief model associated with missed clinic appointments among individuals with sickle cell disease. Hematol. 2018;23(9):683-691. [FREE Full text] [CrossRef] [Medline]
    70. Zhang S, Hamburger E, Kahanda S, Lyttle M, Williams R, Jaser SS. Engagement with a text-messaging intervention improves adherence in adolescents with type 1 diabetes: brief report. Diabetes Technol Ther. 2018;20(5):386-389. [FREE Full text] [CrossRef] [Medline]
    71. Simons D, De Bourdeaudhuij I, Clarys P, De Cocker K, Vandelanotte C, Deforche B. Effect and process evaluation of a smartphone app to promote an active lifestyle in lower educated working young adults: cluster randomized controlled trial. JMIR Mhealth Uhealth. 2018;6(8):e10003. [FREE Full text] [CrossRef] [Medline]
    72. Georgsson M, Staggers N. Patients' perceptions and experiences of a mHealth diabetes self-management system. Comput Inform Nurs. 2017;35(3):122-130. [CrossRef] [Medline]
    73. Kim AJ, Yang J, Jang Y, Baek JS. Acceptance of an informational antituberculosis chatbot among Korean adults: mixed methods research. JMIR Mhealth Uhealth. 2021;9(11):e26424. [FREE Full text] [CrossRef] [Medline]
    74. Kabongo EM, Mukumbang FC, Delobelle P, Nicol E. Explaining the impact of mHealth on maternal and child health care in low- and middle-income countries: a realist synthesis. BMC Pregnancy Childbirth. 2021;21(1):196. [FREE Full text] [CrossRef] [Medline]
    75. Yeager CM, Benight CC. Engagement, predictors, and outcomes of a trauma recovery digital mental health intervention: longitudinal study. JMIR Ment Health. 2022;9(5):e35048. [FREE Full text] [CrossRef] [Medline]
    76. Van Rhoon L, McSharry J, Byrne M. Development and testing of a digital health acceptability model to explain the intention to use a digital diabetes prevention programme. Br J Health Psychol. 2022;27(3):716-740. [FREE Full text] [CrossRef] [Medline]
    77. Burns K, Nicholas R, Beatson A, Chamorro-Koc M, Blackler A, Gottlieb U. Identifying mobile health engagement stages: interviews and observations for developing brief message content. J Med Internet Res. 2020;22(9):e15307. [FREE Full text] [CrossRef] [Medline]
    78. Rattray NA, Miech EJ, True G, Natividad D, Laws B, Frankel RM, et al. Modeling contingency in veteran community reintegration: a mixed methods approach. J Mix Methods Res. 2023;17(1):70-92. [FREE Full text] [CrossRef] [Medline]
    79. Winchester B, Cragun D, Redlinger-Grosse K, Walters ST, Ash E, Baldry E, et al. Application of motivational interviewing strategies with the extended parallel process model to improve risk communication for parents of children with familial hypercholesterolemia. J Genet Couns. 2022;31(4):847-859. [CrossRef] [Medline]
    80. Greenhalgh T, Pawson R, Wong G, Westhorp G, Greenhalgh J, Manzano A, et al. The realist interview. The RAMESES II Project. 2017. URL: https://www.ramesesproject.org/media/RAMESES_II_Realist_interviewing.pdf [accessed 2023-08-04]
    81. Westhorp G, Manzano A. Realist evaluation interviewing—A 'starter set' of questions. The RAMESES II Project. 2017. URL: https://www.ramesesproject.org/media/RAMESES_II_Realist_interviewing_starter_questions.pdf [accessed 2023-08-04]
    82. Mukumbang FC, Marchal B, Van Belle S, van Wyk B. Using the realist interview approach to maintain theoretical awareness in realist studies. Qual Res. 2019;20(4):485-515. [FREE Full text] [CrossRef]
    83. Kuckartz U, Rädiker S. Analyzing Qualitative Data with MAXQDA: Text, Audio, and Video. Gewerbestrasse. Springer; 2019.
    84. Handley M, Bunn F, Goodman C. Supporting general hospital staff to provide dementia sensitive care: a realist evaluation. Int J Nurs Stud. 2019;96:61-71. [FREE Full text] [CrossRef] [Medline]
    85. Baumgartner M, Thiem A. Identifying complex causal dependencies in configurational data with coincidence analysis. R J. 2015;7(1):176-184. [FREE Full text] [CrossRef]
    86. Miech EJ, Freitag MB, Evans RR, Burns JA, Wiitala WL, Annis A, et al. Facility-level conditions leading to higher reach: a configurational analysis of national VA weight management programming. BMC Health Serv Res. 2021;21(1):797. [FREE Full text] [CrossRef] [Medline]
    87. Petrik AF, Green B, Schneider J, Miech EJ, Coury J, Retecki S, et al. Factors influencing implementation of a colorectal cancer screening improvement program in community health centers: an applied use of configurational comparative methods. J Gen Intern Med. 2020;35(Suppl 2):815-822. [FREE Full text] [CrossRef] [Medline]
    88. Renmans D, Holvoet N, Criel B. No mechanism without context: strengthening the analysis of context in realist evaluations using causal loop diagramming. New directions for evaluation. 2020;2020(167):101-114. [CrossRef]
    89. Yakovchenko V, Morgan TR, Miech EJ, Neely B, Lamorte C, Gibson S, et al. Core implementation strategies for improving cirrhosis care in the veterans health administration. Hepatol. 2022;76(2):404-417. [CrossRef] [Medline]
    90. Coorey GM, Neubeck L, Usherwood T, Peiris D, Parker S, Lau AYS, et al. Implementation of a consumer-focused eHealth intervention for people with moderate-to-high cardiovascular disease risk: protocol for a mixed-methods process evaluation. BMJ Open. 2017;7(1):e014353. [FREE Full text] [CrossRef] [Medline]
    91. Nelson LA, Spieker A, Greevy R, LeStourgeon LM, Wallston KA, Mayberry LS. User engagement among diverse adults in a 12-month text message-delivered diabetes support intervention: results from a randomized controlled trial. JMIR Mhealth Uhealth. 2020;8(7):e17534. [FREE Full text] [CrossRef] [Medline]
    92. Agarwal P, Mukerji G, Desveaux L, Ivers NM, Bhattacharyya O, Hensel JM, et al. Mobile app for improved self-management of type 2 diabetes: multicenter pragmatic randomized controlled trial. JMIR Mhealth Uhealth. 2019;7(1):e10321. [FREE Full text] [CrossRef] [Medline]
    93. Desveaux L, Halko R, Marani H, Feldman S, Ivers NM. Importance of team functioning as a target of quality improvement initiatives in nursing homes: a qualitative process evaluation. J Contin Educ Health Prof. 2019;39(1):21-28. [FREE Full text] [CrossRef] [Medline]

    ‎
    CIMO: context-intervention-mechanism-outcome
    CMO: context-mechanism-outcome
    CNA: coincidence analysis
    COM-B: capability, opportunity, motivation, and behavior
    DBCI: digital behavioral changing intervention
    HAPA: health action process approach
    HbA1c: hemoglobin A1c
    KiT: Keeping in Touch
    msc: minimally sufficient condition
    RCT: randomized controlled trial
    READDY: Readiness of Emerging Adults With Diabetes Diagnosed in Youth
    T1D: type 1 diabetes
    TAM: technology acceptance model

    Edited by A Mavragani; submitted 31.01.23; peer-reviewed by S Stones, K De Cocker, N Alvarado; comments to author 27.04.23; revised version received 27.06.23; accepted 24.07.23; published 13.09.23.

    Copyright

    ©Ruoxi Wang, Geneviève Rouleau, Gillian Lynn Booth, Anne-Sophie Brazeau, Noor El-Dassouki, Madison Taylor, Joseph A Cafazzo, Marley Greenberg, Meranda Nakhla, Rayzel Shulman, Laura Desveaux. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 13.09.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.