Development of a Web-Based Intervention for Addressing Distress in Caregivers of Patients Receiving Stem Cell Transplants: Formative Evaluation With Qualitative Interviews and Focus Groups

Introduction

Background

According to the 2015 National Alliance for Caregiving and the American Association of Retired Persons Public Policy Institute, there are “43.5 million adults in the United States that have provided unpaid care to an adult or child in the prior 12 months” [1]. Estimates from 2013 suggest that unpaid caregivers provide upward of $470 billion in care to their family members [2]. Additionally, missing work due to caregiving responsibilities leads to an estimated $25 billion in lost productivity [2]. While the impact of caregiving on the economy is significant, the emotional impact on the caregiver in terms of depression, anxiety, and distress is also substantial [2].

Caregivers of patients receiving hematopoietic stem cell transplants (HSCTs) are at risk of experiencing significant distress, including depression and anxiety [3,4], given the significant level of patient care that they need to provide. Patients receiving HSCTs require full-time care during the acute transplant and early post-transplant periods, as they face a compromised immune system and thus increased susceptibility to infections, as well as treatment side effects such as fatigue, insomnia, depression, anxiety, and sexual dysfunction [4-8]. As such, caregivers experience multifactorial stressors associated with caring for HSCT patients, leading to complex psychological outcomes that include depression and anxiety, and they are often reluctant to participate in support services given the extra time needed and the demands of their busy schedules. As a result, they can become “silent patients,” being so overburdened with caregiving responsibilities that they neglect to take care of themselves. Furthermore, when the stress of caregiving becomes chronic, caregivers can be at higher risk of developing health problems, which can lead to poorer care for their loved ones [9]. Given the bidirectional nature of the caregiver-patient relationship, helping caregivers manage their stress not only may have beneficial results for the caregiver but also may improve patient outcomes. Thus, convenient access to brief, evidence-based resources is needed to help caregivers manage the stress associated with caregiving [10].

A recent study showed that a brief in-person stress management intervention was effective in reducing the distress of caregivers of patients receiving allogenic HSCTs (allo-HSCTs) [3]. This intervention provided strategies for enhancing the sense of perceived control over stressors inherent to caregiving, understanding role changes, and improving communication during and after transplantation. Additionally, another study assessed caregiver and patient requirements for interactive health communication applications (IHCAs) targeting long-term conditions as well as their criteria for assessing the quality of different programs [11]. IHCAs combine high-quality health information with interactive components (ie, self-assessment tools, behavior change support, peer support, or decision support) and are largely web based. Participants in the study had a favorable view of the potential for Internet interventions to assist with long-term caregiving [11]. However, based on a recent systematic review of Internet-based interventions for caregivers, there are no evidence-based stress management interventions designed specifically for caregivers of HSCT and cancer patients that can be widely disseminated (eg, through technological platforms) [10]. Despite the paucity of evidence-based technological interventions that aim to aid caregivers in managing their stress, 74% of caregivers report being interested in technology that would help them to manage the emotional stress associated with caregiving [12].

Mobile technologies can provide a convenient mode of dissemination for evidence-based resources for managing the stress of caregiving [13]. In a recent study examining caregivers of patients with cancer, most caregivers had access to technological means (eg, Internet and email), and the majority of the caregivers sampled acknowledged that they would potentially utilize caregiver support tools disseminated via the Internet [13]. For overburdened caregivers, mobile technology can provide a valuable source of support for daily self-care that is accessible in their home environment. Mobile technologies have also been effectively implemented as a stand-alone treatment, which is ideal for caregivers with multiple demands who need to care for patients at home [14-23]. As such, the development of an intervention that can be accessed using popular technologies (eg, smartphones, tablets) provides an innovative way to disseminate reliable, empirically supported treatments to caregivers of medically compromised populations who have limited access to in-person support services. In addition, many interventions require heavy resource utilization. Compared to in-person interventions, mobile technologies require fewer personnel resources to disseminate evidence-based treatments and thus may be more cost effective.

Evidence-Based Intervention

Cognitive-behavioral stress management (CBSM) is a therapeutic intervention that focuses on teaching cognitive and behavioral strategies for coping with cancer [24]. Studies of patients with breast cancer have demonstrated the efficacy of CBSM, with participants in the CBSM intervention groups reporting increased optimism [24], reductions in symptoms of depression and increased ratings of quality of life [25], and improved psychological adjustment to illness [26]. Our previously reported intervention, PsychoEducation, Paced Respiration and Relaxation (PEPRR), was modeled after CBSM and effectively reduced perceived stress, anxiety, and depression in caregivers 3 months post-transplant [3]. However, as structured, the dissemination of that intervention was limited because it required caregivers to meet with a trained clinician in person, which necessitated travel time and schedule flexibility for appointments. Thus, PEPRR may be inaccessible or unacceptable to caregivers who are most in need, namely, those who are so overburdened with caregiving that they cannot participate in this in-person, evidence-based treatment.

Objective

The purpose of this study was to conduct formative research (eg, stakeholder groups) and pretesting (eg, focus groups and individual interviews) [27] to gather feedback on the look and feel, content, potential acceptability, anticipated usability, and feasibility of Pep-Pal, a mobilized, adapted version of PEPRR. The formative research was then used to inform the development of the final version of Pep-Pal and will be used to facilitate its dissemination.

Methods

For the purpose of this study, mock-up videos were models of videos (eg, prototypes) used to demonstrate the functionality of the Pep-Pal program and enable testing of the design with potential users. The mock-up videos were adapted based on previously suggested steps for developing and evaluating mHealth interventions [27]. This study followed these two steps: Step One, formative research with expert, patient, and caregiver stakeholder groups to develop a usable prototype to test with caregivers in the next step, and Step Two, pretesting of Pep-Pal mock-up videos with focus groups and individual interviews.

Setting

This study was conducted at the University of Colorado Denver Anschutz Medical Campus.

Step One: Formative Research With Expert, Patient, and Caregiver Stakeholder Groups

A Web-based animated video production service, GoAnimate, INC, was used to develop an initial prototype video session of Pep-Pal. This prototype session consisted of a brief, 10-minute psychoeducation video about stress and stress management delivered by an animated female character posing as a clinical psychologist with a human narrator. The video included content adapted from the PEPRR manual about understanding stress and how stress is experienced by caregivers as well as a brief description of how to manage stress [3]. The prototype video session ended with an animated caregiver character being guided through a body scan exercise. A variety of background music clips were used to enhance engagement.

We conducted three in-person stakeholder meetings, which included one professional stakeholder group (Group One, n=20) and two patient and caregiver stakeholder groups (Groups Two and Three, n=9 total). The professional stakeholder group was included to gather feedback about the acceptability and impressions of the program as well as the potential feasibility of disseminating the intervention. Institutional review board approval was not required for Step One because no demographic data were collected and because feedback was gathered during regular meetings (convenience sample) with experts, patients, and caregivers. Participants in the professional stakeholder group (Group One) included physicians, nurses, social workers, chaplains, and clinical psychologists from a local academic medical center who had expertise in palliative care. Participants in the other stakeholder groups (Groups Two and Three) included men and women between the ages of 30 and 65 who were caregivers of patients with various conditions, such as HSCT recipients. All stakeholder groups watched a 10-minute video example of a Pep-Pal session and provided open-ended feedback in a discussion format. Stakeholder meetings lasted approximately 60 minutes each.

In line with previous recommendations [27], feedback from the stakeholder groups was categorized into several domains: acceptability, anticipated usability, and feasibility. Acceptability was defined as the willingness of a user to use the program for its intended purpose. Anticipated usability was defined as the extent to which the intended audience could understand the program and find the program to be useful and meaningful. Feasibility was defined as the extent to which the program could be made readily available and implemented with the intended audience. In addition, feedback on the look, feel, and content was gathered to enhance acceptability.

Step Two: Pretesting of Pep-Pal Mock-Up Videos With Focus Groups and Individual Interviews

The goal of Step Two was to conduct focus groups and individual interviews with caregiver participants to assess the acceptability, anticipated usability, and feasibility of the Pep-Pal mock-up videos. Step Two of this study was approved by the University of Colorado Anschutz Medical Campus Institutional Review Board, and informed consent was obtained from all participants.

Procedure

Recruitment and eligibility for the focus groups and individual interviews included convenience sampling of caregivers of patients with auto-HSCTs; these caregivers were referred by the Bone Marrow Transplant Clinic and were recruited by phone. The inclusion criteria required participants to be caregivers of patients receiving auto-HSCTs, to be able to speak and read English, and to be at least 18 years of age. The exclusion criteria for caregivers included an absence of a psychiatric or medical condition preventing participation.

Focus Groups

A total of 6 Caucasian women who were spousal caregivers between the ages of 46 and 66 participated in the two focus groups. While 8 caregivers were recruited, 2 were unable to attend the focus groups due to their caregiving responsibilities. Notably, 1 caregiver who did not attend asked, “Is there something I can do online?”, supporting the convenience and accessibility of the intervention.

We conducted two semi-structured focus groups. The focus groups were 60 minutes each and were audiotaped. The caregiver participants in the focus groups watched two mock-up videos and provided opinions regarding the videos’ look and feel, content, acceptability, anticipated usability, and feasibility. The first mock-up video included an introduction to Pep-Pal, which provided instructions about the purpose of the Pep-Pal program and how to best utilize the program. The second mock-up video was entitled “Session One: Introduction to Stress Management” and described introductory coping skills for managing stress.

Individual Interviews

Based on the results of the focus groups, nine videos of full-length sessions were developed, covering the following topics: introduction to stress management, stress and the mind-body connection, how our thoughts can lead to stress (cognitive distortions), strategies for maintaining energy and stamina, coping with uncertainty, managing relations and coping with your needs, getting the support you need, and improving intimacy. Each Pep-Pal session was less than 10 minutes. Before conducting the individual interviews, two trained clinicians from the original PEPRR study [3] reviewed the Pep-Pal videos and independently both rated the videos as covering all topics from PEPRR. This step was integral to confirming the integrity of Pep-Pal’s adaptation from an evidence-based intervention that effectively reduced symptoms of anxiety, depression, and perceived stress in caregivers of allo-HSCT patients.

We conducted individual interviews with the caregivers of auto-HSCT patients to gather specific feedback on the nine video sessions as well as the overall Pep-Pal program. Participants completed a semi-structured interview that assessed five primary domains: (1) the look and feel of the video sessions, (2) the content addressed in the video sessions, (3) the anticipated usability of the video sessions, (4) the acceptability of the video sessions, and (5) the feasibility of the overall program. Participants were asked about their overall impressions of the sessions, what they may have found helpful or unhelpful in each session, their level of comfort in navigating through the videos, and whether they would consider watching the videos again to review the skills. They were also prompted to provide suggestions for any topics that needed improvement or needed to be added to better serve their needs. Each participant completed a 90-minute, semi-structured interview that consisted of watching an introductory video as well as three other sessions. The interviews were audiotaped and transcribed. At the beginning of the interview, participants were given the choice to watch the sessions on a laptop, tablet, or smartphone. Overall, 8 participants (8/9, 89%) chose to use a laptop, 0 participants used a smartphone, and 1 participant (1/9, 11%) used a tablet to view the videos. We analyzed the interviews after the entire data collection process was complete. Participants were white spousal caregivers (n=9) with a mean age of 59.3 and were predominantly women (n=7).

Data Analysis

Applied thematic analysis, which is a thorough yet inductive qualitative approach, was conducted for each interview to identify themes [27]. The first and second authors (NAP and TJ) independently reviewed the transcripts and discussed the themes emerging from the data. They then applied open coding to the transcripts and developed an initial codebook. After discussing the codebook and agreeing upon the codes, TJ coded all transcripts independently and finally identified broad themes within each domain. These themes were then used to further develop and refine the Pep-Pal program. Data saturation was reached when no new themes emerged regarding the look and feel, acceptability, anticipated usability, and feasibility.

Results

Step One: Results of Formative Research With Expert, Patient, and Caregiver Stakeholder Groups

Patients, caregivers, and professional stakeholders provided feedback regarding the look and feel, content, acceptability, anticipated usability, and feasibility of Pep-Pal in order to develop an improved prototype for Step Two. Feedback from stakeholders ranged from “I love what you are trying to do here to meet the needs of caregivers” to “this seems like one more thing for caregivers to do.” In addition, patients, caregivers, and expert stakeholders wanted more introductory information added to explain the benefits for caregivers. The stakeholders preferred a mix of animated and human delivery of information on-screen. Finally, technical feedback included “the music is too fast” and suggestions to provide more information on respite and other resources earlier in the intervention program. In response to the feedback, the program was modified to enhance its acceptability and anticipated usability. See Table 1 for a summary of the feedback from the stakeholder group meetings and the changes made to refine the Pep-Pal prototype.

Refining of Pep-Pal Prototype

Several changes were made to the prototype based on the feedback gathered from the three patient, caregiver, and expert clinician stakeholder meetings. These changes to the prototype then informed the pretesting. The most important addition was the creation of a separate introductory video to orient the caregiver to the purpose of the Pep-Pal program and to how best to utilize Pep-Pal. In terms of content and anticipated usability, instructions were added to the introductory video, explaining that Pep-Pal should not replace any in-person support; rather, Pep-Pal should be used in addition to professional support or in the interim, until further support can be accessed. In the video, caregivers were also encouraged to ask for more support from their providers as needed. To improve the look and feel, specific features of the main animated character in Pep-Pal were changed (eg, different hairstyle and smaller eyes). In addition, more human-delivered content was filmed based on the reported preferences of the stakeholders. In terms of content, the topic of Advance Care Planning was removed because it was deemed to be beyond the scope of this intervention.

Other modifications included changing the background music to have a more relaxing “feel”; slowing the pace of the videos; and adding content, including the recommended “dosage” of Pep-Pal (eg, watch each session at least once, for no more than two sessions per week initially, and then watch the videos in order of personal preference) and content encouraging caregivers to “take care of themselves first.” Major and minor changes resulted in the development of two prototype mock-up videos for pretesting in Step Two: (1) Introduction to Pep-Pal and (2) Introduction to Stress Management.

Step Two: Results From Focus Groups and Individual Interviews

Focus Groups

A summary of the major feedback from the Step Two focus groups is summarized in Table 2 (below). The feedback from the focus group participants was then integrated into the refinement and development of the nine Pep-Pal sessions to be tested in the individual interviews. The focus group participants were asked a series of questions about the program’s look and feel (eg, about the animated characters, music, length, and pace), anticipated usability (ie, satisfaction with the program and clarity of the instructions), and feasibility (ie, how they would use the program).

Individual Interviews

Themes were analyzed across each category for look and feel, content, acceptability, anticipated usability, and feasibility. Table 3 (below) summarizes the major themes and feedback gathered from the individual interviews.

Summary of Results From Individual Interviews

Overall, 9 participants, who were caregivers of patients who had undergone auto-HSCTs, participated in the individual semi-structured interviews and provided their thoughts and feedback on the Pep-Pal program. The interviews assessed domains including the look and feel, content, anticipated usability, acceptability, and feasibility. Different themes emerged, and the feedback gathered was further integrated to inform the development of the final version of the Pep-Pal program. A summary of the results of the qualitative interviews and changes implemented in the final version of the Pep-Pal program is provided in Table 4.

Discussion

Principal Results

The results from the stakeholder groups, focus groups, and individual interviews supported the acceptability, anticipated usability, and feasibility of Pep-Pal. Feedback was integrated into the final version of Pep-Pal. We found that the domains of usability, acceptability, and feasibility were strongly related to content; when the content resonated with participants, their ability to use the program, willingness to accept the program, and faith in the feasibility of the program increased.

We integrated feedback from Step One, encompassing formative research with expert, patient, and caregiver stakeholder groups, and Step Two, encompassing pretesting of the Pep-Pal mock-up videos with focus groups and individual interviews, into the final version of the Pep-Pal program, which will be tested in a pilot study in Step Three. This process was suggested by Whittaker et al [26] for the development and evaluation of mHealth interventions. Currently, Pep-Pal is available on a mobilized website (eg, can be viewed on a smartphone, laptop, computer, or tablet) that hosts the full-session videos, Mini-Pep videos, and information about the research team. Mini-Peps are brief, 3-minute exercises (eg, related to relaxation, mood boosting, or relationship enhancement) that can be conveniently accessed if the caregiver does not have time to watch a full-session video. Mini-Peps were included in the full-session videos during the formative development study described here. The website is password protected and available only as part of the ongoing pilot RCT to assess the preliminary estimates of efficacy and obtain further information regarding acceptability, anticipated usability, and feasibility. A total of 60 caregivers of patients with advanced illness will be enrolled in this study.

Limitations

Several limitations were present. First, the patient and caregiver stakeholders were volunteers as part of research advisory groups and thus may represent participants who are more motivated than hard-to-reach or more vulnerable caregiver populations. Second, the developer of the Pep-Pal intervention coordinated the stakeholder groups, focus groups, and individual interviews, so participants may have been reluctant to provide more critical feedback. Third, all participants were white spousal caregivers, which may limit generalizability. Fourth, due to time constraints, detailed demographic data, including socioeconomic status and education level, were not gathered from all participants, thus further limiting the generalizability of this study. However, future studies will gather detailed demographic information. Fifth, the focus group sample size was smaller than anticipated (n=6 per group), and each group was composed of white women, introducing bias as a consequence of the selection of respondents. Finally, participants could not watch all sessions due to time limitations, and thus, saturation was not reached in terms of content. In the corresponding RCT, the developer of Pep-Pal will not conduct the qualitative interviews, the quantitative assessments will be completed on the Web, participants will be representative of more hard-to-reach caregiver populations, and participants will have access to all videos.

Future Directions

Pep-Pal is being testing in a pilot RCT with caregivers of patients with advanced illness (eg, patients with illness warranting an HSCT, patients with advanced cancer, and those in Phase I oncology trials) to determine the acceptability and feasibility of and preliminary efficacy estimates for Pep-Pal to reduce symptoms of anxiety, depression, perceived stress, and sexual dysfunction. Ultimately, the goal is to conduct a multi-site efficacy RCT of Pep-Pal with caregivers to facilitate future widespread dissemination of Pep-Pal.