This paper is in the following e-collection/theme issue:

Non-randomized Protocols and Methods (ehealth) (568) LGBTQ Issues (449) Violence including Domestic Violence/Abuse (121) Web-based and Mobile Health Interventions (3304) Design and Usability of Websites for Special User Groups (165) User-centred Design Case Studies (597)

Published on in Vol 10, No 4 (2021): April

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/26554, first published .
Generating Intervention Concepts for Reducing Adolescent Relationship Abuse Inequities Among Sexual and Gender Minority Youth: Protocol for a Web-Based, Longitudinal, Human-Centered Design Study

Generating Intervention Concepts for Reducing Adolescent Relationship Abuse Inequities Among Sexual and Gender Minority Youth: Protocol for a Web-Based, Longitudinal, Human-Centered Design Study

Generating Intervention Concepts for Reducing Adolescent Relationship Abuse Inequities Among Sexual and Gender Minority Youth: Protocol for a Web-Based, Longitudinal, Human-Centered Design Study

Authors of this article:

Robert W S Coulter1, 2, 3 Author Orcid Image ;   Shannon Mitchell1 Author Orcid Image ;   Kelly Prangley1 Author Orcid Image ;   Seth Smallwood1 Author Orcid Image ;   Leyna Bonanno1 Author Orcid Image ;   Elizabeth N Foster1 Author Orcid Image ;   Abby Wilson4 Author Orcid Image ;   Elizabeth Miller2, 3 Author Orcid Image ;   Carla D Chugani2, 3 Author Orcid Image
Article Authors Cited by (7) Tweetations (3) Metrics

    Protocol

    1Department of Behavioral and Community Health Sciences, Graduate School of Public Health, University of Pittsburgh, Pittsburgh, PA, United States

    2Department of Pediatrics, University of Pittsburgh School of Medicine, Pittsburgh, PA, United States

    3Division of Adolescent and Young Adult Medicine, UPMC Children's Hospital of Pittsburgh, Pittsburgh, PA, United States

    4LUMA Institute, Pittsburgh, PA, United States

    Corresponding Author:

    Robert W S Coulter, PhD, MPH

    Department of Behavioral and Community Health Sciences

    Graduate School of Public Health

    University of Pittsburgh

    130 DeSoto St

    Pittsburgh, PA, 15261

    United States

    Phone: 1 4126240647

    Email: robert.ws.coulter@pitt.edu


    Background: Sexual and gender minority youth (SGMY; eg, lesbian, gay, bisexual, and transgender youth) are at greater risk than their cisgender heterosexual peers for adolescent relationship abuse (ARA; physical, sexual, or psychological abuse in a romantic relationship). However, there is a dearth of efficacious interventions for reducing ARA among SGMY. To address this intervention gap, we designed a novel web-based methodology leveraging the field of human-centered design to generate multiple ARA intervention concepts with SGMY.

    Objective: This paper aims to describe study procedures for a pilot study to rigorously test the feasibility, acceptability, and appropriateness of using web-based human-centered design methods with SGMY to create novel, stakeholder-driven ARA intervention concepts.

    Methods: We are conducting a longitudinal, web-based human-centered design study with 45-60 SGMY (aged between 14 and 18 years) recruited via social media from across the United States. Using MURAL (a collaborative, visual web-based workspace) and Zoom (a videoconferencing platform), the SGMY will participate in four group-based sessions (1.5 hours each). In session 1, the SGMY will use rose-thorn-bud to individually document their ideas about healthy and unhealthy relationship characteristics and then use affinity clustering as a group to categorize their self-reported ideas based on similarities and differences. In session 2, the SGMY will use rose-thorn-bud to individually critique a universal evidence-based intervention to reduce ARA and affinity clustering to aggregate their ideas as a group. In session 3, the SGMY will use a creative matrix to generate intervention ideas for reducing ARA among them and force-rank the intervention ideas based on their potential ease of implementation and potential impact using an importance-difficulty matrix. In session 4, the SGMY will generate and refine intervention concepts (from session 3 ideations) to reduce ARA using round robin (for rapid iteration) and concept poster (for fleshing out ideas more fully). We will use content analyses to document the intervention concepts. In a follow-up survey, the SGMY will complete validated measures about the feasibility, acceptability, and appropriateness of the web-based human-centered design methods (a priori benchmarks for success: means >3.75 on each 5-point scale).

    Results: This study was funded in February 2020. Data collection began in August 2020 and will be completed by April 2021.

    Conclusions: Through rigorous testing of the feasibility of our web-based human-centered design methodology, our study may help demonstrate the use of human-centered design methods to engage harder-to-reach stakeholders and actively involve them in the co-creation of relevant interventions. Successful completion of this project also has the potential to catalyze intervention research to address ARA inequities for SGMY. Finally, our approach may be transferable to other populations and health topics, thereby advancing prevention science and health equity.

    International Registered Report Identifier (IRRID): DERR1-10.2196/26554

    JMIR Res Protoc 2021;10(4):e26554

    doi:10.2196/26554

    Keywords

    sexual and gender minorities; adolescent; psychosocial intervention; internet-based intervention; intimate partner violence 


    Background

    Sexual and gender minority youth (SGMY; eg, lesbian, gay, bisexual, and transgender youth) are at greater risk than their cisgender heterosexual peers for experiencing adolescent relationship abuse (ARA; ie, physical, sexual, or psychological abuse in a romantic relationship) [1-4]. According to a recent nationally representative sample of high school youth in the United States, 13% of sexual minority youth and only 7% of heterosexual youth reported physical ARA in the past year [5]. Even greater disparities are present for sexual ARA, with a prevalence of 16% among sexual minority youth and only 7% among heterosexual youth [5]. Gender minority youth are also at greater risk for ARA than their cisgender peers [6]. SGMY ARA inequities are problematic because ARA is associated with many poor health outcomes later in life, such as mental health disorders and HIV [7,8]. Thus, prevention efforts for SGMY may mitigate health inequities more broadly.

    Despite researchers and national agencies calling for interventions to reduce ARA among SGMY [9-11], there exist few evidence-based interventions addressing this public health inequity [9,12]. While there are several efficacious interventions for reducing ARA for the entire adolescent population [13], a 2019 systematic review revealed that there were no evidence-based ARA interventions specifically for SGMY at that time [12]. More recently, one study examined the efficacy of a universal intervention for reducing ARA among sexual minority youth; for sexual minority youth, this intervention reduced stalking victimization but not sexual violence, sexual harassment, and physical dating violence victimization [14]. More research is needed to address the lack of evidence-based interventions to prevent and reduce ARA in SGMY.

    One innovative method for stimulating new stakeholder-driven intervention ideas to catalyze ARA research among SGMY is to leverage the field of human-centered design [15]. Human-centered design is a discipline focused on improving existing or developing new products, services, or experiences by involving the perspectives of the target population at every possible stage [16-27]. Human-centered design methods often incorporate multiple ways of soliciting user and other stakeholder input, including through observation and dialog, cooperative design activities, and the shared creation of meaning by collaboratively synthesizing, critiquing, and ranking self-reported data and observations. In a practical sense, this often means that individuals involved in human-centered design processes create their own artifacts, including assembling, disassembling, and reassembling qualitative data points written on Post-it notes or a digital analog.

    When bringing stakeholders together to design interventions, human-centered design generally harnesses the strengths and limits the weaknesses of more traditional approaches (eg, focus groups). For example, focus groups traditionally use group-based discussions and interviews generally work with participants one-on-one. Human-centered design methods, however, often combine tasks to be completed in groups with tasks to be completed as individuals, thereby harnessing the strengths of focus groups and interviews [28,29]. Focus groups are also prone to groupthink (conformity of individuals working in groups, despite their individual differences, which can lead to inaccurate results or poor decision making [30]) that may inadvertently reinforce social hierarchies that silence certain people (eg, marginalized or shy people) [31-34]. In contrast, many human-centered design techniques require each participant to brainstorm independently and record all their ideas in written format (before sharing output with other group participants), making data collection more comprehensive and equitable [28,35]. In addition, focus groups are prone to social desirability bias because the moderator has an active role in guiding and influencing the discussion [32-34,36]. In human-centered design, facilitators have a less subjective role as they usually only lead participants through task instructions without any probing [28]. Finally, conventional methods (eg, focus groups or expert panels) for bringing stakeholders to translate research findings into intervention concepts are often challenging and time-intensive [37-41]. To overcome these barriers, human-centered design uses structured activities that are time-limited, engaging, and accessible to laypersons, including youth [16-27]. Overall, human-centered design activities can be applied as a novel form of stakeholder-engaged research to rapidly generate and iterate intervention concepts to reduce emergent public health problems.

    To date, health research that uses human-centered design methods has been predominantly conducted with stakeholders in-person [42], but most in-person research activities have been impeded by the COVID-19 pandemic. Although, the pandemic has presented many barriers for safely conducting in-person research, it has simultaneously catalyzed the use of web-based technologies for interacting and collaborating. Young people have especially become accustomed to using web-based technologies (eg, Zoom, a videoconferencing platform) because many schools have transitioned to remote learning. Therefore, youth are uniquely poised to use web-based human-centered design methods. Previous research has shown that youth can feasibly engage in web-based research as well as in-person human-centered design activities [43-47]. However, to our knowledge no study has explicitly tested whether human-centered design methods are feasible, acceptable, and appropriate for engaging with youth in a fully web-based environment. A study that rigorously pilot tests such methods, by setting a priori benchmarks, can help inform the public health field about the utility of conducting these methods on the web and further demonstrate and codify the use of web-based human-centered design as a method for stakeholder-engaged research.

    Study Aims

    This paper describes a protocol for conducting web-based human-centered design sessions with SGMY to create novel intervention ideas for addressing ARA. The specific aims of this study are as follows:

    1. Test the feasibility, acceptability, and appropriateness of conducting web-based human-centered design sessions with SGMY (primary study aim).

    2. Elucidate the beliefs of SGMY about healthy and unhealthy characteristics of intimate relationships.

    3. Elicit feedback from SGMY about the School Health Center Healthy Adolescent Relationships Program (SHARP), which is a universal evidence-based intervention for reducing ARA [48,49], and about adapted SHARP materials that are tailored to SGMY [50,51].

    4. Brainstorm intervention ideas for reducing ARA inequities for SGMY and force-rank the intervention ideas based on their potential ease of implementation and potential impact.

    5. Generate, iterate, vote on, and refine the intervention concepts to reduce ARA inequities for SGMY.


    Overview

    We followed both the Standards for Reporting Qualitative Research [52] and the Strengthening the Reporting of Observational Studies in Epidemiology statement [53] to craft our study, present our methods, and report our results. We used these two guidelines because there are no formal reporting requirements for human-centered design studies.

    Study Design

    We are conducting a web-based, longitudinal human-centered design study to engage with 45 to 60 SGMY participants in small groups to generate intervention concepts for reducing SGMY inequities in ARA. We conducted longitudinal sessions in multiple cohorts of 8 to 15 participants each. Participants completed a web-based screener, four web-based, group-based human-centered design sessions, and a web-based follow-up survey (Figure 1). This study is primarily funded by the National Center for Advancing Translational Sciences at the National Institutes of Health (UL1TR001857).

    ‎
    Figure 1. Study flow.
    View this figure

    Study Population

    We aim to enroll 45 to 60 sexual and gender minority high school students recruited via social media advertisements. Eligible youth are aged 14 years to 18 years, live in the United States, identify as sexual and/or gender minorities, and have internet, video camera, audio, and microphone access to attend the virtual sessions.

    Recruitment

    Participants are conveniently sampled and recruited throughout the United States using web-based advertisements posted on two social media platforms, Facebook and Instagram, using an approach similar to that in our previous research [54]. This approach allows SGMY from multiple geographic locations (eg, rural and urban areas and East and West) to enroll in the study without overextending our resources. Facebook is an appropriate recruitment platform because it is highly used by adolescents, approximately 71% of teens use Facebook [55]. Similarly, Instagram is used by approximately 72% of teens in the United States, with the majority using the site daily [56]. We created multiple photo and video advertisements for these sites, including those depicting youth with diverse gender expressions, race, and ethnicities.

    Screener Survey

    Upon clicking the advertisements, potential participants are redirected to a brief web-based self-reported screener survey administered via Research Electronic Data Capture (REDCap), a free and secure Health Insurance Portability and Accountability Act–compliant system for managing web-based surveys and databases. Following a brief description of the study, the screener included questions about potential participants’ age, race, ethnicity, sexual identity, sex assigned at birth, gender (including transgender status), high school name, city and state of their high school, computer access, camera access, audio access, microphone access, and contact information. All participants received the ARA and SGMY resource lists after the screener.

    Purposive Sampling

    After potential participants completed the screener survey, a research assistant will assess their eligibility. The research assistant will send a sociodemographically diverse group of eligible youth a web-based consent form, which must be completed to participate.

    Consent Process

    Potential participants are sent a link to a web-based consent form administered via DocuSign, a website that allows participants to safely and securely use a virtual signature. Youth in this study will consent for themselves because we obtained a waiver of parental consent. Our study is no more than minimal risk and requiring SGMY to obtain parental permission could out them as SGMY to their parents or guardians, which may put them at an increased risk of experiencing abuse. The consent form described all essential components of the study, including (but not limited to) the study purpose, the study background, study risks and benefits, privacy and confidentiality, participant payments, and voluntary nature of the study.

    Once the consent form is virtually signed, the research assistant receives a PDF version of the signed form and emails or texts participants a link to the Zoom videoconference meeting, where the web-based human-centered design sessions are conducted. The message includes instructions on how to best prepare for the session and how to access MURAL, the web-based collaborative workspace used during our web-based sessions. The participants are sent reminders 2-3 times before each session.

    In addition to the web-based consent form, participants provide verbal consent at the beginning of each web-based human-centered design session. A research assistant reads aloud a verbal consent script, asks if there are any questions, and then participants provide their consent by using Zoom’s thumbs up or via Zoom’s chat feature. In addition, before completing the follow-up survey, a brief consent script is provided to participants and participants voluntarily consented to taking the survey using a click-to-consent procedure.

    Human-Centered Design Activities

    We use MURAL and Zoom to conduct the web-based human-centered design activities. With each cohort of SGMY, we will conduct 4 sessions, each lasting 1.5 hours in length. All the sessions will be conducted in English. We will audio-record each session and take pictures of each session’s activity results. We only record the participants’ voices and the resultant data (not images of participants). The participants will receive a US $25 incentive for each session. Participants do not have to attend all sessions and can begin at any time, although we encourage attendance at all sessions because output from some sessions are then used as inputs in subsequent sessions.

    Each session begins with an introduction to Zoom, an icebreaker, and (in the first 2 sessions) a short lesson on the topic at hand (ie, ARA in SGMY). Next, the participants are randomly assigned to different Zoom breakout rooms, with each room composed of 2 to 5 participants with 1 or 2 facilitators each. In these small groups, the facilitators introduce participants to MURAL and guide the participants through a series of human-centered design activities. Each session’s human-centered design activities are outlined and briefly described in Table 1 and described in detail in the sections below. At the end of each meeting, all participants are brought back together and report on the ideas generated during the human-centered design activities.

    Table 1. Human-centered design activities by session.
    Session and activityPurpose
    Session 1

    		Rose, thorn, budTo have participants individually brainstorm healthy, unhealthy, and questionable aspects of intimate relationships for SGMYa

    		Affinity clusteringTo have participants discuss all the healthy, unhealthy, and questionable aspects of intimate relationships they generated, and to have participants group similar ideas together
    Session 2

    		Rose, thorn, budTo have participants provide feedback on the original SHARPb materials or the adapted SGMY-specific SHARP materials

    		Affinity clusteringTo have participants discuss all their feedback on the SHARP-related materials and to have participants group similar ideas together
    Session 3

    		Creative matrixTo have participants brainstorm intervention ideas at each level of the social-ecological model for reducing SGMY inequities in adolescent relationship abuse

    		Impact-difficulty matrixTo have participants plot their self-generated intervention ideas based on their potential ease of implementation and potential impact, thereby prioritizing the intervention ideas with the lowest potential resource expenditure and greatest potential impact
    Session 4

    		Round robinTo have participants evolve their intervention ideas into fuller intervention concepts using quick drafting and iteration via group authorship

    		Visualize the voteTo quickly poll SGMY’s preferences and opinions about two of their favorite intervention concepts

    		Concept posterTo have participants work together to refine intervention concepts by illustrating and describing its essential elements

    aSGMY: sexual and gender minority youth.

    bSHARP: School Health Center Healthy Adolescent Relationships Program.

    Session 1

    The purpose of our first session is to elucidate what SGMY believe are healthy and unhealthy characteristics of intimate relationships (study aim 2). We will accomplish this by using 2 human-centered design techniques: rose-thorn-bud and affinity clustering activities. Both activities are set up in a single MURAL workspace (Figure 2).

    ‎
    Figure 2. MURAL workspace setup for session 1.
    View this figure

    Rose-thorn-bud is a technique in which participants identify different aspects of a concept [20,21], in this case, healthy intimate relationships. Each participant is assigned a virtual desk (Figure 2) with 3 different colored sticky notes: pink, blue, and green. The facilitator explains the color-coding system to the participants. Participants are to type aspects of healthy relationships on pink sticky notes (or roses); they are then instructed to type components or aspects of unhealthy relationships on blue sticky notes (or thorns); and green sticky notes (or buds) are used to note aspects of relationships that are uncertain qualities with potential to be healthy and/or unhealthy. The facilitator shows participants how to create additional sticky notes and instructs the participants to write one idea per sticky note, generating as many sticky notes as possible. For 7 minutes, participants work independently to complete their sticky notes.

    Affinity clustering is used after completing the rose-thorn-bud activity to sort sticky notes according to their similarities and differences [20,21]. The facilitator asks a participant to read and explain one of their sticky notes (of any color) and the facilitator places it along the horizontal axis. The facilitator then asks another participant to describe one of their sticky notes and identify whether it is a similar or unique idea compared with the previously placed sticky note. Once determined, the facilitator then places this sticky note on the workspace below the previous sticky note if similar, or next to the previous sticky note if unique. This process is repeated until all sticky notes are in the workspace and are grouped according to the participants’ satisfaction. It is important to note that during affinity clustering, the facilitator encourages participants to align ideas based on its content, as opposed to the color of the sticky note. This type of clustering allows participants (as well as analysts who later view or interpret the data) to obtain 2 layers of meaning: the overall concepts of the groupings as well as what color sticky notes comprise those groupings. Overall, session 1 yields a visual representation of the SGMY’s mental models of intimate relationships.

    Session 2

    The purpose of session 2 is to acquire feedback from SGMY about the SHARP intervention materials as well as the adapted SGMY-specific SHARP materials (study aim 3) [48-51]. SHARP is a universal (not SGMY-specific) provider-based intervention implemented in routine school-based health center visits [48]. During clinic visits, SHARP providers introduce a palm-size brochure that contains information about healthy relationships and ARA resources, conduct ARA assessments, make referrals to ARA services, if necessary, and discuss healthy and unhealthy relationships with their patients. All SHARP materials were developed with input from numerous stakeholders, including clinicians, advocates, researchers, and youth. Compared with usual care, SHARP improves adolescents’ recognition of ARA, knowledge of ARA resources, and self-efficacy in using ARA harm reduction behaviors [48]. After the universal SHARP materials were created, investigators and stakeholders (including youth) generated SGMY-specific brochure materials tailored to SGMY [50,51]. We selected the SHARP intervention because (1) it is an illustrative example of an efficacious ARA prevention intervention and study participants may not be familiar with ARA interventions and (2) SGMY can provide feedback about the evidence-based intervention, which might confirm the applicability of the materials to SGMY in 2020 or offer ways to improve the brochure for current clinical practice or future research. We will have SGMY critique the SHARP brochure materials using two human-centered design techniques: rose-thorn-bud and affinity clustering activities. Both of these are set up in a single MURAL workspace (Figures 3 and 4).

    ‎
    Figure 3. MURAL Workspace Setup for Session 2 on Original School Health Center Healthy Adolescent Relationships Program (SHARP) Intervention.
    View this figure
    ‎
    Figure 4. MURAL Workspace Setup for Session 2 on Adapted School Health Center Healthy Adolescent Relationships Program (SHARP) Materials Tailored Specifically for Sexual Minority Youth.
    View this figure

    Rose-thorn-bud is a technique for having participants characterize different aspects of a concept (in this case, the SHARP intervention) as good, problematic, or having potential [20,21]. Each participant is assigned a virtual desk (Figures 3 and 4) with 3 different colored sticky notes: pink, blue, and green. The facilitator explained the color-coding system to the participants. Participants are asked to write the components and aspects of what they like about the SGMY intervention on pink sticky notes (or roses); they are then instructed to type components they dislike on blue sticky notes (or thorns); and green sticky notes (or buds) are used to note aspects that have the potential to be inclusive but could be improved. The facilitator shows participants how to create additional sticky notes and instructs the participants to write one idea per sticky note, generating as many sticky notes as possible. For 7 minutes, participants work independently to complete their sticky notes.

    Affinity clustering is used after completing the rose-thorn-bud activity to sort items according to their similarities and differences [20,21]. The facilitator asks one participant to read and explain one of their sticky notes (of any color) and the facilitator places it along the horizontal axis. The facilitator then asks another participant to describe one of their sticky notes and identify whether it is a similar or unique idea compared with the previously placed sticky note. Once determined, the facilitator then places this sticky note on the workspace below the previous sticky note if similar, or next to the previous sticky note if unique. This process is repeated until all sticky notes are in the workspace and are grouped according to the participants’ satisfaction.

    Session 3

    The purpose of session 3 is to brainstorm novel intervention components to reduce ARA inequities for SGMY and force-rank the intervention components based on their potential ease of implementation and potential impact (study aim 4). We accomplish this by using 2 human-centered design strategies: a creative matrix and an importance-difficulty matrix. All of these are conducted in one MURAL workspace (Figure 5).

    ‎
    Figure 5. MURAL workspace setup for session 3.
    View this figure

    The creative matrix is used to brainstorm as many intervention concepts for reducing ARA among SGMY as possible by leveraging the power of constraints (column by column) and a combination of goals (column headings) and enablers (essentially categories of potential solutions) [16,18,20,21,23,26,27]. As shown in Figure 5, the column headers indicate different levels of the social-ecological model, that is, individual, interpersonal, family and community, structural, and organizational determinants, whereas the row headings indicate the modalities: technology, events or programs, physical environments, policies or procedures, games or competitions, and wild card (ie, any other ideas that do not fit with the labeled categories). By providing different levels of the social-ecological model, SGMY are primed to think about creating intervention components that address known risk factors (eg, minority stressors [57]) and protective factors that contribute to ARA inequities for SGMY [57-63]. The activity is completed in 1 column at a time, with 4 minutes per column. Participants are asked to develop ideas (one idea per sticky note) for each cell or box by working independently. Participants are working individually at this time, reducing groupthink (eg, influence of more vocal contributors). In this activity, participants are encouraged to generate as many ideas as possible within a short period. Emphasis is typically placed on quantity (over quality) to eliminate barriers to contribution, get participants comfortable with the activity, and give them the opportunity to share ideas they may have already had before being prompted with the matrix and create space for new ideas informed by exploration of each row and column combination.

    After completing the creative matrix, participants engage in a structured activity to funnel the large number of ideas generated in the creative matrix into a manageable number of ideas to bring to the next activity. Facilitators ask participants to identify 4 sticky notes each: most important, most doable, their favorite, and the most out-of-the-box. These 4 criteria for down-selection were designed to ensure that ideas pursued in subsequent activities are sufficiently diverse. Each participant works independently for 3 minutes while they choose their ideas. The participants cannot choose the same ideas.

    An importance-difficulty matrix [20-22,26] is then used to force-rank the ideas identified in the previous step. First, participants are asked to collaboratively rank the ideas that they chose on the level of importance along the x-axis. Importance here is defined as how important or impactful the participants think the idea is to reduce ARA for SGMY. Next, they are asked to rank these ideas on the level of difficulty along the y-axis while retaining the same ordered ranking of likely impact or importance. The purpose of this activity is to identify the intervention concepts that are the most important to the participants and, given the inevitable resource constraints, account for their potential difficulty to implement.

    Session 4

    The purpose of session 4 is to generate, iterate, vote on, and refine intervention concepts to reduce ARA inequities for SGMY (study aim 5). We accomplish this by using three human-centered design strategies: round robin, visualize the vote, and concept posters. All of these are set up in one MURAL workspace (Figure 6).

    ‎
    Figure 6. MURAL workspace setup for session 4.
    View this figure

    In total, 5 of the potentially most impactful and easiest to design and implement ideas generated from the importance-difficulty matrix in session 3 are carried over to session 4 for the round robin activity. Round robin is used to evolve ideas into fuller intervention concepts using drafting and iteration among participants [20,21,24,25]. Each desk has 4 sections: problem statement (idea from previous session), proposed solution, why the solution might fail, and the final concept. This structure was developed by the LUMA Institute (see Figure 7 for a reader-friendly version of the round robin structure) [20,21]. Each participant starts at their own desk (a specific rectangle in the MURAL workspace) with the initial intervention concept. For 3 minutes, participants are asked to propose a solution to their intervention concept. Then, participants rotate workspaces by moving to the space to their right (or, if at the last desk, moving to the first desk). Then, participants are asked to provide feedback on why the next intervention concept might fail for 5 minutes. Participants move to the desk to their right one final time and for 7 minutes generate a final concept based on the feedback provided by the previous participants.

    ‎
    Figure 7. Example of one round robin setup in MURAL.
    View this figure

    Visualizing the vote is an activity that allows the facilitator to anonymously poll the participants and is a feature offered by MURAL [20,21]. After reading through the final round robin concepts, facilitators show participants where the voting ballots are and ask participants to vote for their 2 favorite concepts. Once this voting process is complete, 2 round robin concepts with the most votes overall are carried over to the final activity, the concept poster.

    The concept poster is used to illustrate and present the main points of the new intervention concepts [20,21]. The concept poster format, developed by the LUMA Institute [20,21], includes generating a creative title and identifying the following: the target population, the problem it solves, the big idea, how it works, why it might fail, what a prototype looks like, and how to measure success (see Figure 8 for a reader-friendly version of the concept poster activity). Facilitators first show the participants an example of a completed concept poster to illustrate the purpose of the concept poster. Participants are then invited to collaboratively develop 2 concept posters (allowing 15 minutes per poster). During this activity, facilitators answer participants’ questions and provide guidance on an as-needed basis.

    ‎
    Figure 8. Example of one concept poster setup in MURAL.
    View this figure
    Follow-Up Survey Data Collection

    After session 4, participants are asked to complete self-administered follow-up surveys via REDCap. The follow-up surveys assess participants’ perceptions about the feasibility, appropriateness, and acceptability of the web-based human-centered design session, as well as participants’ qualitative feedback on session logistics. The follow-up survey is activated and sent within 1 week after session 4. Surveys remain open for up to 2 weeks. The follow-up survey contains 3 pages, with a mean of 10 items per page (SD 6; range 6-17). While completing each survey, participants were able to change their answers by clicking a Back button. An incentive of US $10 was given after completion of the follow-up survey.

    Follow-Up Survey Measures

    To accomplish aim 1, we assess participants’ perceptions of the feasibility, acceptability, and appropriateness of the web-based human-centered design sessions.

    Feasibility is defined as the perception among participants that human-centered design sessions can be successfully implemented or carried out on the web [64]. Feasibility is measured using the Feasibility of Intervention Measure [64-66], which is a valid and internally consistent scale based on 4 positively worded items (eg, “The online sessions for this project were implementable”) that use a 5-point Likert scale ranging from completely disagree (1) to completely agree (5). We will create a mean across all items, with a higher mean score indicating greater feasibility.

    Acceptability is defined as the perception among stakeholders that web-based human-centered design sessions are satisfactory or agreeable [66]. Acceptability is measured using the Acceptability of Intervention Measure [65-67], which is a valid and internally consistent scale based on 4 positively worded items (eg, “The online sessions for this project seemed doable”) that use a 5-point Likert scale ranging from completely disagree (1) to completely agree (5). We will create a mean across all items, with a higher mean score indicating greater acceptability.

    Appropriateness is defined as the perception among participants that web-based human-centered design sessions are relevant, compatible, and suitable to address a specific issue [66]. Appropriateness is measured using the Intervention Appropriateness Measure [65], which is a valid and internally consistent scale based on 4 positively worded items (eg, “The online sessions for this project seemed applicable”) that use a 5-point Likert scale ranging from completely disagree (1) to completely agree (5). We will create a mean across all items, with a higher mean score indicating greater appropriateness.

    In addition to our standardized measures, we ask participants exploratory open-ended questions including about what they liked most and liked the least about the web-based suggestions as well as ways to improve the sessions and retention. We also asked closed-ended questions about their opinions about session length and frequency.

    Data Analyses

    For quantitative data analysis, we will use StataSE version 15 (StataCorp). For qualitative data analysis, we will transcribe, deidentify, and quality check all audio-recorded data from sessions 1 to 4 [68-71]. We will also export all session data from MURAL into text-based documents. We will then perform qualitative coding in Dedoose software [72], cross-referencing the audio transcriptions with PDF exports of the MURAL workspaces.

    Sample Characteristics

    Using data from the web-based screener survey, we will use frequencies and percentages to describe the demographics of our study sample.

    Analyses for Study Aim 1

    To assess the feasibility, acceptability, and appropriateness of conducting web-based human-centered design sessions with SGMY, we will calculate the means and 95% CIs for our primary outcome measures of feasibility, acceptability, and appropriateness. Our a priori benchmark for the success of meeting this hypothesis is obtaining a mean significantly higher than 3.75 (out of 5.00).

    Analyses for Study Aims 2 to 5

    To elucidate what SGMY believe are healthy and unhealthy characteristics of intimate relationships, we will use content analyses [73]. Using data from each session separately, we will code the artifact or audio-recording transcripts using an open-coding approach (ie, we will not create an a priori codebook, but will use in vivo coding instead) [68,69,74]. A total of 2 qualitative coders will independently read data from that session. Coders will convene to discuss preliminary findings and develop a draft codebook using inductive coding, allowing new codes to be included in the codebook as they emerge. Once the coders agree that all the major codes are identified, we will create a final codebook with definitions, rules, and examples for each code [70,71]. The 2 coders will then recode all data using the final codes. We will calculate the interrater reliability (ie, Kappa statistic) to examine code application between coders [75]. Coders will discuss any discrepancies until they reach an agreement; any disagreements will be discussed and resolved during research team meetings. We will use either a qualitative descriptive coding approach [76] (wherein we describe and count the number of code applications) or axial coding [77] (wherein we combine inductive codes into broader categories to define emerging patterns or themes). We will identify and interpret patterns in the data using thematic analysis, as informed by Braun and Clark [78].

    Sample Size and Power Calculation

    We calculate statistical power based on our primary outcomes (study aim 1), per the best practices for feasibility studies [64,79-84]. Given a conservative sample size of 45 participants and 5% type I error rate, we have the ability to estimate a 95% CI margin of error ≤0.43 for our primary outcomes, which we derived from the largest upper 95% CI limit of the SD measures in previous studies that used the same outcome measures [16-27].

    For our other study aims (study aims 2-5) about the outcomes of our human-centered design methods, the purpose is idea generation, not saturation [16-27]. Given our previous work implementing similar activities, a sample of 45 to 60 participants is sufficient to generate unique and useful ideas.

    Researcher Characteristics and Reflexivity

    Our team represents a wide range of sexual identities (ie, gay, lesbian, queer, bisexual, and heterosexual) and gender identities (ie, nonbinary, cisgender women, and cisgender men). We also comprise a multidisciplinary multitiered team, representing the fields of public health, counseling, human-centered design, library science, medicine, medical anthropology, political science, psychology, rehabilitation science, social work, and many roles in academia, including undergraduate, masters, research assistants, assistant professors, and full professors. Our diverse representation of sexual and gender identities complements our range of multidisciplinary expertise and professional backgrounds in informing data collection and analysis through lived experience.

    We acknowledge that there are several ways in which our backgrounds may have influenced our research. Primarily, our facilitators for the human-centered design sessions were sexual minorities and gender minorities. This could help put our SGMY participants at ease, especially in rare instances when the interviewer disclosed their sexual or gender minority identities to participants. Furthermore, the diversity and range of disciplines represented by our research team provide us with the ability to interpret our findings.

    Ethics Statement

    All study procedures were approved by the Human Research Protection Office of the University of Pittsburgh. To protect participants from having to reveal their sexual or gender minority identities to their caregivers, thereby potentially putting them in harm’s way, we received a waiver of parental consent [85]. This allowed the participants to provide consent for themselves. To further protect participants, we asked SGMY (before and during each session) to find a quiet and private space where they could participate in activities with minimal distraction and interruption. We also allowed participants to use Zoom’s chat function if they are unable or feel uncomfortable speaking out loud. This study was also protected by a Certificate of Confidentiality from the National Institutes of Health. It is important to note that we never inquired about participants’ personal experiences with ARA. Nevertheless, we provided all people who completed the screener with the ARA and SGMY resource lists. In addition, the study participants received resource lists after each session.


    This study was funded in February 2020. Data collection began in August 2020 and will be completed in April 2021. From August 2020 through December 2020, 778 individuals clicked the link to the screening questionnaire (Figure 9). In total, 370 individuals completed the screening questionnaire, of which 274 individuals met all eligibility criteria. A total of 50 participants were invited to participate, and 22 consented to participate. In total, 16 SGMY participated in at least one human-centered design session. All data collection will be completed by April 2021.

    ‎
    Figure 9. Flow diagram of the study as of December 2020.
    View this figure

    Impact

    This study protocol has several methodological innovations that can inform future public health research that aims to incorporate methods from the field of human-centered design. First, our method is an example of how human-centered design research can be performed on the web. This is important because it is a resource-friendly and accessible method for engaging diverse stakeholders from a wide geographic region. Moreover, the COVID-19 pandemic hindered our ability to engage safely with participants, and these web-based methods enable our research to continue despite current limitations on in-person research activities. Our study also demonstrates how to combine human-centered design, which is often quite flexible, into research processes, which tend to be much more structured. By rigorously testing the feasibility of such an approach, our study has the potential to demonstrate and codify the use of human-centered design as a novel stakeholder-engaged research method.

    Our study has the potential to lead to many substantive innovations in the field of ARA interventions among SGMY. If successful, our study could yield several novel intervention concepts. Importantly, these interventions are directly derived from SGMY themselves, as opposed to researchers. By centering youth voices and opinions in this manner, the generated interventions may be highly acceptable and impactful, but that cannot be determined until the interventions are tested further. Nevertheless, given the great lack of interventions, our study can help catalyze the field of SGMY ARA intervention research.

    Limitations

    Although there are many strengths to this study, it is not without limitations. Although participants in our sample are sociodemographically diverse, they are not necessarily representative. For example, SGMY without internet access are excluded from our study, and sessions were only conducted in English. In addition to youth, the inclusion of other stakeholders (eg, parents and school personnel) would likely have important insights for intervention concepts, but they are not included in this study. Despite our study yielding potentially new intervention concepts, this study will not produce complete interventions. Additional work from researchers, designers, and stakeholders will be necessary to develop and test the derived interventions. Finally, this is a pilot study testing the feasibility of our methods. We are not testing the effectiveness of our methods versus more traditional methods (eg, focus groups) for producing intervention concepts, which can be executed in future trials if our methodology proves to be feasible.

    Conclusions

    Interventions to reduce ARA among SGMY are lacking. To address this gap, our study investigates the feasibility of a new method for generating new intervention concepts. This work has the potential to contribute to substantive health impacts as well as immediate methodological impact by integrating human-centered design methods into public health research.

    Acknowledgments

    This research was supported by the National Institutes of Health through the National Center for Advancing Translational Sciences (grant UL1TR001857) and the National Institute on Alcohol Abuse and Alcoholism (grant K01AA027564 to RC). The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. The National Institutes of Health was not involved in the study design, writing of the protocol, or the decision to submit for publication.

    Conflicts of Interest

    None declared.

    1. Olsen EOM, Vivolo-Kantor A, Kann L. Physical and sexual teen dating violence victimization and sexual identity among U.S. high school students, 2015. J Interpers Violence 2020 Sep 29;35(17-18):3581-3600. [CrossRef] [Medline]
    2. Kann L, Olsen EO, McManus T, Harris WA, Shanklin SL, Flint KH, et al. Sexual identity, sex of sexual contacts, and health-related behaviors among students in grades 9-12 - United States and selected sites, 2015. MMWR Surveill Summ 2016 Aug 12;65(9):1-202. [CrossRef] [Medline]
    3. Martin-Storey A. Prevalence of dating violence among sexual minority youth: variation across gender, sexual minority identity and gender of sexual partners. J Youth Adolesc 2015 Jan 10;44(1):211-224. [CrossRef] [Medline]
    4. Luo F, Stone DM, Tharp AT. Physical dating violence victimization among sexual minority youth. Am J Public Health 2014 Oct;104(10):e66-e73. [CrossRef]
    5. Basile KC, Clayton HB, DeGue S, Gilford JW, Vagi KJ, Suarez NA, et al. Interpersonal violence victimization among high school students - Youth Risk Behavior Survey, United States, 2019. MMWR Suppl 2020 Aug 21;69(1):28-37 [FREE Full text] [CrossRef] [Medline]
    6. Dank M, Lachman P, Zweig JM, Yahner J. Dating violence experiences of lesbian, gay, bisexual, and transgender youth. J Youth Adolesc 2014 May 17;43(5):846-857. [CrossRef] [Medline]
    7. Miller E, McCaw B. Intimate partner violence. N Engl J Med 2019 Feb 28;380(9):850-857. [CrossRef]
    8. Savin-Williams RC. Verbal and physical abuse as stressors in the lives of lesbian, gay male, and bisexual youths: Associations with school problems, running away, substance abuse, prostitution, and suicide. J Consult Clin Psychol 1994 Apr;62(2):261-269. [CrossRef]
    9. Institute of Medicine. The health of lesbian, gay, bisexual, and transgender people building a foundation for better understanding (2011). Washington, DC: National Academies Press; 2011.
    10. Pérez-Stable EJ. Sexual and gender minorities formally designated as a health disparity population for research purposes. 2016.   URL: https://www.nimhd.nih.gov/about/directors-corner/message.html [accessed 2017-11-08]
    11. Healthy people 2020. U.S. Department of Health and Human Services.   URL: https://www.healthypeople.gov/2020 [accessed 2021-03-09]
    12. Coulter RWS, Egan JE, Kinsky S, Friedman MR, Eckstrand KL, Frankeberger J, et al. Mental health, drug, and violence interventions for sexual/gender minorities: a systematic review. Pediatrics 2019 Sep 19;144(3):- [FREE Full text] [CrossRef] [Medline]
    13. De Koker P, Mathews C, Zuch M, Bastien S, Mason-Jones AJ. A systematic review of interventions for preventing adolescent intimate partner violence. J Adolesc Health 2014 Jan;54(1):3-13. [CrossRef] [Medline]
    14. Coker AL, Bush HM, Clear ER, Brancato CJ, McCauley HL. Bystander program effectiveness to reduce violence and violence acceptance within sexual minority male and female high school students using a cluster RCT. Prev Sci 2020 Apr 06;21(3):434-444. [CrossRef] [Medline]
    15. Coulter RWS, Siconolfi DE, Egan JE, Chugani CD. Advancing LGBTQ health equity via human-centered design. Psychiatr Serv 2020 Feb 01;71(2):109. [CrossRef] [Medline]
    16. Abel TD. Design education: teaching design science by incorporating eye-tracking data into undergraduate visual communication design studios. 2015.   URL: https://www.designedasia.com/2014/Full_Papers/2014/8_Design%20Education.pdf [accessed 2021-03-09]
    17. Innovation Design Engineering Organization. The field guide to human-centered design. In: IDEO Design Kit; 1st edition. Canada: IDEO; 2015:1-192.
    18. Laverde K, Bouck C. A collaborative approach advances as a competency: learning professionals came together to explore the concept of having a global mindset and exchange ideas on how to embed it into their organizations. Association for Talent Development.: TD Magazine; 2014.   URL: https:/​/www.​td.org/​magazines/​td-magazine/​a-collaborative-approach-advances-global-mindset-as-a-competency [accessed 2019-05-25]
    19. Liedtka J, Ogilvie T, Brozenske R. The designing for growth field book: a step-by-step project guide. New York: Columbia Business School Publishing; 2014.
    20. Innovating for people: handbook of human-centered design methods. Pittsburgh, PA: LUMA Institute; 2012:1-86.
    21. A taxonomy of innovation. Harv Bus Rev. 2014.   URL: https://hbr.org/2014/01/a-taxonomy-of-innovation [accessed 2021-03-09]
    22. McCabe M. Collaborative ethnography in business environments. New York: Routledge; 2016:1-150.
    23. McKenna S, Mazur D, Agutter J, Meyer M. Design activity framework for visualization design. IEEE Trans Visual Comput Graphics 2014 Dec 31;20(12):2191-2200. [CrossRef]
    24. Partogi M, JafariNaimi N. Fostering organizational change through co-designing collaborative media. In: Proceedings of the 19th International Conference on Supporting Group Work. 2016 Presented at: GROUP '16: Proceedings of the 19th International Conference on Supporting Group Work; November, 2016; Sanibel Island Florida USA p. 441-444. [CrossRef]
    25. Paulus PB, Yang HC. Idea generation in groups: a basis for creativity in organizations. Organ Behav Hum Decis Process 2000 May;82(1):76-87. [CrossRef]
    26. Ronning B. Design thinking in organizations. 2018.   URL: http://www.benronning.com/writing/2018/8/23/design-thinking-in-organizations [accessed 2019-05-25]
    27. Witteman HO, Dansokho SC, Colquhoun H, Coulter A, Dugas M, Fagerlin A, et al. User-centered design and the development of patient decision aids: protocol for a systematic review. Syst Rev 2015 Jan 26;4(1):11 [FREE Full text] [CrossRef] [Medline]
    28. Beres LK, Simbeza S, Holmes CB, Mwamba C, Mukamba N, Sharma A, et al. Human-centered design lessons for implementation science: improving the implementation of a patient-centered care intervention. J Acquir Immune Defic Syndr 2019 Dec;82 Suppl 3(3):230-243 [FREE Full text] [CrossRef] [Medline]
    29. Marcus A, editor. Design, user experience, and usability: design thinking and methods. Switzerland: Springer; 2016.
    30. Park WW. A review of research on groupthink. J Behav Decis Making 2011 May 25;3(4):229-245. [CrossRef]
    31. Nyumba TO, Wilson K, Derrick CJ, Mukherjee N. The use of focus group discussion methodology: insights from two decades of application in conservation. Methods Ecol Evol 2018 Jan 11;9(1):20-32. [CrossRef]
    32. Acocella I. The focus groups in social research: advantages and disadvantages. Qual Quant 2011 Sep 3;46(4):1125-1136. [CrossRef]
    33. Huer MB, Saenz TI. Challenges and strategies for conducting survey and focus group research with culturally diverse groups. Am J Speech Lang Pathol 2003 May;12(2):209-220. [CrossRef]
    34. Halcomb EJ, Gholizadeh L, DiGiacomo M, Phillips J, Davidson PM. Literature review: considerations in undertaking focus group research with culturally and linguistically diverse groups. J Clin Nurs 2007 Jun;16(6):1000-1011. [CrossRef] [Medline]
    35. Liedtka J. Why design thinking works. Harv Bus Rev. 2018.   URL: https://hbr.org/2018/09/why-design-thinking-works [accessed 2021-03-09]
    36. Harvey SA. Observe before you leap: why observation provides critical insights for formative research and intervention design that you'll never get from focus groups, interviews, or KAP surveys. Glob Health Sci Pract 2018 May 23;6(2):299-316. [CrossRef]
    37. Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect 2014 Oct 19;17(5):637-650 [FREE Full text] [CrossRef] [Medline]
    38. Concannon TW, Fuster M, Saunders T, Patel K, Wong JB, Leslie LK, et al. A systematic review of stakeholder engagement in comparative effectiveness and patient-centered outcomes research. J Gen Intern Med 2014 Dec 4;29(12):1692-1701 [FREE Full text] [CrossRef] [Medline]
    39. Domecq JP, Prutsky G, Elraiyah T, Wang Z, Nabhan M, Shippee N, et al. Patient engagement in research: a systematic review. BMC Health Serv Res 2014 Feb 26;14(1):89 [FREE Full text] [CrossRef] [Medline]
    40. Franche RL, Baril R, Shaw W, Nicholas M, Loisel P. Workplace-based return-to-work interventions: optimizing the role of stakeholders in implementation and research. J Occup Rehabil 2005 Dec;15(4):525-542. [CrossRef] [Medline]
    41. Wallerstein N, Duran B. Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity. Am J Public Health 2010 Apr;100(S1):40-46. [CrossRef]
    42. Bazzano AN, Martin J, Hicks E, Faughnan M, Murphy L. Human-centred design in global health: a scoping review of applications and contexts. PLoS One 2017 Nov 1;12(11):- [FREE Full text] [CrossRef] [Medline]
    43. Lyon AR, Dopp AR, Brewer SK, Kientz JA, Munson SA. Designing the future of children's mental health services. Adm Policy Ment Health 2020 Sep 6;47(5):735-751 [FREE Full text] [CrossRef] [Medline]
    44. Kia-Keating M, Santacrose DE, Liu S, Adams J. Using community-based participatory research and human-centered design to address violence-related health disparities among Latino/a youth. Fam Community Health 2017;40(2):160-169 [FREE Full text] [CrossRef] [Medline]
    45. Dodds S, Hess AC. Adapting research methodology during COVID-19: lessons for transformative service research. J Serv Manag 2020 Aug 10;32(2):203-217. [CrossRef]
    46. Shrier LA, Burke PJ, Jonestrask C, Katz-Wise SL. Applying systems thinking and human-centered design to development of intervention implementation strategies: an example from adolescent health research. J Public Health Res 2020 Oct 14;9(4):1746 [FREE Full text] [CrossRef] [Medline]
    47. Nolan C, Packel L, Hope R, Levine J, Baringer L, Gatare E, et al. Design and impact evaluation of a digital reproductive health program in Rwanda using a cluster randomized design: study protocol. BMC Public Health 2020 Nov 13;20(1):1701 [FREE Full text] [CrossRef] [Medline]
    48. Miller E, Goldstein S, McCauley HL, Jones KA, Dick RN, Jetton J, et al. A school health center intervention for abusive adolescent relationships: a cluster RCT. Pediatrics 2015 Jan 22;135(1):76-85. [CrossRef] [Medline]
    49. Hanging out or hooking up: teen safety card. Futures Without Violence. 2020.   URL: https://www.futureswithoutviolence.org/hanging-out-or-hooking-up-teen-safety-card/ [accessed 2020-12-16]
    50. Trans/GNC? In a relationship? Caring relationships, healthy you safety card. Futures Without Violence. 2020.   URL: https://store.futureswithoutviolence.org/index.php/product/trans-gnc-safety-card/ [accessed 2020-12-16]
    51. Caring relationships, healthy you (LGBQ) safety card. Futures Without Violence. 2020.   URL: https://store.futureswithoutviolence.org/index.php/product/caring-relationships-lgbq-safety-card/ [accessed 2020-12-16]
    52. O’Brien BC, Harris IB, Beckman TJ, Reed DA, Cook DA. Standards for reporting qualitative research. Academic Medicine 2014;89(9):1245-1251. [CrossRef]
    53. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med 2007 Oct 16;147(8):573-577 [FREE Full text] [CrossRef] [Medline]
    54. Coulter RW, Sang JM, Louth-Marquez W, Henderson ER, Espelage D, Hunter SC, et al. Pilot testing the feasibility of a game intervention aimed at improving help seeking and coping among sexual and gender minority youth: protocol for a randomized controlled trial. JMIR Res Protoc 2019 Feb 15;8(2):- [FREE Full text] [CrossRef] [Medline]
    55. Lenhart A, Duggan M, Perrin A, Stepler R, Rainie L, Parker K. Teen, social media and technology overview 2015. Pew Research Center. 2015.   URL: http://www.pewinternet.org/2015/04/09/teens-social-media-technology-2015/ [accessed 2017-03-07]
    56. Anderson M, Jiang J. Pew Research Center. 2018.   URL: https://www.pewresearch.org/internet/2018/05/31/teens-social-media-technology-2018/ [accessed 2021-03-09]
    57. Meyer IH. Prejudice, social stress, and mental health in lesbian, gay, and bisexual populations: Conceptual issues and research evidence. Psychol Bull 2003 Sep;129(5):674-697 [FREE Full text] [CrossRef] [Medline]
    58. Ard KL, Makadon HJ. Addressing intimate partner violence in lesbian, gay, bisexual, and transgender patients. J Gen Intern Med 2011 Aug 30;26(8):930-933 [FREE Full text] [CrossRef] [Medline]
    59. Carvalho AF, Lewis RJ, Derlega VJ, Winstead BA, Viggiano C. Internalized sexual minority stressors and same-sex intimate partner violence. J Fam Violence 2011;26(7):501-509. [CrossRef]
    60. Gillum TL, DiFulvio G. "There's so much at stake": sexual minority youth discuss dating violence. Violence Against Women 2012 Jul 23;18(7):725-745. [CrossRef] [Medline]
    61. Kulkin HS, Williams J, Borne HF, de la Bretonne D, Laurendine J. A review of research on violence in same-gender couples: a resource for clinicians. J Homosex 2007 Sep;53(4):71-87. [CrossRef] [Medline]
    62. Stephenson R, Finneran C. Minority stress and intimate partner violence among gay and bisexual men in Atlanta. Am J Mens Health 2017 Jul 30;11(4):952-961 [FREE Full text] [CrossRef] [Medline]
    63. Adams BJ, Turner B, Wang X, Marro R, Miller E, Phillips G, et al. Associations between LGBTQ-affirming school climate and intimate partner violence victimization among adolescents. Prev Sci 2021 Feb 21;22(2):227-236. [CrossRef] [Medline]
    64. Bowen DJ, Kreuter M, Spring B, Cofta-Woerpel L, Linnan L, Weiner D, et al. How we design feasibility studies. Am J Prev Med 2009 May;36(5):452-457 [FREE Full text] [CrossRef] [Medline]
    65. Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci 2017 Aug 29;12(1):108 [FREE Full text] [CrossRef] [Medline]
    66. Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011 Mar 19;38(2):65-76 [FREE Full text] [CrossRef] [Medline]
    67. Henninger K. Exploring the relationship between factors of implementation, treatment integrity and reading fluency. Springer Nature 2010. [CrossRef]
    68. Saldaña J. The coding manual for qualitative researchers, 3rd ed. Thousand Oaks, California, United States: SAGE Publications Ltd; 2015.
    69. Miles MB, Huberman AM, Saldaña J. Qualitative data analysis: a methods sourcebook and the coding manual for qualitative researchers. Thousand Oaks, CA: SAGE Publications Ltd; 2014.
    70. MacQueen KM, McLellan E, Kay K, Milstein B. Codebook development for team-based qualitative analysis. Complement Alternat Med J 2016 Sep 14;10(2):31-36. [CrossRef]
    71. Fernald DH, Duclos CW. Enhance your team-based qualitative research. Ann Fam Med 2005 Jul 01;3(4):360-364 [FREE Full text] [CrossRef] [Medline]
    72. Dedoose Version 8.0.35, web application for managing, analyzing, and presenting qualitative and mixed method research data (2018). Sociocultural Research Consultants, LLC. 2018.   URL: https://www.dedoose.com/userguide/appendix [accessed 2021-03-09]
    73. Hsieh HF, Shannon SE. Three approaches to qualitative content analysis. Qual Health Res 2005 Nov;15(9):1277-1288. [CrossRef] [Medline]
    74. Thomas DR. A general inductive approach for qualitative data analysis. 2003.   URL: https://frankumstein.com/PDF/Psychology/Inductive%20Content%20Analysis.pdf [accessed 2021-03-16]
    75. Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas 2016 Jul 02;20(1):37-46. [CrossRef]
    76. Sandelowski M. Whatever happened to qualitative description? Res Nurs Health 2000 Aug;23(4):334-340. [CrossRef] [Medline]
    77. Thomas DR. A general inductive approach for analyzing qualitative evaluation data. Am J Eval 2016 Jun 30;27(2):237-246. [CrossRef]
    78. Braun V, Clarke V. Using thematic analysis in psychology. Qua. Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
    79. Kistin C, Silverstein M. Pilot studies: a critical but potentially misused component of interventional research. J Am Med Assoc 2015 Oct 20;314(15):1561-1562 [FREE Full text] [CrossRef] [Medline]
    80. Kraemer HC, Mintz J, Noda A, Tinklenberg J, Yesavage JA. Caution regarding the use of pilot studies to guide power calculations for study proposals. Arch Gen Psychiatry 2006 May 01;63(5):484-489. [CrossRef] [Medline]
    81. Leon AC, Davis LL, Kraemer HC. The role and interpretation of pilot studies in clinical research. J Psychiatr Res 2011 May;45(5):626-629 [FREE Full text] [CrossRef] [Medline]
    82. Moore CG, Carter RE, Nietert PJ, Stewart PW. Recommendations for planning pilot studies in clinical and translational research. Clin Transl Sci 2011 Oct;4(5):332-337 [FREE Full text] [CrossRef] [Medline]
    83. Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, et al. A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010 Jan 06;10(1) [FREE Full text] [CrossRef] [Medline]
    84. Tickle-Degnen L. Nuts and bolts of conducting feasibility studies. Am J Occup Ther 2013 Feb 22;67(2):171-176 [FREE Full text] [CrossRef] [Medline]
    85. Mustanski B. Ethical and regulatory issues with conducting sexuality research with LGBT adolescents: a call to action for a scientifically informed approach. Arch Sex Behav 2011 Aug 29;40(4):673-686. [CrossRef] [Medline]

    ‎
    ARA: adolescent relationship abuse
    REDCap: Research Electronic Data Capture
    SGMY: sexual and gender minority youth
    SHARP: School Health Center Healthy Adolescent Relationships Program

    Edited by G Eysenbach; submitted 18.12.20; peer-reviewed by M Kang, JM Suelves; comments to author 23.01.21; revised version received 31.01.21; accepted 25.02.21; published 12.04.21

    Copyright

    ©Robert W S Coulter, Shannon Mitchell, Kelly Prangley, Seth Smallwood, Leyna Bonanno, Elizabeth N Foster, Abby Wilson, Elizabeth Miller, Carla D Chugani. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 12.04.2021.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.