This paper is in the following e-collection/theme issue:

Formative Studies and eHealth/mHealth Development (515) Public (e)Health, Digital Epidemiology and Public Health Informatics (641) Prevention and Health Promotion (895)

Published on in Vol 5, No 4 (2016): Oct-Dec

Internet-Based Delivery of Evidence-Based Health Promotion Programs Among American Indian and Alaska Native Youth: A Case Study

Internet-Based Delivery of Evidence-Based Health Promotion Programs Among American Indian and Alaska Native Youth: A Case Study

Internet-Based Delivery of Evidence-Based Health Promotion Programs Among American Indian and Alaska Native Youth: A Case Study

Authors of this article:

Christine M Markham1 Author Orcid Image ;   Stephanie Craig Rushing2 Author Orcid Image ;   Cornelia Jessen3 Author Orcid Image ;   Gwenda Gorman4 Author Orcid Image ;   Jennifer Torres1 Author Orcid Image ;   William E Lambert5 Author Orcid Image ;   Alexander V Prokhorov6 Author Orcid Image ;   Leslie Miller7 Author Orcid Image ;   Kelly Allums-Featherston8 Author Orcid Image ;   Robert C Addy1 Author Orcid Image ;   Melissa F Peskin1 Author Orcid Image ;   Ross Shegog1 Author Orcid Image
Article Authors Cited by (13) Tweetations (1) Metrics

    Original Paper

    1Department of Health Promotion and Behavioral Sciences, School of Public Health, University of Texas Health Science Center at Houston, Houston, TX, United States

    2Northwest Portland Area Indian Health Board, Portland, OR, United States

    3Alaska Native Tribal Health Consortium, Anchorage, AK, United States

    4Inter Tribal Council of Arizona, Inc., Phoenix, AZ, United States

    5Oregon Health & Science University, Portland, OR, United States

    6The University of Texas MD Anderson Cancer Center, Houston, TX, United States

    7Rice University, Houston, TX, United States

    8The Cooper Institute, Dallas, TX, United States

    Corresponding Author:

    Christine M Markham, PhD

    Department of Health Promotion and Behavioral Sciences

    School of Public Health

    University of Texas Health Science Center at Houston

    7000 Fannin St., Rm 2622

    Houston, TX, 77030

    United States

    Phone: 1 713 500 9646

    Fax:1 713 500 9602

    Email: Christine.Markham@uth.tmc.edu


    Background: American Indian and Alaska Native (AI/AN) youth face multiple health challenges compared to other racial/ethnic groups, which could potentially be ameliorated by the dissemination of evidence-based adolescent health promotion programs. Previous studies have indicated that limited trained personnel, cultural barriers, and geographic isolation may hinder the reach and implementation of evidence-based health promotion programs among AI/AN youth. Although Internet access is variable in AI/AN communities across the United States, it is swiftly and steadily improving, and it may provide a viable strategy to disseminate evidence-based health promotion programs to this underserved population.

    Objective: We explored the potential of using the Internet to disseminate evidence-based health promotion programs on multiple health topics to AI/AN youth living in diverse communities across 3 geographically dispersed regions of the United States. Specifically, we assessed the Internet’s potential to increase the reach and implementation of evidence-based health promotion programs for AI/AN youth, and to engage AI/AN youth.

    Methods: This randomized controlled trial was conducted in 25 participating sites in Alaska, Arizona, and the Pacific Northwest. Predominantly AI/AN youth, aged 12-14 years, accessed 6 evidence-based health promotion programs delivered via the Internet, which focused on sexual health, hearing loss, alcohol use, tobacco use, drug use, and nutrition and physical activity. Adult site coordinators completed computer-based education inventory surveys, connectivity and bandwidth testing to assess parameters related to program reach (computer access, connectivity, and bandwidth), and implementation logs to assess barriers to implementation (program errors and delivery issues). We assessed youths’ perceptions of program engagement via ratings on ease of use, understandability, credibility, likeability, perceived impact, and motivational appeal, using previously established measures.

    Results: Sites had sufficient computer access and Internet connectivity to implement the 6 programs with adequate fidelity; however, variable bandwidth (ranging from 0.24 to 93.5 megabits per second; mean 25.6) and technical issues led some sites to access programs via back-up modalities (eg, uploading the programs from a Universal Serial Bus drive). The number of youth providing engagement ratings varied by program (n=40-191; 48-60% female, 85-90% self-identified AI/AN). Across programs, youth rated the programs as easy to use (68-91%), trustworthy (61-89%), likeable (59-87%), and impactful (63-91%). Most youth understood the words in the programs (60-83%), although some needed hints to complete the programs (16-49%). Overall, 37-66% of the participants would recommend the programs to a classmate, and 62-87% found the programs enjoyable when compared to other school lessons.

    Conclusions: Findings demonstrate the potential of the Internet to enhance the reach and implementation of evidence-based health promotion programs, and to engage AI/AN youth. Provision of back-up modalities is recommended to address possible connectivity or technical issues. The dissemination of Internet-based health promotion programs may be a promising strategy to address health disparities for this underserved population.

    Trial Registration: Clinicaltrials.gov NCT01303575; https://clinicaltrials.gov/ct2/show/NCT01303575 (Archived by WebCite at http://www.webcitation.org/6m7DO4g7c)

    JMIR Res Protoc 2016;5(4):e225

    doi:10.2196/resprot.6017

    Keywords

    adolescents; American Indian; Alaska Native; health promotion; Internet 


    As of 2012, an estimated 5.2 million individuals in the United States identified as American Indian and Alaska Native (AI/AN) alone or combined with other races, comprising 2% of the nation’s population [1]. The AI/AN population is diverse, with 567 federally recognized tribes; 60% of AI/AN individuals live in metropolitan areas and 22% live on reservations or other trust lands [1]. The AI/AN population is younger and faster growing than other US racial groups, with 30% under 18 years of age, making adolescent health a major priority [1]. Nationally, AI/AN youth are more likely to engage in sexual risk and substance use behaviors than their racial/ethnic peers [2], contributing to health disparities in teen births, sexually transmitted infections, unintentional injuries, and suicide. In 2014, AI/AN females aged 15-19 years had the third highest teen birth rate in the United States (27.3 per 1000 vs 38.0 for Hispanics, 34.9 for blacks, and 17.3 for whites) [3]; however, AI/AN females represent the highest prevalence of repeat teen births (21.6% versus 20.9% for Hispanics, 20.4% for blacks, and 14.8% for whites) [4]. Furthermore, in 2011 AI/AN females aged 15-24 years reported the highest age-specific rates of chlamydial infections among US women [5]. Between 1999 and 2009, AI/AN youth aged 15-19 were more than twice as likely as white youth to be at risk for unintentional injuries, and three times as likely to be at risk for suicide [6].

    AI/AN youth also face disparities related to obesity and diabetes. Previous studies have indicated that 20-30% of AI/AN children are obese [7], compared to 17% of children nationally [8]. Furthermore, in 2001 AI/AN youth aged 10-19 years were nine times more likely to be diagnosed with type 2 diabetes compared to non-Hispanic whites (1.74 per 1000 vs 0.19 per 1000) [9]. Sequela from these health conditions often carry over into adulthood, making heart disease and cancer the leading causes of death among AI/AN adults [10]. AI/AN adults also experience higher rates of hearing loss than other racial/ethnic groups [11].

    These health disparities could potentially be ameliorated by the dissemination and implementation of evidence-based adolescent health promotion programs. The United States Department of Health and Human Services’ Office of Adolescent Health, Substance Abuse and Mental Health Services Administration, and Centers for Disease Control and Prevention (CDC) have identified adolescent health promotion programs proven to impact behavior change across various health domains, including sexual health, violence and substance use prevention, physical activity, and nutrition [12-14]. However, the public health impact of health promotion programs depends both upon their efficacy and their reach [15]. To be impactful, evidence-based programs must have sufficient reach to their intended audience, be implemented as intended in the real world, and adequately engage their audience [16].

    Limited data exist regarding the reach and implementation of evidence-based health promotion programs among AI/AN youth. The Bureau of Indian Education (BIE) supports 183 primary and secondary schools in 23 states, serving approximately 50,000 students. A school health assessment of BIE schools conducted in New Mexico in 2006 reported that 38 of 39 schools (97%) provided health education and/or health promotion activities, and 67% used a comprehensive health education curriculum [17]. Curriculum content centered predominantly on violence and substance use prevention, physical activity, and nutrition. Less than 50% of the schools provided education on pregnancy prevention, highlighting the sensitive nature of sexual health education in AI/AN communities. Ten of the 39 schools (26%) had certified health educators. No information was provided regarding whether curriculum content was evidence-based or culturally sensitive [17]; however, few culturally based or culturally sensitive behavioral health promotion programs exist for AI/AN youth [18]. Furthermore, data from the 2013 Youth Risk Behavior Surveillance Survey indicate that AI/AN students in Alaska and Montana (states with high proportions of rural and remote communities) were less likely to receive school-based human immunodeficiency virus/acquired immune deficiency syndrome education compared to white students [19]. Although restricted in scope, these findings indicate that limited trained personnel, cultural barriers, and geographic isolation may hinder the reach and implementation of evidence-based health promotion programs among AI/AN youth [1,20].

    Technology-based programs may offer a viable strategy to increase the reach and implementation of evidence-based health promotion programs in this underserved population [21]. Furthermore, AI/AN youth themselves have repeatedly voiced the need for technology-based health programs to address sensitive health topics [22,23]. Although Internet access is highly variable in tribal communities across the United States, it is swiftly and steadily improving [24]. For example, BIE-funded schools are part of the federal ConnectED initiative to increase Internet connectivity and educational technology in schools. Technology usage rates among AI/AN youth exceed national averages, and many use the Internet to access health information [25,26]. These factors suggest that utilizing the Internet could increase the reach of evidence-based health promotion to AI/AN communities. Internet-based programs can also improve the implementation of programs as intended, given the reduced need for specialized facilitator training and enhanced confidentiality to deliver sensitive topics, such as sexual health [21]. Internet-based programs may also increase student engagement with program activities [21,27], and provide the ability to tailor instructions to individual characteristics (eg, gender, risk factor, or stage of change) [28]. Most importantly, Internet-based health promotion programs have been shown to impact behavior change across multiple health domains, including sexual health, substance use, physical activity and nutrition, and hearing protection [26-31].

    Recently, Internet-based sexual health promotion programs have been specifically developed or adapted for AI/AN youth, with high satisfaction ratings reported [21,24]. The purpose of this study was to examine the potential of using the Internet to increase the reach and implementation of evidence-based health promotion programs across a variety of health topics, and to engage early adolescent AI/AN youth in 3 geographically dispersed regions in the United States. The findings from this study have broader implications in understanding the degree to which Internet-based programs may increase the dissemination and utilization of evidence-based health promotion programs in tribal communities.


    Study Design

    This study presents a secondary analysis of data collected during the implementation phase of a randomized controlled trial (Clinicaltrials.gov NCT01303575) that assessed the effectiveness of Native It’s Your Game (Native IYG; a Web-based sexual health education program adapted for AI/AN youth) relative to a comparison suite of 5 evidence-based Internet-based health promotion programs, across 3 geographically dispersed regions in the United States. A detailed description of the adaptation process for Native IYG is provided in a supplemental file [32]. Primary outcomes of the randomized controlled trial are forthcoming. Data presented here provide insight into the potential of using the Internet to increase the reach and implementation of evidence-based adolescent health promotion programs across a variety of health topics to AI/AN youth in diverse geographic regions, and the ability of Internet-based programs to engage AI/AN youth.

    Participants

    Participants were primarily self-identified AI/AN youth aged 12-14 years and adult site coordinators (teachers, counselors, nurses, wellness coordinators, and college students) that were recruited from 25 study sites. The sites were located in 13 urban and 12 rural/tribal settings in Alaska, Arizona, and the Pacific Northwest (Oregon, Idaho, and Washington), and comprised schools, tribal community centers, after school programs, and summer youth programs. Given the importance of confidentiality when partnering with AI/AN communities, specific tribal names have been withheld. The study was approved by the Alaska Area Institutional Review Board (IRB), the Portland Area Indian Health Service IRB, the University of Texas Health Science Center at Houston (UTHealth) IRB, and 16 tribal organizations (ie, tribal councils, tribal health boards, villages, and community agencies).

    Procedure

    Study activities were coordinated regionally by 3 organizations that collectively serve 295 federally recognized AI/AN tribes. Regional staff used convenience sampling to recruit tribal communities that were interested in participating in an early adolescent sexual health study. Regional staff sent flyers to schools, tribal community centers, after school programs, and summer camp programs, and advertised on organizational websites, social media outlets, and/or via newsletters. Participating sites were randomized to a treatment condition (Native IYG, n=14) or a comparison condition (n=11) featuring a suite of 5 evidence-based Internet-based health promotion programs that were not focused on sexual health.

    Site Coordinator Training

    Site coordinators at each study site completed the Collaborative Institutional Training Initiative Program’s online certification in human subject research and a live webinar (tailored for treatment or control conditions) coordinated by UTHealth research staff, which explained intervention content and protocols for logging-in youth, documentation, and maintaining confidentiality.

    Internet-Based Health Promotion Programs

    Native IYG is a 13-lesson, multimedia sexual health education curriculum for AI/AN youth (aged 12-14 years). The curriculum was adapted from an Internet-based curriculum for urban middle schools titled It’s Your Game-Tech (IYG-Tech) [32]. Lessons are approximately 35 minutes long. Adaption and formative development, guided by feedback from AI/AN youth and adults, comprised surface alterations (eg, changing the program logo) and deep cultural adaptations (eg, adding a blessing by AI elders and videos featuring AI/AN youth). A detailed description of the adaptation process has been published elsewhere [32].

    The 5 evidence-based, Internet-based programs that comprised the comparison suite addressed hearing loss (Dangerous Decibels), alcohol use (N-Squad), tobacco use (A Smoking Prevention Interactive Experience; ASPIRE), drug use (Reconstructors), and physical activity and nutrition (The Quest to Lava Mountain). Each program has undergone usability testing with non-AI/AN youth as part of its own formative development, and has demonstrated efficacy to positively impact health behaviors and/or related psychosocial factors in other adolescent populations. The Dangerous Decibels Virtual Exhibit was developed by the Oregon Museum of Science and Industry as an online component of a public health campaign to reduce the incidence and prevalence of noise-induced hearing loss and tinnitus (ringing in the ear) by improving knowledge, attitudes, and protective behaviors of school-aged children [33]. N-Squad and Reconstructors, developed by the Rice University Center for Technology in Teaching and Learning, are Internet-based adventures for middle school students to learn about alcohol’s interaction in the digestive, circulatory, and nervous systems, and explore the science behind drugs of abuse [34,35]. ASPIRE is an online tobacco prevention and cessation curriculum, developed jointly by researchers at The University of Texas MD Anderson Cancer Center and UTHealth, with demonstrated efficacy in preventing smoking onset in high school youth [36]. The Quest to Lava Mountain, developed as part of the Texas Department of Agriculture NutriGram program by The Cooper Institute, is an educational game designed to raise awareness about healthy eating and physical activity [37].

    Site coordinators logged participants onto the programs on desktop or laptop computers located in quiet locations (eg, a computer lab, empty classroom, or library). Sites with insufficient bandwidth to accommodate simultaneous Internet access by multiple users were provided with uploadable versions of their respective programs on a Universal Serial Bus (USB) drive or digital video disc (DVD).

    Data Collection

    Assessing Reach and Implementation

    Prior to implementation, site coordinators completed a computer-based education inventory survey [32] and connectivity and bandwidth testing [38] to document access parameters related to program reach. The site coordinators used problem logs during implementation to document program errors and technical issues that impacted implementation.

    Assessing Youth Engagement

    We used Likert-type scales adapted from previous studies to assess youths’ perceptions of engagement [39-42]. Ease of use was based on how easy it was to use the program, using a Likert-type 3-point scale (very easy to kind of hard). Understandability was based on whether youth understood the words used and if they needed hints from an adult to play the game, using a Likert-type 3-point scale (yes, no, and don’t know). Credibility was based on youths’ perceptions of content correctness, using a Likert-type 3-point scale (right, wrong, and don’t know) and trustworthiness (yes, no, and don’t know). Likeability was based on how much youth liked the program activities, using a Likert-type 5-point scale (dislike a lot to like a lot). Perceived impact was based on whether youth thought the program would help them make healthy decisions, using a Likert-type 3-point scale (yes, no, and don’t know). Motivational appeal was based on whether youth would play more program lessons if available, if they would recommend the program to classmates, and whether the program was as much or more fun than other lessons at school, including other health lessons, computer-based lessons, and their favorite video game, using a Likert-type 3-point scale (yes, no, and don’t know).

    Ratings were collected via an Internet-based Qualtrics usability survey administered at the completion of each program. Demographic characteristics (gender, age, and self-identified race/ethnicity) were collected during the study’s baseline survey, using an Internet-based Qualtrics self-report survey [43]. Upon entering the study, youth received a unique study identification number to link data across surveys. No names were associated with the surveys. All youth provided signed parental consent and youth assent prior to participating in the study. A detailed description of study procedures has been published elsewhere [43].

    Data Analysis

    We used descriptive statistics (frequencies, median/mean, and/or range) to summarize data on reach parameters (computer access, connectivity, and bandwidth) and implementation (program errors and technical issues). Regarding youth engagement, for each parameter we calculated the percent of youth who rated each program favorably, and the range of ratings across all 6 programs, from lowest to highest.


    Reach and Implementation

    Twenty-four site coordinators (24/25, 96%) provided complete or partial information related to Internet reach parameters. Eighteen computer labs, one classroom, and one after school classroom across the 3 regions were primarily composed of personal computers (13/20, 65%) and Mac computers (6/20, 30%), were mostly wired (16/18, 89%), and most had access to the Internet (22/24, 92%). Primary Web browsers included Chrome (7/20, 35%), Safari (5/20, 25%), Internet Explorer (5/20, 25%), Firefox (2/20, 10%), and Mozilla (1/20, 5%). Download speeds ranged from 0.24 megabits per second (Mbps) to 93.5 Mbps (mean 25.6 Mbps, standard deviation 31.14; median 6.37 Mbps).

    At some sites, the method of program delivery changed during implementation. Treatment sites commenced the study accessing Native IYG as an online streaming program (n=12) or as an uploadable program from a USB drive (n=2). During the study, several sites that initially accessed Native IYG as an online streaming program converted to uploading Native IYG from a USB drive (n=4) due to the inability of local bandwidth to accommodate larger Native IYG video files while providing simultaneous streaming for multiple youth.

    Comparison condition sites commenced the study by accessing the suite of health promotion programs via online streaming (n=8) or a combination of online streaming and an uploadable program from DVDs (n=2). Information regarding program access was missing from one site. During the study, most sites continued to access these programs via Internet connections (n=8).

    The most commonly reported problems that were documented by site coordinators during implementation included frozen screens (4/6 programs), activities taking a long time to load (3/6 programs)—both of which were related to multiple simultaneous users—and trouble navigating the programs (3/6 programs).

    Youth Engagement

    During implementation, 387 youth received Native IYG, of whom 191 (49%) provided feedback; 136 youth received the comparison suite of programs, of whom 108 (79%) completed at least one feedback survey. Across programs, participants were 48-60% female, with a mean age of 13.1-13.3 years, and 85-90% self-identified as AI/AN (Table 1).

    Table 1. Demographic characteristics for AI/AN youth who provided ratings for each program (n=40-191): Alaska, Arizona, and Pacific Northwest, 2012-2014.


    		Native IYG
    (n=191)    		Dangerous Decibels
    (n=62)    		N-Squad
    (n=62)    		ASPIRE
    (n=52)    		Reconstructors
    (n=45)    		Lava Mountain
    (n=40)
    Gender, n (%)

    		Female114 (59.7)30 (48.4)34 (54.8)25 (48.1)24 (53.3)21 (52.5)

    		Male77 (40.3)32 (51.6)28 (45.2)27 (51.9)21 (46.7)19 (47.5)
    Self-identify as AI/AN, n (%)164 (85.9)54 (87.1)56 (90.3)46 (88.5)40 (88.9)34 (85.0)
    Mean age (standard deviation)13.1 (0.98)13.2 (0.83)13.3 (0.84)13.3 (0.77)13.2 (0.78)13.3 (0.88)

    Youth generally rated the programs as easy to use (68-91%) and the majority understood the words in the programs (60-83%). However, some participants needed adult assistance or hints to complete the programs (16-49%). Over half of the youth reported that the program content was credible, rating it correct (58-90%) and trustworthy (61-89%), and 63-91% reported that the programs would help them make better choices. Regarding likeability, 59-87% of youth liked the programs a little or a lot. In terms of motivational appeal, 35-63% of youth reported that they would play more lessons from each program, if available; 37-66% would recommend the programs to a classmate. Although fewer youth considered the programs as much or more fun than their favorite video game (25-61%), youth stated that the programs were as much or more fun than other lessons at school (62-87%), other health lessons at school (61-79%), and other computer-based lessons at school (57-85%; Multimedia Appendix 1).


    AI/AN youth face multiple health challenges compared to youth of other racial/ethnic groups. Viable program delivery strategies that overcome limited personnel with training in health education, cultural barriers, and geographic isolation (ie, the Internet) are needed to increase the reach and implementation fidelity of evidence-based adolescent health promotion programs in tribal communities. We examined the potential of using the Internet to deliver 6 evidence-based health promotion programs to AI/AN youth living in urban and rural settings in 3 geographically diverse regions of the United States. The health topics addressed included sexual health, hearing loss, substance use, physical activity, and nutrition. Despite variability in connectivity and bandwidth, most sites were able to access the programs via the Internet. However, technical and connectivity issues led some sites to access the programs via back-up modalities (ie, uploading the programs from a USB drive or DVD). Practitioners interested in implementing Internet-based programs in tribal communities are advised to provide contingency plans as back-ups to technical failures. We also recommend conducting bandwidth assessments, especially when dealing with multiple simultaneous users, prior to implementing an Internet-based program.

    Adult site coordinators from each tribal community (including teachers, counselors, nurses, wellness coordinators, and college students) facilitated youths’ access to the Internet-based programs. Regarding implementation fidelity and program errors, site coordinators reported issues with frozen screens (4/6 programs), activities taking a long time to load (3/6 programs), and trouble navigating the programs (3/6 programs). Given that 16-49% of youth needed assistance to complete the programs, some adult oversight is recommended when implementing Internet-based programs for youth. Although the site coordinators received webinar training prior to program implementation, the training focused primarily on research study-specific protocols (eg, logging students onto computers using study identification numbers and reporting technical issues). The training related to actual program implementation, such as reviewing specific program content, was limited to approximately one hour, indicating that a broad range of personnel (including those not certified in health education) may be able to implement Internet-based health promotion programs.

    Overall, AI/AN youth rated the programs favorably. Although fewer youth rated the programs as being as engaging as their favorite video game (a lofty expectation for any educational program), over 60% stated that the programs were as much or more fun than other lessons at school. These findings indicate that Internet-based health promotion programs are engaging to AI/AN youth in the diverse settings of both rural (reservation and village) and urban locations. These results also support previous studies indicating that Internet-based health promotion programs provide an engaging educational format for Millennium learners. Internet-based programs are uniquely positioned to allow for: the provision of motivational learning experiences delivered via video, animated characters, and gaming formats; the provision of role modeling activities that influence normative perceptions and skills; confidential and personalized presentation of sensitive and potentially embarrassing issues (eg, sexual health); and intervention messages that are tailored to specific user characteristics [27,28]. These features may help to engage and retain AI/AN youth in health promotion programs.

    It is worth noting that youth rated Native IYG most favorably across all programs on 10 of the 13 usability parameters, possibly because this program was specifically adapted for AI/AN youth. This finding aligns with previous studies that point to the value of cultural tailoring [44]. Incorporating traditional AI/AN values and teaching methods into Internet-based health promotion programs may strengthen or reinforce a sense of cultural identity and belonging among AI/AN youth, better align with health epistemologies and learning styles, and help protect against engaging in early risk behaviors [24]. The evidence-based approach that we used to adapt IYG for AI/AN youth may provide a useful model for adapting other health promotion programs [32]. The favorable ratings for Native IYG are also noteworthy, given the sensitivity associated with sexual health education in AI/AN communities, and indicate that Internet-based programs may be an appropriate delivery channel for other sensitive health topics.

    Across all programs, we experienced attrition in the number of youth who received the programs compared to those who completed a feedback survey. This attrition may have been due to respondent burden or fatigue in completing surveys after each program. However, the relatively high motivational appeal ratings across programs (eg, at least 60% of youth stated that the programs were as much or more fun than other lessons at school) suggest that the programs would be engaging for AI/AN youth in school, after school, or in community-based settings.

    This study has several strengths. First, most feasibility studies for technology-based health promotion programs feature small numbers of participants (typically <30 users), as they do not require statistical significance to determine major usability problems [45]. In contrast, at least 40 AI/AN youth reviewed each program, with a relatively even mix of males and females in each program. Second, participants were recruited from a range of urban and rural settings across 3 geographic regions, thereby increasing the representation of youth from varied tribal communities that represent differing cultural perspectives, traditions, and values, as well as varied urbanicity and Internet access. Third, the study included Internet-based programs across a variety of health promotion topics that are important to AI/AN communities. Taken together, these factors may enhance the generalizability of findings to AI/AN communities beyond the study sample.

    Despite these strengths, several limitations should be noted. First, the sample was restricted to early adolescent AI/AN youth, aged 12-14 years; thus, findings may not apply to older AI/AN youth. Second, social desirability may have biased youths’ ratings of the programs; however, the fact that some items scored lower than others (eg, comparing programs to their favorite video game) suggests that youth answered honestly, based on their experience with each program. Third, the intervention program, Native IYG, was specifically adapted for AI/AN youth. In contrast, no attempt was made to adapt the comparison group programs for AI/AN youth; this factor may have negatively impacted program ratings relative to Native IYG. Finally, although the use of percent ranges to evaluate engagement is sufficient for this kind of exploratory trial, more thorough quantitative analyses, such as appropriately powered randomized pilot tests of the programs’ impact on behavioral determinants or randomized controlled efficacy field trials, are needed to determine long-term behavioral impact.

    Despite these limitations, this study demonstrates the potential of using the Internet to disseminate evidence-based health promotion programs to AI/AN youth across 3 separate geographic regions. These findings may have broader implications for understanding the degree to which Internet-based programs may enhance the reach and implementation of evidence-based health promotion programs in tribal communities, and provide an educational format that is engaging for AI/AN youth.

    Acknowledgments

    All authors contributed to the conception and design of the study, and the process of drafting and revising the manuscript. This study was funded by the Centers for Disease Control and Prevention (CDC #5U48DP001949-02). The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the CDC. We would like to thank the youth and site coordinators for their participation in the study.

    Authors' Contributions

    CMM served as joint Principal Investigator with RS on the conceptualization and design of the study, drafted the initial manuscript, and approved the final manuscript as submitted. SCR, CJ, and GG served as Regional Principal Investigators for the study, contributed to the intervention design, coordinated site recruitment in their respective regions, codesigned the data collection instrument, critically reviewed and revised the manuscript, and approved the final manuscript as submitted. WEL, AVP, and LM were program developers for Dangerous Decibels, ASPIRE, N-Squad, and Reconstructors, respectively. These authors critically reviewed and revised the manuscript, and approved the final manuscript as submitted. KA-F served as a representative for The Cooper Institute (the program developer for The Quest to Lava Mountain), critically reviewed and revised the manuscript, and approved the final manuscript as submitted. JT coordinated intervention development and data collection across all 3 regions, assisted in data cleaning and initial analyses, drafted sections of the initial manuscript, critically reviewed and revised the manuscript, and approved the final manuscript as submitted. RCA conducted statistical analyses, critically reviewed and revised the manuscript, and approved the final manuscript as submitted. MFP served as Co-Investigator for the study and codesigned the data collection instrument, study design, and analyses. She critically reviewed and revised the manuscript and approved the final manuscript as submitted. RS served as joint Principal Investigator with CMM on the conceptualization and design of the study, critically reviewed and revised the manuscript, and approved the final manuscript as submitted.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    PDF File (Adobe PDF File), 31KB
    1. United States Department of Health & Human Services Office of Minority Health. American Indian/Alaska Native profile.: Office of Minority Health; 2016.   URL: http://www.minorityhealth.hhs.gov/omh/browse.aspx?lvl=3&lvlid=62 [accessed 2016-11-06] [WebCite Cache]
    2. de Ravello L, Everett Jones S, Tulloch S, Taylor M, Doshi S. Substance use and sexual risk behaviors among American Indian and Alaska Native high school students. J Sch Health 2014 Jan;84(1):25-32 [FREE Full text] [CrossRef] [Medline]
    3. Hamilton BE, Martin JA, Osterman MJ, Curtin SC, Matthews TJ. Births: final data for 2014. Natl Vital Stat Rep 2015 Dec;64(12):1-64 [FREE Full text] [Medline]
    4. Centers for Disease Control and Prevention (CDC). Vital signs: repeat births among teens - United States, 2007-2010. MMWR Morb Mortal Wkly Rep 2013 Apr 5;62(13):249-255 [FREE Full text] [Medline]
    5. Centers for Disease Control and Prevention Indian Health Service. Indian Health Surveillance report - Sexually transmitted diseases, 2011. Indian Health Service. 2014.   URL: http://www.cdc.gov/std/stats/ihs/ihs-surv-report-2011_062314.pdf [accessed 2016-11-05] [WebCite Cache]
    6. Wong CA, Gachupin FC, Holman RC, MacDorman MF, Cheek JE, Holve S, et al. American Indian and Alaska Native infant and pediatric mortality, United States, 1999-2009. Am J Public Health 2014 Jun;104 Suppl 3:S320-S328. [CrossRef] [Medline]
    7. Gordon A, Oddo V. Addressing child hunger and obesity in Indian Country: Report to Congress. 2012.   URL: http://www.fns.usda.gov/sites/default/files/IndianCountry.pdf [accessed 2016-05-21] [WebCite Cache]
    8. Ogden CL, Carroll MD, Kit BK, Flegal KM. Prevalence of childhood and adult obesity in the United States, 2011-2012. JAMA 2014 Feb 26;311(8):806-814 [FREE Full text] [CrossRef] [Medline]
    9. SEARCH for Diabetes in Youth Study Group, Liese AD, D'Agostino RB, Hamman RF, Kilgo PD, Lawrence JM, et al. The burden of diabetes mellitus among US youth: prevalence estimates from the SEARCH for Diabetes in Youth Study. Pediatrics 2006 Oct;118(4):1510-1518. [CrossRef] [Medline]
    10. Espey DK, Jim MA, Cobb N, Bartholomew M, Becker T, Haverkamp D, et al. Leading causes of death and all-cause mortality in American Indians and Alaska Natives. Am J Public Health 2014 Jun;104 Suppl 3:S303-S311. [CrossRef] [Medline]
    11. Shoenborn C, Heyman K. Health disparities among adults with hearing loss: United States, 2000-2006. NCHS Health & Stats. 2008.   URL: http://www.cdc.gov/nchs/data/hestat/hearing00-06/hearing00-06.pdf [accessed 2016-05-19] [WebCite Cache]
    12. United States Department of Health and Human Services Office of Adolescent Health. Evidence-based teen pregnancy prevention programs. 2016.   URL: http://www.hhs.gov/ash/oah/oah-initiatives/tpp_program/db [accessed 2016-05-20] [WebCite Cache]
    13. Substance Abuse and Mental Health Services Administration. National Registry of Evidence-based Programs and Practices (NREPP). 2016.   URL: http://www.samhsa.gov/nrepp [accessed 2016-11-06] [WebCite Cache]
    14. Centers for Disease Control and Prevention. The guide to community preventive services: the community guide. 2016.   URL: http://www.thecommunityguide.org/adolescenthealth/index.html [accessed 2016-05-20] [WebCite Cache]
    15. Abrams DB, Orleans CT, Niaura RS, Goldstein MG, Prochaska JO, Velicer W. Integrating individual and public health perspectives for treatment of tobacco dependence under managed health care: a combined stepped-care and matching model. Ann Behav Med 1996;18(4):290-304. [CrossRef] [Medline]
    16. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999 Sep;89(9):1322-1327. [Medline]
    17. Castro G. School health assessment of Bureau of Indian Education schools in New Mexico. 2008.   URL: http://www.bie.edu/cs/groups/xbie/documents/text/idc-008079.pdf [accessed 2016-11-06] [WebCite Cache]
    18. Development Services Group, Inc. Literature review: tribal youth behavioral health. Substance Abuse and Mental Health Services Administration NREPP Learning Center. 2015.   URL: http:/​/www.​nrepp.samhsa.gov/​Docs/​Literatures/​NREPP%20Literature%20Review_Tribal%20Youth%20Behavioral%20Health.​pdf [accessed 2016-11-03] [WebCite Cache]
    19. Centers for Disease Control and Prevention. Youth online: high school YRBS. 2015.   URL: http://www.cdc.gov/healthyyouth/data/yrbs/index.htm [accessed 2016-04-29] [WebCite Cache]
    20. Cobb N, Espey D, King J. Health behaviors and risk factors among American Indians and Alaska Natives, 2000-2010. Am J Public Health 2014 Jun;104 Suppl 3:S481-S489. [CrossRef] [Medline]
    21. Kaufman CE, Schwinn TM, Black K, Keane EM, Big Crow CK. The promise of technology to advance rigorous evaluation of adolescent pregnancy prevention programs in American Indian and Alaska Native tribal communities. Am J Public Health 2016 Sep;106(S1):S18-S20. [CrossRef]
    22. Center for Native American Youth. Health innovation equity: recommendations from Native American youth. 2016.   URL: https:/​/assets.​aspeninstitute.org/​content/​uploads/​files/​content/​upload/​Aetna%20Focus%20Group%20Summary%20Report_CNAY_final.​pdf [accessed 2016-10-05] [WebCite Cache]
    23. Anderson KM, Olson S. National Academies of Sciences, Engineering, and Medicine. Advancing health equity for Native American youth: workshop summary. Washington, DC: The National Academies Press; 2016   URL: https:/​/urldefense.​proofpoint.com/​v2/​url?u= https-3A__www.​nap.​edu_catalog_21766_advancing-2Dhealth-2Dequity-2Dfor-2Dnative-2Damerican-2Dyouth-2Dworkshop-2Dsummary&d=CwIDAw&c= 6vgNTiRn9_pqCD9hKx9JgXN1VapJQ8JVoF8oWH1AgfQ&r=ZU85w6YR5jmr6pQwQEwdqBAfKv_NrBuZZ31lcNExDG8&m= 1K_gB7hFdjV4a0FZZPlhG6MfpjgzqpXNBwK7avPwm6k&s=0PL9iWful2_gT8olBj0lipg0Jo68Ars0lLaPrUGi4k0&e= [accessed 2016-10-20]
    24. Craig Rushing S, Gaston A, Kaufman C, Markham C, Jessen C, Gorman G, et al. Using technology to promote health and wellbeing among American Indian and Alaska Native teens and young adults. In: Dyson LE, Grant S, Hendriks M, editors. Indigenous People and Mobile Technologies. New York: Routledge, Taylor & Francis Group; 2016:163-178.
    25. Craig Rushing S, Stephens D. Use of media technologies by Native American teens and young adults in the Pacific Northwest: exploring their utility for designing culturally appropriate technology-based health interventions. J Prim Prev 2011 Aug;32(3-4):135-145. [CrossRef] [Medline]
    26. Rideout V, Foehr U, Roberts D. Generation M2: media in the lives of 8 to 18-year-olds. Menlo Park, CA: Henry J Kaiser Family Foundation; 2010 Jan 20.   URL: https://kaiserfamilyfoundation.files.wordpress.com/2010/01/mh012010presentl.pdf [accessed 2016-11-14] [WebCite Cache]
    27. Blasi L. Increasing the transfer of simulation technology from R&D into school settings: an approach to evaluation from overarching vision to individual artifact in education. Simul Gaming 2006 Jun 01;37(2):245-267. [CrossRef]
    28. Lustria MLA, Cortese J, Noar SM, Glueckauf RL. Computer-tailored health interventions delivered over the Web: review and analysis of key components. Patient Educ Couns 2009 Feb;74(2):156-173. [CrossRef] [Medline]
    29. Roberto AJ, Zimmerman RS, Carlyle KE, Abner EL. A computer-based approach to preventing pregnancy, STD, and HIV in rural adolescents. J Health Commun 2007;12(1):53-76. [CrossRef] [Medline]
    30. Schwinn TM, Schinke SP, Di NJ. Preventing drug abuse among adolescent girls: outcome data from an Internet-based intervention. Prev Sci 2010 Mar;11(1):24-32 [FREE Full text] [CrossRef] [Medline]
    31. McCullagh MC, Banerjee T, Cohen MA, Yang JJ. Effects of interventions on use of hearing protectors among farm operators: a randomized controlled trial. Int J Audiol 2016;55 Suppl 1:S3-12 [FREE Full text] [CrossRef] [Medline]
    32. Shegog R, Craig Rushing S, Gorman G, Jessen C, Torres J, Lane TL, et al. NATIVE-It's Your Game: adapting a technology-based sexual health curriculum for American Indian and Alaska Native youth. J Prim Prev 2016 Aug 12. [CrossRef] [Medline]
    33. Martin WH, Griest SE, Sobel JL, Howarth LC. Randomized trial of four noise-induced hearing loss and tinnitus prevention interventions for children. Int J Audiol 2013 Feb;52 Suppl 1:S41-S49. [CrossRef] [Medline]
    34. Klisch Y, Miller LM, Beier ME, Wang S. Teaching the biological consequences of alcohol abuse through an online game: impacts among secondary students. CBE Life Sci Educ 2012;11(1):94-102 [FREE Full text] [CrossRef] [Medline]
    35. Klisch Y, Bowling KG, Miller LM, Ramos MA. The impact of science education games on prescription drug abuse attitudes among teens: a case study. J Drug Educ 2013;43(3):255-275. [CrossRef] [Medline]
    36. Prokhorov AV, Kelder SH, Shegog R, Murray N, Peters R, Agurcia-Parker C, et al. Impact of A Smoking Prevention Interactive Experience (ASPIRE), an interactive, multimedia smoking prevention and cessation curriculum for culturally diverse high-school students. Nicotine Tob Res 2008 Sep;10(9):1477-1485. [CrossRef] [Medline]
    37. Sharma SV, Shegog R, Chow J, Finley C, Pomeroy M, Smith C, et al. Effects of The Quest to Lava Mountain computer game on dietary and physical activity behaviors of elementary school children: a pilot group-randomized controlled trial. J Acad Nutr Diet 2015 Aug;115(8):1260-1271. [CrossRef] [Medline]
    38. Ookla. Speedtest. 2016.   URL: http://www.speedtest.net/ [accessed 2016-05-21] [WebCite Cache]
    39. Shegog R, Markham C, Peskin M, Dancel M, Coton C, Tortolero S. “It's Your Game”: an innovative multimedia virtual world to prevent HIV/STI and pregnancy in middle school youth. Stud Health Technol Inform 2007;129(Pt 2):983-987. [Medline]
    40. Shegog R, Peskin MF, Markham C, Thiel M, Karny E, Addy RC, et al. It's Your Game-Tech: toward sexual health in the digital age. Creat Educ 2014 Aug;5(15):1428-1447 [FREE Full text] [CrossRef] [Medline]
    41. Shegog R, Markham CM, Leonard AD, Bui TC, Paul ME. “+CLICK”: pilot of a web-based training program to enhance ART adherence among HIV-positive youth. AIDS Care 2012;24(3):310-318. [CrossRef] [Medline]
    42. Escobar-Chaves SL, Shegog R, Markham C, Brehm BJ. The McSpoons: using puppetry's narrative impact to reduce family TV time. Arts Health 2010 Mar;2(1):47-66. [CrossRef]
    43. Markham CM, Craig Rushing S, Jessen C, Lane TL, Gorman G, Gaston A, et al. Factors associated with early sexual experience among American Indian and Alaska Native youth. J Adolesc Health 2015 Sep;57(3):334-341. [CrossRef] [Medline]
    44. Kreuter M, Oswald D, Bull F, Clark EM. Are tailored health education materials always more effective than non-tailored materials? Health Educ Res 2000 Jun;15(3):305-315 [FREE Full text] [Medline]
    45. Nielsen J. Usability Engineering. San Francisco, CA: Morgan Kaufmann Publishers; 1993.

    ‎
    AI/AN: American Indian and Alaska Native (AI/AN)
    BIE: Bureau of Indian Education
    CDC: Centers for Disease Control and Prevention
    DVD: digital video disc
    IRB: Institutional Review Board
    IYG: It’s Your Game
    Mbps: megabits per second
    USB: Universal Serial Bus
    UTHealth: University of Texas Health Science Center at Houston

    Edited by G Eysenbach; submitted 21.05.16; peer-reviewed by S Bull, VY Hiratsuka, G Kok; comments to author 23.08.16; revised version received 15.10.16; accepted 28.10.16; published 21.11.16

    Copyright

    ©Christine M Markham, Stephanie Craig Rushing, Cornelia Jessen, Gwenda Gorman, Jennifer Torres, William E Lambert, Alexander V Prokhorov, Leslie Miller, Kelly Allums-Featherston, Robert C Addy, Melissa F Peskin, Ross Shegog. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 21.11.2016.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.