This paper is in the following e-collection/theme issue:

CIHR funded proposals with peer-review reports (Canada) (43) Non-randomized Protocols and Methods (ehealth) (567) HIV/AIDS/STI Prevention and Care (873)

Published on in Vol 8, No 7 (2019): July

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/13801, first published .
Community-Directed Bacterial Sexually Transmitted Infection Testing Interventions Among Men Who Have Sex With Men: Protocol for an E-Delphi Study in Toronto, Canada

Community-Directed Bacterial Sexually Transmitted Infection Testing Interventions Among Men Who Have Sex With Men: Protocol for an E-Delphi Study in Toronto, Canada

Community-Directed Bacterial Sexually Transmitted Infection Testing Interventions Among Men Who Have Sex With Men: Protocol for an E-Delphi Study in Toronto, Canada

Authors of this article:

Ann N Burchell1, 2 Author Orcid Image ;   Ryan Lisk3 Author Orcid Image ;   Anna Yeung1 Author Orcid Image ;   Jayoti Rana1 Author Orcid Image ;   Jean Bacon4 Author Orcid Image ;   Jason Brunetta5 Author Orcid Image ;   Mark Gilbert6 Author Orcid Image ;   Dionne Gesink7 Author Orcid Image ;   Ramandip Grewal1 Author Orcid Image ;   Charlie B. Guiang2, 8 Author Orcid Image ;   Michael Kwag9 Author Orcid Image ;   Carmen H Logie10 Author Orcid Image ;   Leo Mitterni8 Author Orcid Image ;   Rita Shahin11 Author Orcid Image ;   Darrell HS Tan1 Author Orcid Image
Article Authors Cited by (7) Tweetations (1) Metrics

    Protocol

    1Centre for Urban Health Solutions, Li Ka Shing Knowledge Institute, St Michael's Hospital, Toronto, ON, Canada

    2Department of Family and Community Medicine, St Michael's Hospital, Toronto, ON, Canada

    3ACT, Toronto, ON, Canada

    4Ontario HIV Treatment Network, Toronto, ON, Canada

    5Maple Leaf Medical Clinic, Toronto, ON, Canada

    6BC Centre for Disease Control, Vancouver, BC, Canada

    7Dalla Lana School of Public Health, University of Toronto, Toronto, ON, Canada

    8Hassle Free Clinic, Toronto, ON, Canada

    9Community-Based Research Centre, Vancouver, BC, Canada

    10Factor-Inwentash Faculty of Social Work, University of Toronto, Toronto, ON, Canada

    11Toronto Public Health, Toronto, ON, Canada

    Corresponding Author:

    Ann N Burchell, PhD

    Centre for Urban Health Solutions

    Li Ka Shing Knowledge Institute

    St Michael's Hospital

    209 Victoria St

    Toronto, ON, M5B 1T8

    Canada

    Phone: 1 4168646060 ext 77498

    Email: burchella@smh.ca


    Background: HIV-positive and HIV-negative (gay, bisexual, and other) men who have sex with men (MSM) have experienced a dramatic increase in bacterial sexually transmitted infections (STIs)—syphilis, gonorrhea, and chlamydia. STI testing and treatment mitigate adverse health outcomes and substantially reduce transmission; yet, testing rates remain below recommended levels. Innovation is needed to produce the required increases in testing levels, frequency, and the use of appropriate testing technologies in ways that are engaging, nonstigmatizing, and acceptable to men.

    Objective: The aim of this study is to build consensus with regard to interventions with the greatest potential for improving local STI testing services for MSM communities in Toronto, Canada.

    Methods: Following a literature review of evidence regarding the effectiveness of novel testing interventions, and focus groups, and surveys to describe local barriers and facilitators of testing among MSM, we will conduct a Web-based, modified Delphi study (e-Delphi). We will form expert panels of community members and STI test providers. Panelists will rate potential interventions in terms of their priority, using a 7-point Likert scale from definitely not a priority to definitely a priority. They will also rank their preferences by selecting their top 3 preferred interventions. Surveys will be distributed in 3 rounds, with feedback on the distribution of responses from preceding rounds provided in rounds 2 and 3. We will define consensus as having ≥60% (18/30) members indicate a preference within 2 adjacent response points. Qualitative data on disagreements will be obtained using open-ended text responses to explain for ratings and rankings that are different from the majority.

    Results: On the basis of a literature review and identification of barriers and facilitators to STI testing among community members and test providers in Toronto, we have selected 8 potential interventions for inclusion in the e-Delphi panel surveys. These include 4 interventions that streamline STI testing for asymptomatic individuals, 2 interventions that are targeted at clients and 2 interventions that are targeted at providers.

    Conclusions: Findings will provide community direction for informed decision making regarding the implementation of STI testing interventions in this setting. They will characterize the intervention climate for innovation to STI testing services, including perceived needs for changes to test delivery, relative priorities for change, and readiness for implementation. These methods may be transferable to other urban jurisdictions experiencing similar epidemics and for other contexts where stakeholder input is needed to manage sensitive areas of concern.

    International Registered Report Identifier (IRRID): PRR1-10.2196/13801

    JMIR Res Protoc 2019;8(7):e13801

    doi:10.2196/13801

    Keywords

    sexual and gender minorities; sexually transmitted diseases; community-based research; mass screening; patient acceptance of health care 


    Delphi Studies

    Delphi studies are a valuable approach for building consensus around an issue where little knowledge or agreement previously existed [1]. They use “structured anonymous communication between experts…to gather consensus perspectives about an issue or topic that can then be…used to inform decision making” [2]. Traditionally applied using an in person format, this design is increasingly being adapted for Web-based environments. Briefly, the Web-based modified Delphi study (e-Delphi) involves rounds of Web-based questionnaires in which experts are asked to provide their opinion on particular topics [1,3]. Initially this is done independently, but in subsequent rounds, experts are made aware of the opinions of the group when making their decisions, with the goal of reaching consensus. The key features of the e-Delphi methods are that they are iterative and anonymous, which are particularly beneficial for community-based and patient-oriented research [2,4]. Anonymity and the Web-based format encourage opinion sharing from all panel members, thus preventing dominant individuals from controlling discussion; this is important within hierarchical environments involving the health care system [2,4].

    Bacterial Sexually Transmitted Infections

    We will adapt the e-Delphi method to learn community perspectives to address a pressing health care system issue in our setting: the rise of bacterial sexually transmitted infections (STIs)—specifically syphilis, gonorrhea, and chlamydia. These infections pose a heavy burden on population health, with most cases occurring among HIV-positive and HIV-negative gay, bisexual, and other men who have sex with men (MSM) [5-9]. Untreated syphilis may progress to neurosyphilis, in which symptoms such as meningitis or dementia may develop [10]. Globally, public health agencies are pressing for increased vigilance of antibiotic-resistant gonorrhea strains [11,12]. Certain serovars of Chlamydia trachomatis may cause Lymphogranuloma venereum with painful proctitis and rectal bleeding. Unlike HIV, these STIs can easily transmit via oral sex [13]. In 2014, there were 109,263 chlamydia, 16,285 gonorrhea, and 2357 syphilis cases reported in Canada, much greater than a decade earlier [14]. The true counts are even higher, as many cases are asymptomatic and go unreported. Gonorrhea and syphilis rates have dramatically increased among males in the province of Ontario, with nearly all syphilis cases and approximately 40% of gonorrhea cases among MSM and >40% of syphilis cases among HIV-positive MSM [15,16]. We have documented that 23% of HIV-positive MSM have had syphilis, and new infections occur at minimum rates of 1 gonorrhea, 1 chlamydia, and 4 syphilis cases per 100 person years [5-8]. Most cases occur in the city of Toronto, with no signs of a decline [17-19]. Within Toronto, the syphilis epidemic is mature and not restricted to a core sociodemographic group among MSM [8,20], requiring broadscale approaches for control.

    STI testing and treatment could mitigate adverse health outcomes and substantially reduce population-level transmission among MSM [21]. However, innovation is needed to produce the required increases in testing levels, frequency, and the use of appropriate testing technologies in ways that are engaging, nonstigmatizing, and acceptable to men. Canadian STI Guidelines recommend annual screening for bacterial STIs among sexually active MSM and as frequently as every 3 months for individuals at ongoing risk for STIs [13]. Unfortunately, there are suboptimal levels of STI testing and frequency of testing among MSM in Toronto. STI testing patterns are best known for HIV-positive MSM. In 2009, 55% had tested for syphilis, on average, once per year [8]. As of 2013, we observed only a modest increase to 64% being tested annually, with a few testing more frequently than once per year [22]. Testing rates for chlamydia and gonorrhea are lower than those for syphilis [7,23]; from 2010 to 2013, only 25% of HIV-positive MSM tested annually for genital infection using urine-based tests. Few MSM undergo extragenital testing for gonorrhea and chlamydia, despite Canadian and international guidelines [7,12,23-27]. Without rectal and pharyngeal tests, 71% to 100% of cases will be missed [28,29].

    We describe herein our plans to conduct an e-Delphi study as part of a larger mixed-methods study that aims to identify bacterial STI testing interventions for implementation and evaluation among MSM in Toronto. We will assemble 2 expert panels: the first with community members with lived experience as MSM seeking STI testing, and the second with health care providers and public health professionals with expertise in providing STI testing for MSM communities in our setting. Our objective is to build consensus regarding intervention(s) with the greatest potential for improving local STI testing services.


    All procedures have been reviewed and approved by the research ethics boards of St Michael’s Hospital, Toronto, and the University of Toronto.

    Setting

    Toronto is a metropolitan city with a population size of 2.71 million in the province of Ontario [30]. All residents with citizen, permanent resident, refugee, and refugee claimant status have access to provincial or federal health insurance for medically necessary services. STI testing services are available from a variety of sources, including primary care practices, specialist services, or dedicated sexual health clinics.

    Knowledge Synthesis to Select Candidate Interventions

    To select STI testing interventions for primary inclusion in the e-Delphi panel, we undertook a review of the published literature [31]. To further refine the interventions, we conducted focus groups with MSM STI testing clients [32] and surveyed health care providers [33] in Toronto. Briefly, the focus groups were conducted with HIV-positive, HIV-negative, and trans-identified men (of any HIV serostatus) to identify barriers and facilitators to bacterial STI testing. Health care providers were surveyed about their current practices, barriers, and attitudes to improve bacterial STI testing rates. Manuscripts for these findings are in preparation.

    Literature Review

    For our literature review, systematic reviews published in 2016 were used as a baseline and updated. These reviews summarized evidence for the effectiveness of STI control interventions, including screening in and outside clinic-based settings published in 2000 or after [24,34]. In addition to repeating the 2016 searches, we expanded literature searches in MEDLINE up to April 2017 using the following keywords: sexually transmitted diseases/STI, chlamydia, gonorrhea, or syphilis. Inclusion criteria for our search were that the article described an intervention aimed at increasing bacterial STI testing; used high-income country settings in urban or semiurban cities; had a study population that included men; and used a study design that was either a trial with a comparison group (controlled, uncontrolled, or pre-post historical controls) or an observational design if it was set in Canada, focused on MSM, or described a Web-based STI testing service. Publications were ineligible if they included only women or heterosexual couples or if they were a study protocol.

    Next, we classified the interventions into 3 categories: (1) streamlined testing for asymptomatic individuals, (2) interventions targeted toward clients, and (3) interventions targeted toward providers (Table 1). We use the term clients to refer to users of STI testing services, whether or not they are experiencing signs or symptoms of an STI. Using the same strategy as Taylor et al [34], examining outcomes in increasing the proportion tested or increasing frequency of testing, interventions with a comparison group were categorized as very effective (absolute difference (AD) ≥20% or relative difference (RD) ≥100%), moderately effective (AD 5%-19% or RD 10%-99%), or ineffective (AD <5% or RD <10%). Classifications and categorizations were done by JR and verified by ANB. A complete list of the publications used for the final selection of interventions can be found in Multimedia Appendix 1.

    The investigators then reviewed the above findings in a series of meetings and selected promising interventions for the Toronto setting to be included in the Delphi panel exercise. Our selection focused on novel approaches for testing rather than efforts that would reinforce existing STI test practices (eg, patient or provider education alone). To minimize respondent burden for panelists, choices are limited to 6 (for community panelists) or 8 (for provider panelists) intervention options.

    Table 1. Categories of interventions.
    CategoryDefinition
    Streamlined STIa testing for asymptomatic individualsInterventions that focus on testing asymptomatic individuals with a focus on collection of specimens and reducing the time patients spend in clinics
    Client-targeted STI testing interventionsInterventions that are targeted at clients to increase client engagement in STI testing
    Provider-targeted STI testing interventionsInterventions that are targeted at health care providers to increase provision of STI testing

    aSTI: sexually transmitted infection.

    Recruitment of E-Delphi Panelists

    In our application of the e-Delphi method, the term expert is meant to include persons with lived experience alongside health care professionals. We will form 2 panels: the first with community members with lived experience as MSM, seeking STI testing in Toronto (Community Experts), and the second with health care providers and public health professionals with expertise in providing STI testing for MSM communities in Toronto (Provider Experts). We opted to recruit these 2 panels separately, rather than combined, as it was of interest to identify differences in prioritized interventions between the 2 groups, if these exist, rather than forcing consensus between community and provider experts.

    To be eligible for the Community Panel, candidates (1) must be a cis- or trans-identified man aged 18 years and older, living in Toronto, and who has sex with men in the preceding 18 months and (2) must have sought and/or underwent STI testing in Toronto in the preceding 18 months.

    To be eligible for the Provider Panel, candidates must have a minimum of 1-year experience providing STI testing and management care in Toronto.

    ‎
    Figure 1. Flowchart for recruitment of community and provider expert panel. Left: To be eligible for the Community Panel, candidates (1) must be a cis- or trans-identified man aged 18 years and older, living in Toronto, and who has sex with men in the preceding 18 months and (2) must have sought and/or underwent sexually transmitted infection testing in Toronto in the preceding 18 months. Right: To be eligible for the Provider Panel, candidates must have a minimum of 1-year experience providing sexually transmitted infection testing and management care in Toronto.
    View this figure

    The choice of experts for an initial invitation will be informed by our team’s community and professional networks. Experts who agree to participate will be encouraged to refer other eligible experts, mitigating potential bias from our team’s selection of members.

    Using the approach shown in Figure 1, invitations will be informed by our team’s community and professional networks. Community participants will also be recruited via the existing social media channels (eg, Facebook and Twitter) of our community-based partner and through paid banner advertisements on popular gay dating apps (eg, Grindr). In addition, targeted emails will be sent to other organizations that serve the MSM community, including AIDS Service Organizations that cater to specific ethnoracial groups. These methods were successfully used in the recruitment of MSM for the focus groups and other studies conducted by our community-based partner [32]. Providers will be recruited using targeted emails to health care organizations known to serve large MSM patient populations, as we have done previously in our provider survey [33]. We aim to recruit a minimum of 30 experts with diverse backgrounds (including ethnoracial identity, gender identity, sexual orientation, and age) for each panel—feasible and sufficient for a Delphi study [2-4,35]. For community panelists, we have set target goals to recruit a minimum of 40% to be men aged ≤40 years and 40% to identify as non-white race/ethnicity.

    Recruitment email invitations, social media, and dating app advertisements will include a link to an eligibility survey. Interested participants will need to complete the eligibility survey to identify those who meet the inclusion criteria. This step serves to minimize false participation. Eligible participants will then be sent a consent form to provide an email address or a phone number to receive the e-Delphi surveys. This information is not linked to the survey responses.

    E-Delphi Methods

    The online surveys for each round of the e-Delphi will be delivered through Qualtrics (Provo, United States). Qualtrics is a secure Web-based survey platform and allows for anonymous participation. All data collected in Qualtrics will be stored in Canada and are protected with high-end firewalls and are treated confidentially. We will own and manage all the data collected via Qualtrics. All identified and interested members of the expert panel will be sent a personalized link to fill out each round of the survey. Although a personalized link will be used to access the survey, personal information will not be stored, and contact details will be removed in the completed survey.

    Rounds

    In the first round, panelists will review and consider the selected STI testing interventions. The preamble for each intervention will include a brief description and a list of considerations; panelists will also be given the opportunity to provide their opinion in an open text field. Panelists will be asked to rate each bacterial STI intervention on a 7-point Likert scale: 1=definitely not a priority, 2=not a priority, 3=somewhat not a priority, 4=undecided, 5=somewhat of a priority, 6=a priority, 7=definitely a priority. An open text field will be available for panelists to explain their priority choice. Finally, panelists will be invited to suggest an alternative STI testing intervention that was not listed but that they believe to be important. Panelists will also report their sociodemographic characteristics, specifically age, race/ethnicity, transgender identity, sexual orientation, and HIV serostatus (optional).

    In the second round, panelists will be asked to prioritize the same interventions they considered in round 1. However, this time they will see the distribution of responses from the previous round (eg, the proportions of persons selecting each of the Likert scale options), as well as a summary of the rationale for prioritizing that particular intervention. Those who select a priority rating that does not agree with the majority will be asked to provide details for their choice with an open text field question, such as the following example: “Most guys chose an Online App for Booking Bacterial STI testing as ‘A priority’. Why did you not prioritize this option?”. Panelists will also be asked to rank their top 3 interventions that they consider the highest priority. If consensus is achieved after round 2, then that intervention option will be removed for round 3 prioritization (although they would still be included as options for respondents’ top 3 ranked interventions).

    In round 3, panelists will again rate and rank the interventions alongside summaries of the prioritization and ranking responses from round 2, that is, they will have a third chance to rate interventions and a second chance to rank them. Those who rank a bacterial intervention component different from the majority will be asked to provide details for their choice with an open text field question: “One or more of your responses is a different priority than the other experts, please explain why you chose your response.”

    Compensation

    Each survey round will be accessible for 2 weeks, with 1-week breaks to conduct the analyses and provide response summaries for the subsequent round. To encourage retention throughout, we will provide increasing incentives at rates of Can $25, $35, and $40 for completion of rounds 1, 2, and 3 (total Can $100 for all 3 rounds), respectively. To receive this compensation, panelists will be provided with a link at the end of their survey which will take them to a reimbursement form to fill in contact information. The contact information will be collected and stored separately from study data and is asked for the purposes of reimbursement only.

    Analysis

    The analysis of responses from each round will occur iteratively and independently for the Community and Provider panels. The primary purpose is to achieve consensus within each of the panels to identify which subset of the proposed 8 interventions have the greatest potential for increasing testing levels among MSM in Toronto. As there is no standard definition of consensus for Delphi studies [35], we will define consensus as having ≥60% members (≥18/30) indicate a preference within 2 adjacent response points (+/−1) on a 7-point Likert scale. We will supplement the quantitative analyses with a thematic analysis of open-ended text data [36,37] to better understand disagreements within and between panels, should this occur. The top 3 ranked interventions will be determined based on frequency counts.


    Progress to date includes knowledge synthesis and selection of candidate interventions for the e-Delphi surveys. In our updated literature review, we identified 246 publications, of which 88 were in the original published systematic reviews [24,34]. After applying our inclusion and exclusion criteria, 203 publications were excluded because of the following reasons: (1) the article did not describe an intervention aimed at increasing bacterial STI testing (n=176), (2) the intervention was implemented in a rural setting (n=1), (3) the study population included only women or heterosexual participants (n=22), and (4) study protocol of intervention (n=4).

    In our final review, we included 43 publications describing 49 interventions. The largest number of publications were from Australia (n=15). Only 2 publications were from Canada. Effectiveness was categorized for these 49 interventions (Multimedia Appendix 1). A total of 37 interventions were deemed effective, with 24 moderately effective and 13 very effective.

    In the category of streamlined testing among asymptomatic individuals, routine testing was the predominant intervention, with all 9 effective, followed by Web-based or home-based testing, with 6 out of 7 effective. A total of 8 effective interventions in this category incorporated testing of extragenital sites for chlamydia and gonorrhea, with 7 employing self-collection of anal swabs.

    In the category of the client-targeted interventions, the most common intervention was client reminders, with 6 of 8 being effective, followed by 3 effective client counseling interventions. Both client incentive interventions (n=2) were ineffective.

    In the category of the provider-targeted interventions, audit and feedback (n=2) and provider alerts (n=2) were effective. The effectiveness of provider education interventions was variable with 1 very effective and 1 ineffective study.

    On the basis of the above evidence for effective interventions and emerging findings from our focus groups and provider survey, we selected the following interventions and their rationales for inclusion in the e-Delphi surveys (Table 2).

    Table 2. Descriptions and rationale for interventions.
    Category and interventionDescriptionRationale for inclusionSummary of effectiveness
    Streamlined testing among asymptomatic individuals

    		Routine testingClients are tested at every visit using standing orders.Routine STIa testing was effective in improving STI testing rates in all 9 studies identified by reducing stigma and normalizing testing.Very effective: 5/10 studies [38-42]; Moderately effective: 4/10 studies [43-46]; Unknown effectiveness: 1/10 studies [47]
    Web-based/home-based testingSTI tests are ordered on the Web, client can opt for in-person lab testing or mailed self-testing kits.Web-based or home testing was effective in improving STI testing rates in most studies, identified by increasing convenience and reducing the need to see a health care provider.Very effective: 2/11 studies [48,49]; Moderately effective: 4/11 studies [50-53]; Ineffective: 1/11 studies [54]; Unknown: 4/11 studies [55-58]
    Nurse/nonphysician-led testingA health care provider who is not a doctor (such as a nurse) collects information on a client’s sexual history and symptoms and collects samples.A total of 2 identified studies demonstrated that having nurses provide testing is effective in improving STI testing rates with reducing the need to see a doctor and increased convenience.Moderately effective: 2/2 studies [59]
    Express testing at clinics with self-collection of sampleOn the basis of a self-completed questionnaire on sexual history and symptoms, clients are directed to self-collected testing if asymptomatic.Express testing was effective in improving STI testing rates in 1 study by increasing convenience and reducing the need to see a health care provider.Moderately effective: 1/2 studies (express clinic with self-collection of some specimens) [60], 1/2 studies (self-collection of samples in clinic) [61]
    Client-targeted



    		Client remindersClient gives permission to clinic to receive reminders via short message service text message, email, or mailed letter.Client reminders were effective in improving STI testing rates in most studies identified. Clients are notified to test, and it becomes part of the health care routine.Very effective: 4/9 studies [62-65]; Moderately effective: 3/9 studies [66-68]; Ineffective: 1/9 studies [69]; Unknown effectiveness: 1/9 studies [70]
    Web-based educational and testing booking appClients find information about bacterial STIs on an app/website and use it to book an appointment at a clinic.A Web-based personally controlled health system manager was effective in improving STI testing rates by increasing knowledge and convenience.Moderately effective: 1/1 study [71]
    Provider-targeted



    		Provider audit and feedbackProviders receive a report on their own STI testing practices.Providing feedback reports on STI testing rates was effective in improving STI testing rates by identifying good performance and areas to improve.Very effective: 1/2 studies [72]; Moderately effective: 1/2 studies [73]
    Provider remindersProviders receive alerts through electronic medical record systems to prompt an offer of STI testing.Provider reminders to test clients at increased risk of STI acquisition were effective in improving STI testing rates by notifying provider to offer STI testing.Moderately effective: 2/2 studies [74,75]

    aSTI: sexually transmitted infection.


    Overview

    By conducting an e-Delphi exercise with community members and providers in Toronto, Canada, we will produce evidence to allow for community-directed, informed choices regarding the implementation of novel STI testing interventions for MSM. To maximize the chances for successful implementation, we first need to better understand the barriers to access testing and the intervention contexts in other settings, then work with community partners to determine which candidate intervention(s) would best overcome these barriers and how they may need to be adapted for the local context. Interventions must be acceptable to members of communities that they intend to serve [76]. Our choice of the e-Delphi method to prioritize potential interventions allows community members to have an equal voice alongside professional stakeholders.

    Our plan is not without potential pitfalls. One challenge was selecting interventions for consideration by panelists. Our choices were based on an extensive literature review and qualitative and quantitative data on local patient and provider barriers and facilitators for STI testing. Nevertheless, it is possible that we overlooked or excluded interventions that could be effective in our setting. A second challenge is ensuring diversity in representation among members of the Community and Provider panels, as MSM communities are particularly heterogeneous in large urban cities, such as Toronto. We will seek out as representative a sample as possible to identify diverse perspectives but acknowledge that the opinions of panelists are unlikely to capture all possible views within a small sample size. Motivated panelists are crucial to ensure carefully considered ratings and high response and retention throughout the rounds. We will maximize input by limiting the number of questions asked and providing increasing incentives for completing each round. The potential for false participation is a concern (eg, participation by individuals pretending to meet the inclusion criteria), particularly for the establishment of the Community Panel. Procedures will minimize false participation including study promotion and direct invitations via established MSM community channels, an eligibility questionnaire step as we form the panel (without compensation), and a sliding scale of compensation, such that the highest amount is provided for completion of the third and final questionnaire. Finally, consensus may not be reached at the end of the 3 rounds within and between each expert panel. However, in conducting the Delphi panels, we will gain a better understanding of the interventions with the greatest potential for improving local STI testing services for MSM in Toronto and be better positioned to anticipate potential roadblocks to implementation.

    Conclusions

    Innovative approaches to health care delivery are needed to produce the required increases in bacterial STI testing levels, frequency, and the use of appropriate testing technologies in ways that are engaging, nonstigmatizing, and acceptable for MSM [21]. Many community- and clinic-based bacterial STI test interventions have demonstrated effectiveness in the international literature [24,34] and/or are being attempted as pilot projects in Canada [22,55]. Yet the choice of intervention to implement can be daunting without local evidence regarding the best fit. The results of the proposed e-Delphi will characterize the intervention climate including perceived needs for changes to test delivery, relative priorities for change, and readiness for implementation [77]. Our approach may be transferable to other settings where stakeholder input is needed to manage sensitive areas of concern.

    Acknowledgments

    This study is funded by an HIV/AIDS community–based catalyst grant from the Canadian Institutes of Health Research (CIHR, FRN 150082) and a CIHR Foundation Award to ANB (FDN 148432).

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Effectiveness of 49 interventions from 43 articles.

    PDF File (Adobe PDF File), 651KB
    ‎

    Multimedia Appendix 2

    Peer-reviewer report from the Canadian Institutes of Health Research.

    PDF File (Adobe PDF File), 448KB
    1. Cole ZD, Donohoe HM, Stellefson ML. Internet-based Delphi research: case based discussion. Environ Manage 2013 Mar;51(3):511-523 [FREE Full text] [CrossRef] [Medline]
    2. Brady SR. The Delphi method. In: Jason LA, Glenwick DS, editors. Handbook Of Methodological Approaches To Community-Based Research: Qualitative, Quantitative, And Mixed Methods. New York City: Oxford University Press; 2016:61-68.
    3. Khodyakov D, Hempel S, Rubenstein L, Shekelle P, Foy R, Salem-Schatz S, et al. Conducting online expert panels: a feasibility and experimental replicability study. BMC Med Res Methodol 2011;11:174 [FREE Full text] [CrossRef] [Medline]
    4. Coulter I, Elfenbaum P, Jain S, Jonas W. SEaRCH™ expert panel process: streamlining the link between evidence and practice. BMC Res Notes 2016 Jan 7;9:16 [FREE Full text] [CrossRef] [Medline]
    5. Burchell AN, Allen VG, Gardner SL, Moravan V, Tan DH, Grewal R, OHTN Cohort Study Team. High incidence of diagnosis with syphilis co-infection among men who have sex with men in an HIV cohort in Ontario, Canada. BMC Infect Dis 2015 Aug 20;15:356 [FREE Full text] [CrossRef] [Medline]
    6. Burchell AN, Gardner SL, Mazzulli T, Manno M, Raboud J, Allen VG, et al. Hepatitis C virus seroconversion among HIV-positive men who have sex with men with no history of injection drug use: results from a clinical HIV cohort. Can J Infect Dis Med Microbiol 2015;26(1):17-22 [FREE Full text] [CrossRef] [Medline]
    7. Burchell AN, Grewal R, Allen VG, Gardner SL, Moravan V, Bayoumi AM, OHTN Cohort Study Team. Modest rise in chlamydia and gonorrhoea testing did not increase case detection in a clinical HIV cohort in Ontario, Canada. Sex Transm Infect 2014 Dec;90(8):608-614 [FREE Full text] [CrossRef] [Medline]
    8. Burchell AN, Allen VG, Moravan V, Gardner SL, Raboud J, Tan DH, OHTN Cohort Study Research Team. Patterns of syphilis testing in a large cohort of HIV patients in Ontario, Canada, 2000-2009. BMC Infect Dis 2013 May 28;13:246 [FREE Full text] [CrossRef] [Medline]
    9. Wilton J. Gay Men's Sexual Health Alliance. 2015. The (Re)emergence of STIs Among MSM   URL: http://www.gmsh.ca/about-the-gmsh/news/~15-The-re-emergence-of-STIs-among-MSM; [WebCite Cache]
    10. Taylor MM, Aynalem G, Olea LM, He P, Smith LV, Kerndt PR. A consequence of the syphilis epidemic among men who have sex with men (MSM): neurosyphilis in Los Angeles, 2001-2004. Sex Transm Dis 2008 May;35(5):430-434. [CrossRef] [Medline]
    11. Government of Canada. 2011. Important Notice - Public Health Information Update on the Treatment for Gonococcal Infection   URL: https:/​/www.​canada.ca/​en/​public-health/​services/​infectious-diseases/​sexual-health-sexually-transmitted-infections/​canadian-guidelines/​alerts/​2011/​important-notice-public-health-information-update-on-treatment-gonococcal-infection.​html [WebCite Cache]
    12. Guerra F, Yu Y, Macdonald L, Menon S, Pritchard J, Whelan M, et al. Public Health Ontario. 2018. Ontario Gonorrhea Testing and Treatment Guide, 2nd Edition   URL: https://www.publichealthontario.ca/-/media/documents/guide-gonorrhea-testing-treatment.pdf?la=en [WebCite Cache]
    13. Government of Canada. 2016. Canadian Guidelines on Sexually Transmitted Infections   URL: https:/​/www.​canada.ca/​en/​public-health/​services/​infectious-diseases/​sexual-health-sexually-transmitted-infections/​canadian-guidelines.​html
    14. Public Health Agency of Canada. Centre for Communicable Diseases and Infection Control, Infectious Disease Prevention and Control Branch, Public Health Agency of Canada. 2017. Report on Sexually Transmitted Infections in Canada: 2013-2014   URL: https:/​/www.​canada.ca/​en/​public-health/​services/​publications/​diseases-conditions/​report-sexually-transmitted-infections-canada-2013-14.​html
    15. Public Health Ontario. 2017 Jan 23. Public Health Ontario Rounds: STI Series – Session 1 Overview of bacterial sexually transmitted infections   URL: https://www.publichealthontario.ca/-/media/documents/sti-series-session1.pdf?la=en
    16. Public Health Ontario. 2017 Mar 20. Public Health Ontario Rounds: STI Series – Session 3 Infectious Syphilis   URL: https://www.publichealthontario.ca/-/media/documents/sti-series-session3.pdf?la=en [WebCite Cache]
    17. Public Health Ontario. 2018. Reportable Disease Trends in Ontario: Chlamydia, 2005-2017, Males, Toronto [web tool]   URL: https:/​/www.​publichealthontario.ca/​en/​data-and-analysis/​infectious-disease/​reportable-disease-trends-annually#/​11
    18. Public Health Ontario. 2018. Reportable Disease Trends in Ontario: Gonorrhea, 2005-2017, Males, Toronto [web tool]   URL: https:/​/www.​publichealthontario.ca/​en/​data-and-analysis/​infectious-disease/​reportable-disease-trends-annually#/​18
    19. Public Health Ontario. 2018. Reportable Disease Trends in Ontario: Syphilis, 2005-2017, Males, Toronto [web tool]   URL: https:/​/www.​publichealthontario.ca/​en/​data-and-analysis/​infectious-disease/​reportable-disease-trends-annually#/​54
    20. Gesink D, Wang S, Norwood T, Sullivan A, Al-Bargash D, Shahin R. Spatial epidemiology of the syphilis epidemic in Toronto, Canada. Sex Transm Dis 2014 Nov;41(11):637-648. [CrossRef] [Medline]
    21. Fairley CK, Law M, Chen MY. Eradicating syphilis, hepatitis C and HIV in MSM through frequent testing strategies. Curr Opin Infect Dis 2014 Feb;27(1):56-61. [CrossRef] [Medline]
    22. Burchell AN, Allen VG, Grewal R, MacPherson PA, Rachlis A, Walmsley S, et al. Enhanced syphilis screening among HIV-positive men (ESSAHM): a study protocol for a clinic-randomized trial with stepped wedge design. Implement Sci 2016 Jan 16;11:8 [FREE Full text] [CrossRef] [Medline]
    23. Grewal R, Allen VG, Gardner S, Moravan V, Tan DH, Raboud J, OHTN Cohort Study Research Team. Serosorting and recreational drug use are risk factors for diagnosis of genital infection with chlamydia and gonorrhoea among HIV-positive men who have sex with men: results from a clinical cohort in Ontario, Canada. Sex Transm Infect 2017 Feb;93(1):71-75 [FREE Full text] [CrossRef] [Medline]
    24. Bernstein KT, Chow JM, Pathela P, Gift TL. Bacterial sexually transmitted disease screening outside the clinic--implications for the modern sexually transmitted disease program. Sex Transm Dis 2016 Feb;43(2 Suppl 1):S42-S52 [FREE Full text] [CrossRef] [Medline]
    25. British Association for Sexual Health and HIV. 2018. BASHH Guidelines   URL: https://www.bashh.org/guidelines [accessed 2018-11-19] [WebCite Cache]
    26. Australian STI Management Guidelines. 2018. MSM - Men who have sex with men   URL: http://www.sti.guidelines.org.au/populations-and-situations/msm#testing-advice; [accessed 2018-11-19] [WebCite Cache]
    27. Centers for Disease Control and Prevention. 2018. 2015 Sexually Transmitted Diseases Treatment Guidelines   URL: https://www.cdc.gov/std/tg2015/specialpops.htm#MSM; [accessed 2018-11-19] [WebCite Cache]
    28. Marcus JL, Bernstein KT, Kohn RP, Liska S, Philip SS. Infections missed by urethral-only screening for chlamydia or gonorrhea detection among men who have sex with men. Sex Transm Dis 2011 Oct;38(10):922-924. [CrossRef] [Medline]
    29. Golub S. STI Data From Community-Based PrEP Implementation Suggest Changes to CDC Guidelines. In: It's Complicated: Renal Function and STIs in PrEP Users. 2016 Presented at: Conference on Retroviruses and Opportunistic Infections; February 22-25, 2016; Boston, Massachusetts.
    30. Statistics Canada. 2017. Toronto, C [Census subdivision], Ontario and Toronto, CDR [Census division], Ontario (table). Census Profile. 2016 Census. Statistics Canada Catalogue no. 98-316-X2016001   URL: https://tinyurl.com/y2adxhzd [WebCite Cache]
    31. Rana J, Grewal R, Reinhard R, Burchell A. Narrative Review of Interventions to Improve Bacterial STI Testing in Men, Particularly Gay, Bisexual and Other Men Who Have Sex (gbMSM). In: 27th Annual Canadian Conference on HIV/AIDS Research. 2018 Presented at: CAHR'18; April 26-29, 2018; Vancouver, Canada p. 149.
    32. Burchell AN, Gesink D, Logie C, Nelson L, Rana J, Wang S, et al. Community Perspectives on Ideal Bacterial STI Testing Services for Gay, Bisexual and Other Men Who Have Sex With Men (MSM) in Toronto, Canada. In: 28th Annual Canadian Conference on HIV/AIDS Research. 2019 Presented at: CAHR'19; May 9-12, 2019; Saskatoon, Canada p. 231.
    33. Burchell AN, Rana J, Guiang C, Shahin R, Brunetta J, Mitterni L, et al. Providers' Views on Barriers and Facilitators of Bacterial STI Testing Among Gay, Bisexual and Other Men Who Have Sex With Men (MSM) Who are Living With or at Risk of HIV. In: 28th Annual Canadian Conference on HIV/AIDS Research. 2019 Presented at: CAHR'19; May 9-12, 2019; Saskatoon, Canada p. 183.
    34. Taylor MM, Frasure-Williams J, Burnett P, Park IU. Interventions to improve sexually transmitted disease screening in clinic-based settings. Sex Transm Dis 2016 Feb;43(2 Suppl 1):S28-S41. [CrossRef] [Medline]
    35. von der Gracht HA. Consensus measurement in Delphi studies: review and implications for future quality assurance. Technol Forecast Soc Change 2012 Oct;79(8):1525-1536. [CrossRef]
    36. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol 2006 Jan;3(2):77-101. [CrossRef]
    37. Attride-Stirling J. Thematic networks: an analytic tool for qualitative research. Qual Res 2001 Dec;1(3):385-405. [CrossRef]
    38. Bissessor M, Fairley CK, Leslie D, Howley K, Chen MY. Frequent screening for syphilis as part of HIV monitoring increases the detection of early asymptomatic syphilis among HIV-positive homosexual men. J Acquir Immune Defic Syndr 2010 Oct;55(2):211-216. [CrossRef] [Medline]
    39. Callander D, Baker D, Chen M, Guy R. Including syphilis testing as part of standard HIV management checks and improved syphilis screening in primary care. Sex Transm Dis 2013 Apr;40(4):338-340. [CrossRef] [Medline]
    40. Cohen CE, Winston A, Asboe D, Boag F, Mandalia S, Azadian B, et al. Increasing detection of asymptomatic syphilis in HIV patients. Sex Transm Infect 2005 Jun;81(3):217-219 [FREE Full text] [CrossRef] [Medline]
    41. Andersen B, Eidner PO, Hagensen D, Lomborg S, Hoff G. Opportunistic screening of young men for urogenital chlamydia trachomatis infection in general practice. Scand J Infect Dis 2005;37(1):35-39. [CrossRef] [Medline]
    42. Tebb KP, Pantell RH, Wibbelsman CJ, Neuhaus JM, Tipton AC, Pecson SC, et al. Screening sexually active adolescents for chlamydia trachomatis: what about the boys? Am J Public Health 2005 Oct;95(10):1806-1810. [CrossRef] [Medline]
    43. Botes LP, McAllister J, Ribbons E, Jin F, Hillman RJ. Significant increase in testing rates for sexually transmissible infections following the introduction of an anal cytological screening program, targeting HIV-positive men who have sex with men. Sex Health 2011 Mar;8(1):76-78. [CrossRef] [Medline]
    44. Lawton BA, Rose SB, Elley CR, Bromhead C, MacDonald EJ, Baker MG. Increasing the uptake of opportunistic chlamydia screening: a pilot study in general practice. J Prim Health Care 2010 Sep;2(3):199-207. [CrossRef] [Medline]
    45. Graham S, Guy RJ, Wand HC, Kaldor JM, Donovan B, Knox J, et al. A sexual health quality improvement program (SHIMMER) triples chlamydia and gonorrhoea testing rates among young people attending aboriginal primary health care services in Australia. BMC Infect Dis 2015 Sep 2;15:370 [FREE Full text] [CrossRef] [Medline]
    46. Patton ME, Kirkcaldy RD, Chang DC, Markman S, Yellowman M, Petrosky E, et al. Increased gonorrhea screening and case finding after implementation of expanded screening criteria-urban Indian health service facility in Phoenix, Arizona, 2011-2013. Sex Transm Dis 2016 Dec;43(6):396-401. [CrossRef] [Medline]
    47. Ritchie S, Henley R, Hilton J, Handy R, Ingram J, Mundt S, et al. Uptake, yield and resource requirements of screening for asymptomatic sexually transmissible infections among HIV-positive people attending a hospital outpatient clinic. Sex Health 2014 Mar;11(1):67-72. [CrossRef] [Medline]
    48. Andersen B, Olesen F, Møller JK, Østergaard L. Population-based strategies for outreach screening of urogenital chlamydia trachomatis infections: a randomized, controlled trial. J Infect Dis 2002 Jan 15;185(2):252-258. [CrossRef] [Medline]
    49. Reagan MM, Xu H, Shih SL, Secura GM, Peipert JF. A randomized trial of home versus clinic-based sexually transmitted disease screening among men. Sex Transm Dis 2012 Nov;39(11):842-847 [FREE Full text] [CrossRef] [Medline]
    50. Scholes D, Heidrich FE, Yarbro P, Lindenbaum JE, Marrazzo JM. Population-based outreach for chlamydia screening in men: results from a randomized trial. Sex Transm Dis 2007 Nov;34(11):837-839. [CrossRef] [Medline]
    51. Kløvstad H, Natås O, Tverdal A, Aavitsland P. Systematic screening with information and home sampling for genital chlamydia trachomatis infections in young men and women in Norway: a randomized controlled trial. BMC Infect Dis 2013 Jan 23;13:30 [FREE Full text] [CrossRef] [Medline]
    52. Smith KS, Hocking JS, Chen MY, Fairley CK, McNulty AM, Read P, et al. Dual intervention to increase chlamydia retesting: a randomized controlled trial in three populations. Am J Prev Med 2015 Jul;49(1):1-11. [CrossRef] [Medline]
    53. Kersaudy-Rahib D, Lydié N, Leroy C, March L, Bébéar C, Arwidson P, et al. Chlamyweb study II: a randomised controlled trial (RCT) of an online offer of home-based sampling in France. Sex Transm Infect 2017 Dec;93(3):188-195. [CrossRef] [Medline]
    54. van den Broek IV, van Bergen JE, Brouwers EE, Fennema JS, Götz HM, Hoebe CJ, et al. Effectiveness of yearly, register based screening for chlamydia in the Netherlands: controlled trial with randomised stepped wedge implementation. Br Med J 2012 Jul 5;345:e4316 [FREE Full text] [CrossRef] [Medline]
    55. Gilbert M, Salway T, Haag D, Fairley CK, Wong J, Grennan T, et al. Use of GetCheckedOnline, a comprehensive web-based testing service for sexually transmitted and blood-borne infections. J Med Internet Res 2017 Dec 20;19(3):e81 [FREE Full text] [CrossRef] [Medline]
    56. Mann TA, Uddin Z, Hendriks AM, Bouchard CJ, Etches VG. Get tested why not? A novel approach to internet-based chlamydia and gonorrhea testing in Canada. Can J Public Health 2013 Mar 7;104(3):e205-e209. [CrossRef] [Medline]
    57. Kwan KS, Jachimowicz EA, Bastian L, Marshall L, Mak DB. Online chlamydia testing: an innovative approach that appeals to young people. Med J Aust 2012 Sep 3;197(5):287-290. [CrossRef] [Medline]
    58. Götz HM, van den Broek IV, Hoebe CJ, Brouwers EE, Pars LL, Fennema JS, et al. High yield of reinfections by home-based automatic rescreening of chlamydia positives in a large-scale register-based screening programme and determinants of repeat infections. Sex Transm Infect 2013 Feb;89(1):63-69. [CrossRef] [Medline]
    59. Snow AF, Vodstrcil LA, Fairley CK, El-Hayek C, Cummings R, Owen L, et al. Introduction of a sexual health practice nurse is associated with increased STI testing of men who have sex with men in primary care. BMC Infect Dis 2013 Jul 1;13:298 [FREE Full text] [CrossRef] [Medline]
    60. Knight V, Ryder N, Guy R, Lu H, Wand H, McNulty A. New xpress sexually transmissible infection screening clinic improves patient journey and clinic capacity at a large sexual health clinic. Sex Transm Dis 2013 Jan;40(1):75-80. [CrossRef] [Medline]
    61. Barbee LA, Tat S, Dhanireddy S, Marrazzo JM. Implementation and operational research: effectiveness and patient acceptability of a sexually transmitted infection self-testing program in an HIV care setting. J Acquir Immune Defic Syndr 2016 Dec 1;72(2):e26-e31 [FREE Full text] [CrossRef] [Medline]
    62. Malotte CK, Ledsky R, Hogben M, Larro M, Middlestadt S, Lawrence JS, GCAP Study Group. Comparison of methods to increase repeat testing in persons treated for gonorrhea and/or chlamydia at public sexually transmitted disease clinics. Sex Transm Dis 2004 Nov;31(11):637-642. [CrossRef] [Medline]
    63. Downing SG, Cashman C, McNamee H, Penney D, Russell DB, Hellard ME. Increasing chlamydia test of re-infection rates using SMS reminders and incentives. Sex Transm Infect 2013 Feb;89(1):16-19. [CrossRef] [Medline]
    64. Bourne C, Knight V, Guy R, Wand H, Lu H, McNulty A. Short message service reminder intervention doubles sexually transmitted infection/HIV re-testing rates among men who have sex with men. Sex Transm Infect 2011 Apr;87(3):229-231. [CrossRef] [Medline]
    65. Nyatsanza F, McSorley J, Murphy S, Brook G. 'It's all in the message': the utility of personalised short message service (SMS) texts to remind patients at higher risk of STIs and HIV to reattend for testing-a repeat before and after study. Sex Transm Infect 2016 Dec;92(5):393-395. [CrossRef] [Medline]
    66. Zou H, Fairley CK, Guy R, Bilardi J, Bradshaw CS, Garland SM, et al. Automated, computer generated reminders and increased detection of gonorrhoea, chlamydia and syphilis in men who have sex with men. PLoS One 2013;8(4):e61972 [FREE Full text] [CrossRef] [Medline]
    67. Paneth-Pollak R, Klingler EJ, Blank S, Schillinger JA. The elephant never forgets; piloting a chlamydia and gonorrhea retesting reminder postcard in an STD clinic setting. Sex Transm Dis 2010 Jun;37(6):365-368. [CrossRef] [Medline]
    68. Fernando KA, Fowler T, Harding J, Flew S, Caley M, Phattey J, et al. Detecting re-infection in patients after an initial diagnosis of gonorrhoea: is routine recall for re-screening useful? Int J STD AIDS 2015 Aug;26(9):640-647. [CrossRef] [Medline]
    69. Burton J, Brook G, McSorley J, Murphy S. The utility of short message service (SMS) texts to remind patients at higher risk of STIs and HIV to reattend for testing: a controlled before and after study. Sex Transm Infect 2014 Feb;90(1):11-13. [CrossRef] [Medline]
    70. Harte D, Mercey D, Jarman J, Benn P. Is the recall of men who have sex with men (MSM) diagnosed as having bacterial sexually transmitted infections (STIs) for re-screening a feasible and effective strategy? Sex Transm Infect 2011 Dec;87(7):577-582. [CrossRef] [Medline]
    71. Mortimer NJ, Rhee J, Guy R, Hayen A, Lau AY. A web-based personally controlled health management system increases sexually transmitted infection screening rates in young people: a randomized controlled trial. J Am Med Inform Assoc 2015 Jul;22(4):805-814. [CrossRef] [Medline]
    72. Kalwij S, French S, Mugezi R, Baraitser P. Using educational outreach and a financial incentive to increase general practices' contribution to chlamydia screening in south-east London 2003-2011. BMC Public Health 2012 Sep 18;12:802 [FREE Full text] [CrossRef] [Medline]
    73. Merritt TD, Durrheim DN, Hope K, Byron P. General practice intervention to increase opportunistic screening for chlamydia. Sex Health 2007 Dec;4(4):249-251. [CrossRef] [Medline]
    74. Lister NA, Smith A, Fairley CK. Introduction of screening guidelines for men who have sex with men at an STD clinic, the Melbourne sexual health centre, Australia. Sex Health 2005;2(4):241-244. [CrossRef] [Medline]
    75. Bissessor M, Fairley CK, Leslie D, Chen MY. Use of a computer alert increases detection of early, asymptomatic syphilis among higher-risk men who have sex with men. Clin Infect Dis 2011 Jul 1;53(1):57-58. [CrossRef] [Medline]
    76. Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health 1998;19:173-202. [CrossRef] [Medline]
    77. Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009 Aug 7;4:50 [FREE Full text] [CrossRef] [Medline]

    ‎
    AD: absolute difference
    CIHR: Canadian Institutes of Health Research
    MSM: men who have sex with men
    RD: relative difference
    STI: sexually transmitted infection

    Edited by G Eysenbach; submitted 28.02.19; peer-reviewed by P Millson, C van Gemert, M Zlotorzynska, T Risling; comments to author 11.05.19; revised version received 28.05.19; accepted 29.05.19; published 04.07.19

    Copyright

    ©Ann N Burchell, Ryan Lisk, Anna Yeung, Jayoti Rana, Jean Bacon, Jason Brunetta, Mark Gilbert, Dionne Gesink, Ramandip Grewal, Charlie B Guiang, Michael Kwag, Carmen H Logie, Leo Mitterni, Rita Shahin, Darrell HS Tan. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 04.07.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.