This paper is in the following e-collection/theme issue:

Grant Proposals (funded, non-ehealth) (69) Knowledge Translation and Implementation Science (398)

Published on in Vol 8, No 5 (2019): May

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/14382, first published .
Optimizing the Impact of Public-Academic Partnerships in Fostering Policymakers’ Use of Research Evidence: Proposal to Test a Conceptual Framework

Optimizing the Impact of Public-Academic Partnerships in Fostering Policymakers’ Use of Research Evidence: Proposal to Test a Conceptual Framework

Optimizing the Impact of Public-Academic Partnerships in Fostering Policymakers’ Use of Research Evidence: Proposal to Test a Conceptual Framework

Authors of this article:

Christina D Kang-Yi1, 2 Author Orcid Image
Article Authors Cited by (4) Tweetations Metrics

    Proposal

    1Center for Mental Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States

    2Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, United States

    Corresponding Author:

    Christina D Kang-Yi, PhD

    Center for Mental Health

    Perelman School of Medicine

    University of Pennsylvania

    3535 Market Street, 3rd Floor

    Philadelphia, PA, 19104

    United States

    Phone: 1 215 746 6715

    Email: ckangyi@upenn.edu


    Background: Previous research has reported that public-academic partnerships (PAPs) can effectively promote PAP leaders’ use of research evidence in improving youth outcomes. However, the existing literature has not yet clarified whether and how PAP leaders’ use of research evidence evolves along the PAP life cycle and whether PAP partners’ concordant perceptions of usefulness of their PAP has an impact on PAP leaders’ use of research evidence. Developing a conceptual framework that recognizes the PAP life cycle and empirically identifying contexts and mechanisms of PAPs that promote PAP leaders’ use of research evidence from the PAP life cycle perspective are imperative to guide researchers and policymakers to successfully lead PAPs and foster policymakers’ use of research evidence for improving youth outcomes.

    Objective: Utilizing an integrated framework of organizational life cycle perspective, a social partnership perspective, and a realist evaluation, this study examines the extent to which PAP development and PAP leaders’ use of research evidence can be characterized into life cycle stages and identifies PAP contexts and mechanisms that explain the progress of PAPs and PAP leaders’ use of research evidence through life cycle stages.

    Methods: Recruiting PAPs across the United States that aim to improve mental health and promote well-being of youth aged 12-25 years, the study conducts a document analysis and an online survey of PAPs to inform policymakers and academic researchers on the contexts and mechanisms to increase PAP sustainability and promote policymakers’ use of research evidence in improving youth outcomes.

    Results: Fifty-three PAPs that meet the recruitment criteria have been identified, and document review of PAPs and participant recruitment for the online survey of PAP experience have been conducted.

    Conclusions: This paper will help policymakers and researchers gain a deeper knowledge of the contexts and mechanisms for each PAP life cycle stage in order to optimize PAP leaders’ use of research evidence in achieving positive youth outcomes.

    International Registered Report Identifier (IRRID): DERR1-10.2196/14382

    JMIR Res Protoc 2019;8(5):e14382

    doi:10.2196/14382

    Keywords

    public-academic partnership; use of research evidence; youth; mental health; well-being 


    Background

    This proposed project aims to develop a conceptual framework to understand dynamic and complex public-academic partnerships (PAPs) and reveal contexts and mechanisms for each PAP life cycle stage in order to optimize PAP leaders’ use of research evidence in improving youth mental health and well-being. The proposed project defines a PAP as a partnership between the state and county policymakers (administrators and program directors) and researchers at academic institutes, formed to promote evidence-informed policymaking and practice. PAP leaders are policymakers who influence public identification of problems, design and implement programs, and make policy decisions as administrators and program directors.

    This project focuses specifically on PAPs that aim to improve mental health and promote well-being of youth aged 12-25 years. Many psychiatric disorders such as mood disorders, substance abuse problems, and schizophrenia develop during adolescence [1]. About 20% of US youths aged 13-16 years will experience a psychiatric disorder during their lifetime [2-4], and the rates are higher among youth served by the public care sector [5-7]. PAPs are vital to improve the health and well-being of vulnerable populations [8-11]. PAPs seek to bridge the historic divide and disconnect that has evolved among researchers and policymakers and improve the degree to which the knowledge generated by researchers is utilized for the benefit of the individuals being served by the public care sector [12,13]. A critical means through which PAPs accomplish this aim is the use of research evidence. Research evidence is defined as relevant conceptual frameworks or reviews and empirical findings from systematic qualitative, quantitative, or mixed research methods projects [14]. Use of research evidence is defined as acquiring, evaluating, and directly applying research evidence [15,16] and conceptually using research [17,18] to understand the nature of and frame social and community problems; to design and implement public services and programs; and to make policy decisions [19,20].

    Previous research has reported that PAPs can effectively promote PAP leaders’ use of research evidence in improving youths’ outcomes [12,20,10]. However, the existing literature has not yet informed the specific mechanisms of PAPs that promote PAP leaders’ use of research evidence in improving youths’ outcomes; whether and how PAP leaders’ use of research evidence evolves along the PAP life cycle; and whether PAP partners’ concordant perceptions of usefulness of their PAP has an impact on PAP leaders’ use of research evidence. Developing a conceptual framework that recognizes the PAP life cycle, and empirically identifying contexts and mechanisms of PAPs that promote PAP leaders’ use of research evidence from the PAP life cycle perspective is imperative to guide researchers and policymakers to successfully lead PAPs and foster policymakers’ use of research evidence in improving youth outcomes. This proposed project will examine the extent to which PAP development and PAP leaders’ use of research evidence can be characterized into life cycle stages, and identify PAP contexts and mechanisms that explain the progress of PAPs and PAP leaders’ use of research evidence through life cycle stages.

    Theoretical and Empirical Rationale

    Although a number of empirical studies have helped identify specific factors that are associated with PAP success and failure, there is no research evidence that offers a framework of contexts and mechanisms that should occur at each stage of PAP development in order to maximize the chances of success and increase PAP leaders’ use of research evidence. The present project proposes an integrated framework of social partnerships, organizational life cycle, and realist evaluation perspectives to address the complex contexts and corresponding mechanisms that result in successful sustainment of PAPs and foster PAP leaders’ use of research evidence, for PAPs that serve youth in the public care sector. This project classifies the PAP life cycle stages into initiated/not initiated, formed/failed to be formed, matured/not matured, and sustained/declined (Figure 1) [21-23].

    The social partnership perspective [7,22-28] posits that partnerships have common roots in their intended impact on some societal problems such as education, poverty, and health by building on the capabilities, resources, and expertise of each partner [25,27,28]. This perspective aligns well with PAP development, as PAPs are typically initiated to address social issues such as youth mental health and well-being. A social partnership perspective is that three key partnership processes should occur concurrently to evolve through stages such as initiation, formation, and maturity: issue crystallization; coalition building through mutual benefits and trust, top management support, convener’s role, and on-the-spot decision-making power; and purpose formulation through determining structure, goals, primary function, and process of setting agenda [20]. Although the perspective offers a compelling explanation for factors that support successful PAP initiation, formation, and maturity, the framework does not explain how PAP partnerships sustain. The organizational life cycle perspective [29-34] offers a longitudinal and sequenced approach to explain how PAPs grow and change over time. Organizations go through stages of birth, maturity, and decline and the goals, priorities, and definitions of organizational effectiveness differ across not only organizations but also these stages within organizations [29,34]. By applying this perspective, we can explain how PAP partnerships transition through life cycle stages and the potential presence of a sustained versus declined stage, which the social partnership perspective is missing. An integration of aspects of the organizational life cycle perspective with social partnership perspective contributes to the conceptualization of PAP progress by distinguishing specific life cycle stages relevant to the progress of organizations. The realist-evaluation approach [35-37] helps deepen the conceptualization of how the contexts of PAPs are involved in both their sustainability and use of research evidence. Realist evaluation focuses on three concepts—context, mechanism, and outcomes—which link together to form a context-mechanism-outcome (CMO) configuration (Figure 1). This proposed project adapts the definitions of context, mechanism, and outcome from the study of Jagosh et al [38] who used the realist evaluation approach to understand partnerships in community participatory research.

    ‎
    Figure 1. Proposed conceptual framework to understand public-academic partnerships as social partnerships transitioning through life cycle stages.
    View this figure

    Describing context, mechanism, and outcome relationships based on the realist evaluation framework helps generate and refine an explanatory theory, which, in this case, are the theory of how PAPs evolve and what promotes or inhibits PAP leaders’ use of research evidence (Multimedia Appendix 1). The features of a realist evaluation approach can be applied to PAPs to clarify the nature and contributions of contexts and mechanisms to PAP sustainability and PAP leaders’ use of research evidence according to the PAP life cycle stage in real-world settings.

    Neither social partnerships nor organizational life cycle perspectives recognize the embedding of PAPs in a system and do not offer a means to connect partnership contexts and mechanisms with PAP leaders’ use of research evidence. The realist evaluation approach is based on determining not just what approaches work, but more specifically, “what works, for whom, under what circumstances, and why and how” [35-37]. Thus, this perspective helps deepen the conceptualization of how the contexts of PAPs are involved in both their sustainability and use of research evidence.

    Considering that PAP leaders’ perception of their academic partners and coalition building with their academic partners can have an impact on PAP sustainability [16,25,39], PAP sustainability and PAP leaders’ use of research evidence are likely to be associated. However, the lack of empirical findings warrants further studies. The integrated conceptual framework that includes a theory-building process will contribute to further examination of the relationship between PAP sustainability and PAP leaders’ use of research evidence. Extending the previous research [16,25,39], this proposed project seeks to examine the following aspects:

    1. Do PAPs go through a life cycle of being initiated, formed, matured, and sustained?
      • Can the partnership processes (issue crystallization, coalition building, and purpose formulation) be traced through the PAP life cycle stages? If yes, how do partnership processes differ by PAP life cycle stage?
    2. Which factors promote or interfere in the progression and ultimate sustainability of PAPs?
      • Do PAP partners’ perceptions of alignment between PAP purpose formulation and their own organizational purpose formulation differ by PAP life cycle stage?
      • Do PAP partners’ perceptions of PAP coalition building differ by PAP life cycle stage?
    3. Are there different patterns of use of research evidence (in terms of obtaining, evaluating, and using research evidence) associated with each PAP life cycle?
    4. Which PAP factors promote or inhibit PAP leaders’ use of research evidence?
      • Does PAP leaders’ use of research evidence differ by PAP partners’ perceptions of alignment between PAP purpose formulation and their own organizational purpose formulation?
      • Does PAP leaders’ use of research evidence differ by PAP partners’ perceptions of PAP coalition building?

    Sampling and Participant Recruitment

    PAPs that are comprised of at least one or more state or local county child welfare or mental health agencies, and one or more academic researchers will be eligible for inclusion. In the proposed project, mental health includes both mental health and substance abuse. The PAPs can be formed on a project/program/intervention basis or as a consortium. PAPs will be included if their aims include improving mental health and well-being outcomes for youth aged 12-25 years. If multiple academic researchers from one academic research institute are working on separate projects/programs/interventions with the same public care agency, these partnerships will be counted as multiple PAPs, as the partnerships may be in different life cycle stages with different contexts and mechanisms that affect PAP leaders’ use of research evidence differently. If one academic researcher has multiple projects with one public care agency, multiple projects will be treated as a single PAP. PAPs that were terminated before 2007 will be excluded. PAPs considered to be in an “initiating” or “failed to be formed” stage will be excluded from this recruitment stage, as members of these PAPs are likely to be difficult to identify. PAPs situated outside the United States or focusing on youth outside the United States will be also excluded. A variety of strategies including online search and contacting partners through emails and phone calls will be undertaken to recruit as many eligible PAPs as possible and collect consistent information from all PAPs. If only the PAP leaders or the academic researchers of each PAP agree to participate, we will still include the PAP in the study for supplementary data.

    Document Review and Online Survey of Public-Academic Partnership Leaders and Academic Researchers

    The project team will invite all PAPs that meet the inclusion criteria to participate and concurrently conduct PAP document review and an online survey of PAP leaders and academic researchers. We will seek PAP documents that include information on PAP structure, goals, primary function, actor roles, process of setting agenda, and funding sources for the PAP processes. The document review will apply the iterative CMO configuration process of the realist evaluation approach [36,38]. An iterative CMO configuration process will be conducted, in which the PAP documents are reviewed by the project team utilizing a review protocol drafted to serve as the guiding tool in the CMO configuration of PAP life cycle based on the potential CMO of each PAP life cycle stage (Multimedia Appendix 1). PAP mechanisms and PAP leaders’ use of research evidence by life cycle stage will be detailed through the document review, and the protocol will be refined through multiple rounds of review of PAP documents.

    Online survey of PAP leaders and academic researchers will collect data on partners’ PAP experience and PAP leaders’ use of research evidence. The intent is to obtain over a 75% survey response rate. PAP Survey 1: The Structured Interview for Evidence Use (SIEU) scale [16] will determine PAP leaders’ engagement level of research evidence, which refers to the frequency of using various types of sources for research evidence; PAP leaders’ ratings of the importance of evaluating the validity, reliability, and relevance of research evidence; and various factors leading PAP leaders to use/ignore research evidence in deciding to adopt a new program or intervention. The original SIEU scale items will be used. The Input scale (20 items) assesses the source of the research evidence PAP leaders obtain. The Process scale assesses how PAP leaders evaluate the research evidence obtained and includes three subscales of self-assessment for validity and reliability of research evidence (10 items), reliance on others (5 items), and self-assessment for relevance (5 items). The Output scale (20 items) assesses if PAP leaders eventually use research evidence or ignore the evidence. The measurement asks respondents to indicate responses using a Likert-type scale ranging from 1 (not at all) to 5 (all the time) for the items contained in the Input scale, and a 5-point Likert-type scale ranging from 1 (not important) to 5 (very important) for the items contained in the Process and Output scales. The SIEU shows high internal consistency reliability (α=.88) [16]. PAP Survey 2: PAP Experience (Multimedia Appendix 2) will measure PAP leaders’ and academic researchers’ most recent PAP experience through items that address issue crystallization (clear issue pursued), purpose formulation (structure, goals, primary function, and agenda setting process), coalition building (mutual benefits and trust, top management support, convener’s role, and on-the spot decision-making power); PAP partners’ perceptions of their PAP life cycle stage; and PAP leaders’ use of research evidence. The online surveys will be built in and administered through the Research Electronic Data Capture, a secure Web-based data collection tool that includes data entry forms and Web surveying features.

    Analysis

    For the document review, the principal investigator and a research assistant will first independently classify the life cycle for each PAP (formed, but not yet matured; matured, but not reached a sustained stage yet; and sustained/declined) by applying the potential CMO of each PAP life cycle stage (Multimedia Appendix 1) and then review the classification of PAPs until a consensus is reached on the classification. Information on PAPs such as partnership structure, goals, and primary function can vary depending on the available documentation. For missing or incomplete data during the document review process, the project team will follow-up with academic researchers and PAP leaders through emails and phone calls to request and obtain the missing information. The online survey data will also complement the missing data from the document review as the domains of the survey questionnaire are consistent with those of the document review protocol.

    Concurrently, the online survey data on PAP leaders’ use of research evidence and experience with PAPs will be analyzed in relation to the CMO configuration process. For the online survey, reliability of the SIEU will be calculated by using the Cronbach α internal consistency for each of the subscales and the total scale. We will descriptively test for mean differences in PAP leaders’ engagement level of research evidence by (1) PAP partners’ rating of the level of alignment between PAP structure, goals, primary function, and process of setting agenda and their organizational structure, goals, primary function, and process of setting agenda; (2) PAP partners’ rating of the level of mutual benefit and trust, top management support, convener’s role, and on-the-spot decision-making power; and (3) PAP life cycle stage (formed, but not yet matured; matured, but not reached a sustained stage yet; and sustained/declined). Concordance levels of PAP leaders’ and academic researchers’ perceptions of PAP contexts and mechanisms will be calculated according to the PAP life cycle stage using Cohen κ coefficients and McNemar test [40]. Pearson product-moment correlations of the concordance levels and SIEU subscale and total scores will test the relationship between the concordance of partner’s perceptions and PAP leaders’ use of research evidence. The potential CMO of each PAP life cycle stage (Multimedia Appendix 1) will be refined through multiple rounds of review of PAP documents and quantitative data for the development of a middle-range theory.


    Fifty-three eligible PAPs have been identified, document review of 20 PAPs have been conducted, and 16 PAP researchers have been reached out for additional information. The principal investigator and the research assistant are in the process of classifying the life cycle for each PAP based on the document review by applying the potential CMO of each PAP life cycle stage (Multimedia Appendix 1). The classification of PAPs will be continued until consensus is reached on the classification. Concurrently, the project team is recruiting PAP leaders and academic researchers who will participate in the online survey and will conduct analysis of CMO PAP life cycle stages and its relationship to PAP leaders’ use of research evidence.


    The proposed project is expected to help policymakers and researchers gain a deeper knowledge of the contexts and mechanisms for each PAP life cycle stage in order to optimize PAP leaders’ use of research evidence in achieving positive youth outcomes. Although we will focus on youth mental health and well-being, our findings are likely to be relevant to other vulnerable populations. Future studies should include PAPs in an “initiating” or “failed to be formed” stages, as the PAPs are likely to provide valuable learning about attempted partnerships.

    Acknowledgments

    The author thanks Drs Yin-Ling Irene Wong, Jill Hamm, Bowen McBeath, Justin Jogosh, and Lawrence Palinkas who provided comments during proposal development, and the external reviewers for their comments. This study is supported by the William T Grant Foundation Officers’ Research Grant.

    Conflicts of Interest

    None declared.

    ‎

    Multimedia Appendix 1

    Potential Context, Mechanism and Outcome (CMO) of each Public-Academic Partnership (PAP) life cycle stage.

    PDF File (Adobe PDF File), 576KB
    ‎

    Multimedia Appendix 2

    Public-Academic Partnership (PAP) Survey Questionnaire.

    PDF File (Adobe PDF File), 81KB
    ‎

    Multimedia Appendix 3

    Peer-reviewer report from the William T Grant Foundation.

    PDF File (Adobe PDF File), 152KB
    1. Locke J, Kang-Yi C, Pellecchia M, Marcus S, Hadley T, Mandell DS. Ethnic Disparities in School-Based Behavioral Health Service Use for Children With Psychiatric Disorders. J Sch Health 2017 Dec;87(1):47-54 [FREE Full text] [CrossRef] [Medline]
    2. Costello EJ, Mustillo S, Erkanli A, Keeler G, Angold A. Prevalence and development of psychiatric disorders in childhood and adolescence. Arch Gen Psychiatry 2003 Aug;60(8):837-844. [CrossRef] [Medline]
    3. Lewinsohn P. Major depressive disorder in older adolescents: Prevalence, risk factors, and clinical implications. Clinical Psychology Review 1998 Nov;18(7):765-794 [FREE Full text] [CrossRef]
    4. Merry SN, Hetrick SE, Cox GR, Brudevold-Iversen T, Bir JJ, McDowell H. Psychological and educational interventions for preventing depression in children and adolescents. Cochrane Database Syst Rev 2011 Dec 07(12):CD003380. [CrossRef] [Medline]
    5. Courtney M, Okpych N, Charles P, Mikell D, Stevenson B, Park K, et al. Findings from the California Youth Transitions to Adulthood Study (CalYOUTH): Conditions of Youth at Age. Chicago, IL: Chapin Hall at the University of Chicago; 2016:19-3125.
    6. Havlicek J, Garcia A, Smith DC. Mental Health and Substance Use Disorders among Foster Youth Transitioning to Adulthood: Past Research and Future Directions. Child Youth Serv Rev 2013 Jan;35(1):194-203 [FREE Full text] [CrossRef] [Medline]
    7. Gricar B, Brown L. Conflict, Power, and Organization in a Changing Community. Human Relations 2016 Apr 22;34(10):877-893 [FREE Full text] [CrossRef]
    8. Finnerty M, Neese-Todd S, Bilder S, Olfson M, Crystal S. Best Practices: MEDNET: a multistate policy maker-researcher collaboration to improve prescribing practices. Psychiatr Serv 2014 Nov 01;65(11):1297-1299 [FREE Full text] [CrossRef] [Medline]
    9. Livingood WC, Goldhagen J, Little WL, Gornto J, Hou T. Assessing the status of partnerships between academic institutions and public health agencies. Am J Public Health 2007 Apr;97(4):659-666. [CrossRef] [Medline]
    10. Srihari VH, Breitborde NJK, Pollard J, Tek C, Hyman L, Frisman LK, et al. Public-academic partnerships: early intervention for psychotic disorders in a community mental health center. Psychiatr Serv 2009 Nov;60(11):1426-1428 [FREE Full text] [CrossRef] [Medline]
    11. Stirman SW, Buchhofer R, McLaulin JB, Evans AC, Beck AT. Public-academic partnerships: the Beck Initiative: a partnership to implement cognitive therapy in a community behavioral health system. Psychiatr Serv 2009 Oct;60(10):1302-1304 [FREE Full text] [CrossRef] [Medline]
    12. Hansen E, Seybolt D, Sundeen S. Building a successful public-academic partnership to support state policy making. Psychiatr Serv 2014 Jun 01;65(6):710-712 [FREE Full text] [CrossRef] [Medline]
    13. Jacobs S, Steiner J. The Yale Textbook of Public Psychiatry. New York: Oxford University Press; 2016.
    14. Tseng V. The uses of reaserch in policy and practice. Sharing Child and Youth Development Knowledge 2012;26:1-23.
    15. Nutley S, Walter I, Davies H. Promoting Evidence-based Practice. Research on Social Work Practice 2009 May 27;19(5):552-559 [FREE Full text] [CrossRef]
    16. Palinkas LA, Garcia AR, Aarons GA, Finno-Velasquez M, Holloway IW, Mackie TI, et al. Measuring Use of Research Evidence: The Structured Interview for Evidence Use. Res Soc Work Pract 2016 Sep;26(5):550-564 [FREE Full text] [CrossRef] [Medline]
    17. Farrell C, Coburn C. William T Grant Foundation. 2016. What is the conceptual use of research, and why is it important?   URL: http://wtgrantfoundation.org/conceptual-use-research-important [accessed 2019-05-15] [WebCite Cache]
    18. McBeath B, Jolles MP, Carnochan S, Austin MJ. Organizational and Individual Determinants of Evidence Use by Managers in Public Human Service Organizations. Human Service Organizations: Management, Leadership & Governance 2015 Jun;39(4):267-289. [CrossRef]
    19. Walter I, Nutley S, Davies H. What works to promote evidence-based practice? A cross-sector review. Evidence & Policy: A Journal of Research, Debate and Practice 2005 Sep 01;1(3):335-364 [FREE Full text] [CrossRef]
    20. Palinkas L, Short C, Wong M. William T Grant Foundation. Research-practice-policy partnerships for implementation of evidence-based practices in child welfarechild mental health   URL: http://wtgrantfoundation.org/library/uploads/2015/10/Research-Practice-Policy_Partnerships.pdf [accessed 2019-05-15] [WebCite Cache]
    21. Sysney L. Organizational Physics. 2011. Organizational development stability chart   URL: http://www.organizationalphysics.com/ [accessed 2017-12-07] [WebCite Cache]
    22. Waddock SA. Understanding Social Partnerships. Administration & Society 2016 Jul 26;21(1):78-100 [FREE Full text] [CrossRef]
    23. Jagosh J, Bush PL, Salsberg J, Macaulay AC, Greenhalgh T, Wong G, et al. A realist evaluation of community-based participatory research: partnership synergy, trust building and related ripple effects. BMC Public Health 2015 Jul 30;15:725 [FREE Full text] [CrossRef] [Medline]
    24. Austin J, Whitehead J, Hesselbein F, Whitehead J. Collaboration Challenge: How Nonprofits and Businesses Succeed Through Strategic Alliances. New York: John Wiley & Sons, Incorporated; 2000.
    25. Selsky JW, Parker B. Cross-Sector Partnerships to Address Social Issues: Challenges to Theory and Practice. Journal of Management 2016 Jul;31(6):849-873. [CrossRef]
    26. Warner M, Sullivan R. Putting partnerships to work: Strategic alliances for development between government and private sector and civil society. New York: Routledge; 2004.
    27. Wymer W, Samu S. Dimensions of Business and Nonprofit Collaborative Relationships. Journal of Nonprofit & Public Sector Marketing 2003 Feb 03;11(1):3-22 [FREE Full text] [CrossRef]
    28. Seitanidi MM, Koufopoulos DN, Palmer P. Partnership Formation for Change: Indicators for Transformative Potential in Cross Sector Social Partnerships. J Bus Ethics 2011 Feb 22;94(S1):139-161. [CrossRef]
    29. Quinn R, Cameron K. Organizational Life Cycles and Shifting Criteria of Effectiveness: Some Preliminary Evidence. Management Science 1983 Jan;29(1):33-51 [FREE Full text] [CrossRef]
    30. Kimberly J, Miles R. The organizational life cycle: Issues in the creation, transformation, and decline of organizations. San Francisco, CA: Jossey-Bass Publishers; 1980.
    31. Tushman M, Romanelli E. Organizational evaluation: A metamorphosis model of convergencereorientation. In: Research in organizational behavior. Greenwich, CT: JAI Press; 1985:171-222.
    32. Miller D. Evolution and revolution: A quantum view of structural change in organizations. J Management Studies 1982 Apr;19(2):131-151. [CrossRef]
    33. Miller D, Friesen P. Archetypes of Organizational Transition. Administrative Science Quarterly 1980 Jun;25(2):268-299. [CrossRef]
    34. Whetten DA. Organizational Growth and Decline Processes. Annu Rev Sociol 1987 Aug;13(1):335-358. [CrossRef]
    35. Pawson R. Evidence-based policy: A realist perspective. Thousand Oaks, CA: Sage Publications; 2006.
    36. Pawson R. The science of evaluation: A realist manifesto. London, UK: Sage Publications; 2013.
    37. Pawson R, Tilley N. Realist evaluation. London, UK: Sage Publications; 1997.
    38. Jagosh J, Macaulay A, Pluye P, Salsberg J, Bush PL, Henderson J, et al. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q 2012 Jun;90(2):311-346 [FREE Full text] [CrossRef] [Medline]
    39. Ellen M, Horowitz E, Vaknin S, Lavis JN. Views of health system policymakers on the role of research in health policymaking in Israel. Isr J Health Policy Res 2016;5:24 [FREE Full text] [CrossRef] [Medline]
    40. Baggott C, Cooper B, Marina N, Matthay KK, Miaskowski C. Symptom assessment in pediatric oncology: how should concordance between children's and parents' reports be evaluated? Cancer Nurs 2014;37(4):252-262. [CrossRef] [Medline]

    ‎
    CMO: context-mechanism-outcome
    PAP: public-academic partnership
    SIEU: Structured Interview for Evidence Use

    Edited by G Eysenbach; submitted 13.04.19; peer-reviewed by D Waller, D Reidenberg; comments to author 08.05.19; revised version received 08.05.19; accepted 10.05.19; published 24.05.19

    Copyright

    ©Christina D Kang-Yi. Originally published in JMIR Research Protocols (http://www.researchprotocols.org), 24.05.2019.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on http://www.researchprotocols.org, as well as this copyright and license information must be included.