Protocol
Abstract
Background: Cognitive impairments are a key aspect of schizophrenia spectrum disorders (SSDs), significantly affecting clinical and functional outcomes. The COVID-19 pandemic has heightened concerns about mental health services and cognitive stimulation opportunities. Despite evidence-based interventions like action-based cognitive remediation (ABCR) and metacognitive training (MCT), a research-to-practice gap exists in their application across mental health settings.
Objective: The iCogCA study aims to address this gap by implementing digital ABCR and MCT through a national Canadian collaborative effort using digital psychological interventions to enhance cognitive health in SSDs.
Methods: The study involves 5 Canadian sites, with mental health care practitioners trained digitally through the E-Cog platform, which was developed by our research group. Over 2.5 years, participants with SSDs will undergo pre- and postintervention assessments for clinical symptoms, cognition, and functioning. Each site will run groups annually for both ABCR and MCT, totaling ~390 participants. A nonrandomized concurrent controlled design will assess effectiveness design, in which one intervention (eg, ABCR) acts as the active control for the other (eg, MCT) and vice versa, comparing cognitive and clinical outcomes between the interventions using generalized linear mixed effect modeling. Implementation strategy evaluation will consider the digital platform’s efficacy for mental health care practitioners’ training, contextual factors influencing implementation, and sustainability, using descriptive statistics for quantitative data and thematic analysis for qualitative data.
Results: A pilot pragmatic trial has been conducted previously at the Montreal site, evaluating 3 early implementation outcomes: acceptability, feasibility, and engagement. Patient and therapist acceptability was deemed as high and feasible (21/28, 75% of recruited service users completed therapy, rated feasible by therapists). Technology did not appear to significantly impede program participation. Therapist-rated levels of engagement were also satisfactory. In the ongoing study, recruitment is underway (114 participants recruited as of winter 2024), and intervention groups have been conducted at all sites, with therapists receiving training via the E-Cog learning platform (32 enrolled as of winter 2024).
Conclusions: At least 3 significant innovations will stem from this project. First, this national effort represents a catalyst for the use of digital technologies to increase the adoption of evidence-based interventions and will provide important results on the effectiveness of digitally delivered ABCR and MCT. Second, the results of the implementation component of this study will generate the expertise needed to inform the implementation of similar initiatives. Third, the proposed study will introduce and validate our platform to train and supervise mental health care practitioners to deliver these interventions, which will then be made accessible to the broader mental health community.
Trial Registration: ClinicalTrials.gov NCT05661448; https://clinicaltrials.gov/study/NCT05661448
International Registered Report Identifier (IRRID): DERR1-10.2196/63269
doi:10.2196/63269
Keywords
Introduction
Background
Schizophrenia and related psychoses present debilitating challenges, imposing an enormous burden on individuals, families, and communities [,] and are characterized by symptom recurrence, social deterioration, and cognitive impairments [-]. Most affected individuals experience persistent positive (eg, hallucinations and delusions) and negative symptoms (eg, amotivation, avolition, and reduced expressivity), alongside notable cognitive challenges like difficulties in verbal memory, executive functions, and attention. In addition to these impairments are cognitive distortions [], affecting reasoning and information processing and, collectively, represent a core feature of schizophrenia, impacting clinical and functional recovery [-]. Thus, addressing overall cognitive health in schizophrenia is crucial. Over the past 25 years, advancements in psychological interventions, including cognitive remediation (CR) and metacognitive training (MCT), have shown promise for the treatment of schizophrenia spectrum disorders (SSDs). Both CR and MCT have been found to enhance cognitive functioning and decrease cognitive biases [,].
Cognitive Health Interventions in Schizophrenia: State of the Evidence and Delivery Format
Meta-analyses affirm the effectiveness of CR in enhancing cognition [,,] and MCT in addressing cognitive biases [,]. Notably, various mental health practitioners with an understanding of cognitive processes can be trained to provide these interventions, providing flexibility in service delivery [-]. Delivered in a group format, these interventions enable practitioners to reach multiple service users simultaneously []. Research indicates the feasibility of using technology for remote cognitive assessment and psychological interventions, as individuals with psychosis express interest and willingness to engage with digital mental health services [-] and find this mode of communication more satisfying and less challenging than observed in the general population [].
Preliminary Work on Remote Delivery of Cognitive Health Interventions
Our preliminary work investigated the remote delivery of action-based cognitive remediation (ABCR) and MCT []. ABCR is a distinct form of CR, which integrates the traditional cognitive training techniques of CR with simulated workplace scenarios and goal setting []. This work involved evaluating 3 crucial early implementation outcomes: acceptability, appropriateness, and feasibility for both patients and therapists. Across 6 cohorts (3 ABCR and 3 MCT) conducted within Montreal, patients expressed high acceptability, with overall satisfaction for expectations and perception of progress. The interventions demonstrated feasibility; 36 participants completed therapy, attending an average of 10 MCT sessions and 12 ABCR sessions.
Using recently developed measures [], therapists also reported excellent acceptability, appropriateness, and feasibility of the interventions []. Several facilitators and barriers to delivering these digital interventions were also identified. Barriers comprised patients’ clinical status (eg, more severe symptomatology and medication side effects), interaction issues (eg, lack of involvement and decreased accountability), technological challenges (eg, access to devices and the internet), program elements (eg, language options), and scheduling conflicts. On the other hand, facilitators included patients’ motivation to learn, therapist characteristics (eg, warmth and proper training), financial support for internet connectivity, program support (eg, education on interventions and ice breakers), and logistical adjustments (eg, offering evening sessions and forming smaller groups).
Objectives
Having established the acceptability, appropriateness, and feasibility of our digital interventions that collectively represent the early stages of implementation, the next step is to examine how the proposed digital strategies promote the uptake of these interventions across mental health care settings. A major challenge with evidence-based psychosocial interventions is that very few are subsequently tested in effectiveness or implementation trials and thus have little impact on population health []. The field of implementation science has emerged over the last 20 years, promoting strategies to adopt and integrate evidence-based interventions and change practice patterns within specific settings. Implementation frameworks, such as the Consolidated Framework for Implementation Research (CFIR) [], provide the roadmap and tools to achieve this. To test this implementation strategy, we propose a study spanning 5 different mental health care sites across Canada. In collaboration with partners and knowledge users, we will conduct a hybrid effectiveness-implementation trial and create a bilingual digital learning platform (English and French) to ensure the long-term use of these digital interventions nationwide. The first objective of this study is to investigate the clinical effectiveness of the digital modality of ABCR and MCT. The second objective of the study is to investigate the implementation strategy involving (1) the contextual factors influencing the digital delivery of cognitive health interventions [], (2) the effectiveness of a digital learning platform (E-Cog) to train mental health practitioners, and (3) the sustainability of the maintenance of these interventions within current clinical settings.
Methods
Study Design
A hybrid effectiveness-implementation trial [-] relying on digital technology will be used. The effectiveness component involves assessing the outcomes of these interventions, while the implementation component focuses on conducting the research in a way that emulates the naturalistic clinical setting. This design is ideal for transferring evidence-based behavioral interventions into real care environments [-], as it confirms clinical effectiveness while targeting necessary procedures to deliver and sustain such interventions in real-world care settings. A nonrandomized concurrent controlled design will be used to assess clinical effectiveness where one intervention acts as the active control for the other. The nonrandomized concurrent controlled design was selected, as it eases access to the preferred intervention by service users, overcoming the challenges of randomizing participants in real care settings and facilitating recruitment.
Study Setting
This study takes place across 5 sites providing care to those with psychotic disorders across Canada. These sites include the Centre intégré universitaire de santé et de services sociaux (CIUSSS) de l’Ouest-de l’Ile de Montréal, Royal Ottawa Health Care Group, Kingston Health Sciences Centre, Ontario Shores Centre for Mental Health Sciences in Toronto, and Vancouver Coastal Health. We will recruit 390 service users across the 5 sites.
Selection Criteria
Inclusion criteria include 18 years of age or older; diagnosis of affective or nonaffective psychosis or related disorder; followed and treated by a clinician at one of the services mentioned earlier; considered symptomatically stable and capable of using the digital platforms and participating in intervention groups, as judged by their primary clinicians; access to a private space to ensure group confidentiality; and provision of emergency contact and consent to allow researchers to contact their clinician or emergency services in the event of an emergency during study procedures. Most criteria are present for the safety of the group and participants. Exclusion criteria include intellectual disability, hospitalization at the time of recruitment, inability to speak or read English or French, and high suicide risk as per evaluation. We will recruit 4-6 mental health practitioners per site who will complete training on the E-Cog training platform to deliver the 2 cognitive health interventions. Practitioners will be eligible if they have a background in psychology, social work, nursing, or any other health-related training.
Recruitment
Recruitment for this study uses a multimodal communication strategy designed to effectively reach potential participants. This strategy includes displaying informational flyers on hospital television screens, distributing email newsletters to health care employees, and presenting intervention details to case managers to facilitate referrals. Additionally, recruitment materials will be posted in clinic waiting rooms. n example of the recruitment pamphlet is provided in . Health care team members can refer eligible participants or obtain consent for the research team to make direct contact. Participants are permitted to be engaged in other psychological or psychosocial interventions while participating in our study. Data on whether they are currently receiving other interventions will be collected.
Ethical Considerations
This study has been approved by the respective research ethics boards (REBs) of the 5 mental health sites and their partner institutions. The study will be conducted in Montreal (CIUSSS de l’Ouest-de l’Ile de Montréal; REB 2023-561) [], Ottawa (Royal Ottawa Health Care Group; REB 2022034) [], Kingston (Kingston Health Sciences Center; REB 6037321) [], Toronto (Ontario Shores Centre for Mental Health Sciences; REB 38116) [], and Vancouver (Vancouver Coastal Health; REB H22-02300) []. The research adheres to the principles outlined in the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans and complies with Canadian legal requirements for scientific research involving human participants, including the Privacy Act and applicable clinical trial regulations []. Participants provide informed consent through a comprehensive process that includes a detailed explanation of the study’s objectives, procedures, potential risks and benefits, and their rights as research participants, in accordance with the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans guidelines. This process will involve both written documentation and verbal discussion, and participants will be informed of their right to withdraw from the study at any time without consequence to their ongoing care. Particular attention will be given to ensure an understanding of these processes among individuals with SSDs. All collected data will be deidentified using unique participant codes, with personal identifiers stored separately in encrypted files accessible only to authorized research personnel. In terms of compensation, participants will receive monetary remuneration for each completed assessment measure, with a standardized gift card amount of US $18 per assessment session or focus group. This compensation structure ensures fair and consistent reimbursement for participants’ time and effort while maintaining ethical standards for human participant research. The compensation details will be clearly communicated to participants during the informed consent process to ensure transparency and avoid undue influence on participation decisions.
Measures
To assess the effectiveness of the 2 digital cognitive health interventions (objective 1), primary outcomes will encompass quantitative measures of cognitive function for ABCR and cognitive biases for MCT. These outcomes were selected in alignment with the primary targets for each intervention and primary outcomes assessed in the past ABCR and MCT studies. To capture the full extent of these outcomes, both objective and self-report measures were selected. The collection of these primary outcomes will be prioritized. Secondary outcomes for both interventions will include clinical and functional measures. Propensity score matching will consider background characteristics such as sociodemographic variables, illness duration, medication dosage, length and nature of prior treatment, and subjective reports of cognitive capacity and biases. To promote data quality, research staff will partake in centralized interrater reliability sessions every 6 months. To evaluate the implementation strategy (objective 2), a combination of quantitative and qualitative data will be gathered. Refer to 
Table 1 for a breakdown of the data types to be collected, organized by objectives, time points, and stakeholders.
| Objectives or data type and measures | Measurement time points | Stakeholder groups | |||
| 1. Effectiveness (quantitative) | |||||
Cognitive tests or tasks:
|
| Service users | |||
Intervention:
|
| Service users | |||
| 2. Implementation (quantitative and qualitative) | |||||
Evaluation of E-Cog platform:
|
| Health care practitioners and service users | |||
Evaluation of factors influencing the implementation of the 2 digital interventions:
|
| Health care practitioners and service users | |||
Sustainability:
|
| Health care practitioners and service users | |||
aCANTAB: Cambridge Neuropsychological Test Automated Battery.
bDenotes primary outcomes in the study.
cMMI: MUSIC Model of Academic Motivation Inventory.
dSTQ: Satisfaction Therapy Questionnaire.
eABCR: action-based cognitive remediation.
Cognitive Health Interventions
Interventions will be delivered through a secure videoconference platform. Network-connected tablets will be provided as necessary for the trial duration. ABCR [,] sessions consist of computer-based cognitive training activities (Brain Training Pro; 60%), teaching of problem-solving strategies (20%), and transfer activities (20%). Transfer activities include discussing and role-playing how cognitive skills are applied in everyday life and teaching potential strategies for overcoming cognitive challenges. ABCR targets include processing speed, attention, memory, executive functions, and social cognition, which are all commonly impaired in psychosis []. ABCR will be delivered in 16 sessions lasting 1.5 hours each over an 8-week period. MCT targets cognitive biases and errors in judgment underlying delusions using the theoretical foundations of cognitive behavioral therapy []. Sessions consist of discussions and activities aimed at increasing participants’ awareness of distortions and expanding their current problem-solving strategies. MCT will be delivered in 12 sessions lasting 45-60 minutes each over 6 weeks.
Both interventions will be offered at all sites. Depending on the site, interventions will either be offered concurrently or in sequence. Intervention group allocation follows a personalized, participant-centered approach. Potential participants will meet individually with the research coordinator at their site, who will provide a comprehensive overview of both interventions. Together, they will collaboratively determine which intervention best aligns with the participant’s personal goals and preferences. To maximize participant benefit and engagement, individuals will have the opportunity to complete the alternative intervention after finishing their initial chosen intervention. This flexible approach ensures that participants can potentially benefit from both interventions.
To improve adherence, participants will be contacted prior to each session with reminders. If participants miss several sessions in a row, therapists or a research assistant will contact them to confirm their interest in continuing the intervention and discuss any barriers to their participation. If any adverse reactions occur during the trial, participant involvement in our study may be discontinued after a discussion with the research team. Finally, to improve and track treatment adherence, sessions will be audio recorded, and 2 sessions from each of the ABCR and MCT groups will be randomly selected for review by 2 experienced facilitators using the treatment integrity assessment tool.
Training Platform
E-Cog training platform [] is a digital platform providing training certifications for ABCR and MCT interventions developed by our group following the analyze, design, develop, implement, and evaluate model for the design of digital learning platforms []. Each training certification includes three training modules: (1) impact of cognitive impairments in psychosis and an introduction to remediation strategies (2 hours), (2) technological tools for digital mental health (1 hour), and (3) theoretical foundations and practical delivery of ABCR (~9 hours) or MCT (~12 hours). After obtaining the training certification, mental health practitioners participate in weekly supervision by a dedicated experienced trainer using a secure videoconferencing software. Screenshots from the training platform can be found in the multimedia appendices (-).
Involvement of Individuals With Lived Experience
Peer support workers have been involved at different stages of the project. The pilot study, which informed this study, involved 2 peer support workers in the project committee. In this study, our peer support worker has been providing continuous consultation, which has led to changes throughout the study (eg, adapting intervention materials). Our peer support worker will also be involved in the intervention groups and will be coconducting the service user qualitative interviews.
Statistical Analysis
Power
Monte Carlo simulations computed in R (R Foundation for Statistical Computing) were used to estimate the required sample size for our proposed models. Our analyses, based on simulated data, suggest that a total sample size of 300 provides enough statistical power (up to 90%) to detect anticipated effect sizes on primary outcomes of cognitive capacity and cognitive bias based on values from our group and those reported in the literature (CR: d=0.50 and MCT: g=0.27) [,,,]. The attrition rate for our digital groups has been approximately 20%; we will nonetheless conservatively adjust for an attrition rate of 30%. When considering the propensity score, this results in a requirement of 390 participants. Comparable studies using the same interventions and statistical methods in in-person settings have included similar sample sizes as those proposed at our individual sites [,]. Further, this sample size will allow us to detect anticipated effect sizes on secondary outcomes related to symptomatology (CR: d=0.28 and MCT: g=0.38) [,] and functioning (CR: d=0.36-0.51 and MCT: d=0.37) [,,]. We also anticipate that the proposed sample size will be adequate to explore sex- and gender-related differences using subgroup analyses of 2 (male and female) and 4 (men, women, nonbinary, and other) groups, respectively.
Data Analysis
Data related to the analysis of objective 1, focusing on clinical effectiveness, will be compared between both interventions. Primary and secondary outcomes will be compared between the 2 interventions, using one as the active control for the other. First, propensity score matching will be used to identify a subset of participants who will comprise the active control group that is equivalent to the intervention group on background variables (sociodemographic variables, illness duration, medication dosage, length and nature of prior treatment as well as subjective reports of cognitive capacity and cognitive biases). Then, Z-standardized outcome data will be compared between the groups with generalized linear mixed effect modeling techniques using R software. Factors of interest will include fixed time (pre, post, and follow-up), fixed treatment (ABCR and MCT), fixed time and treatment interaction (time*treatment), random site (CIUSSS de l’Ouest-de l’Ile de Montréal, the Royal’s Institute in Ottawa, Kingston Health Sciences Centre, Ontario Shores Centre for Mental Health Sciences in Toronto, and Vancouver Coastal Health), random intercepts (participants’ ID), and random slopes (participant*time). Age, illness duration, medication dosage, length, and nature of prior psychological treatment will also be included as fixed covariates. This procedure will be done twice, with the active control group subset using propensity score matching. This approach ensures that participants are not counted twice and is an integral part of the propensity score matching procedure. Statistically, performing the procedure twice helps to validate the robustness of the matching process without introducing bias, as each participant is only included once in each comparison. Specifically, the procedure will be as follows: (1) ABCR as the intervention and MCT as the active control and (2) MCT as the intervention and ABCR as the active control. Missing data pattern will be assessed for whether it is missing completely at random, missing at random, or not at random []. If data are missing at random, multiple imputation will be applied []; if data are missing not at random, then the robustness of the primary analyses will be evaluated through sensitivity analyses.
Analyses for all implementation objectives will be executed using descriptive statistics for quantitative data and thematic analysis for qualitative data, following Braun and Clarke’s [] interpretive descriptive approach. This method emphasizes the researcher’s active role in the analytical process, allowing for a nuanced understanding of themes such as facilitators and barriers to implementation, identified from CFIR semistructured interviews. Investigator triangulation will be used, with 3 independent raters conducting iterative qualitative analyses until a consensus (≥80%) is reached. A convergent quantitative-qualitative mixed methods design [] will facilitate a comprehensive comparison of results from quantitative and qualitative analyses, aiming to develop a holistic understanding of factors influencing implementation. Side-by-side comparison tables will support this analytical process and engage implementation teams in interpretation. To ensure reflexivity, member checks will be conducted with participants to validate findings, ensuring that their perspectives and interpretations are accurately represented and that any potential biases in the researcher’s analysis are addressed. This methodological framework is designed to uphold rigorous standards in qualitative research while ensuring that the findings are credible, dependable, and relevant to broader contexts. All qualitative analyses will be done using NVivo (version 15; Lumivero).
Results
A pilot pragmatic trial [] has been conducted previously at the Montreal site, where 6 cohorts (3 ABCR and 3 MCT) were run, evaluating 3 early implementation outcomes: acceptability, feasibility, and engagement. Of the 28 participants attending at least 1 session, 21 completed more than half of the sessions. All completers reported a positive experience with therapy, 2 of 3 were not bothered by the remote setting, and 16 trusted the confidentiality of the information shared. Technology did not appear to significantly impede program participation. Therapist-rated levels of engagement were also satisfactory [].
This study was approved by the institutional review board by September 2023. From September 2022 to January 2023 staff training was conducted across all 5 sites. Intervention materials including intervention manuals and digital portals began in June 2022 and continued until November 2023. Therapist training began on the E-Cog learning platform in April 2023 and is ongoing. All sites began recruitment in 2023 and have run cohorts of MCT and ABCR in their respective regions. Quantitative data collection occurs prior to and following the intervention group and is ongoing, qualitative data collection for implementation assessment began in summer 2024 and is ongoing. As of winter 2024, a total of 114 participants have been recruited and 32 therapists are enrolled in the E-Cog training platform with 25 therapists having completed the training. The updated project timeline can be accessed for those who create free accounts on the iCogCA Hub web page [].
Discussion
Overview
This study aims to address the crucial clinical needs of individuals with SSD by evaluating the feasibility and effectiveness of delivering cognitive health interventions through a digital platform. We hypothesize that these interventions will yield positive outcomes in terms of accessibility, engagement, symptom reduction, and cognitive functioning.
Principal Findings and Comparison to Prior Work
The objectives of this study are 2-fold, focusing on both service users and mental health care providers. The first objective aims to assess the effectiveness of digital cognitive health interventions in enhancing accessibility and engagement for individuals with SSD. If proven effective, these interventions may provide an evidence-based framework for other service providers providing care for individuals with psychotic disorders and could be adapted for other populations receiving mental health services where these interventions show similar efficacy (eg, mood disorders) [,]. Objective 2 aims to identify facilitators and barriers to digital delivery and a digital training platform for clinicians. We anticipate that this study will introduce and validate our platform to train and supervise mental health care practitioners to deliver these interventions, which will then be made accessible to the broader mental health community. Further, these findings will furnish crucial information for the implementation of digital health technologies to improve access to psychosocial therapies for optimized care. These findings will be used to guide the implementation of similar projects across Canada. Notably, New York State has already fully integrated CR into its psychiatric care systems [], and parallel efforts are underway in Australia [] and the United Kingdom []. Collaborating with a National Steering and Implementation Committee, we will test our set of tools based on the CFIR and make them accessible to the mental health community. Site-specific implementation committees, which include patient partners, will assist with the local implementation of our cognitive health interventions.
Strengths and Limitations
This proposed national collaborative effort serves as a catalyst for leveraging digital technology to enhance the adoption of evidence-based psychosocial interventions. This study has several strengths. The first is the implementation across multiple Canadian sites, which enhances the generalizability of our findings. Second, we also aim to engage a broad pool of participants, both service users and mental health practitioners, by using digital technology to provide the intervention and training. Encouragingly, previous studies have demonstrated that a significant portion of those with severe mental illnesses appear to have the required technological access and ability to join, participate, and benefit from digital services [,]. We have also a planned comprehensive collection of quantitative and qualitative data. Our data collection plan involves service users and mental health practitioners at multiple time points, allowing us to answer our research questions in a multifaceted manner. Finally, we have engaged with diverse stakeholders throughout various stages of the study, including those with lived experiences, clinicians, researchers, and managers.
Some limitations of our study include the nonrandomized concurrent controlled design, which raises concerns about potential group differences. However, we will address this by statistically controlling for measurable background characteristics. Second, although we aim to engage a broader population by providing services using a digital format, we acknowledge that there could be variability to technology access. We have planned to provide tablets and internet to service users without access to technology. For service users or health care providers unfamiliar with technology, we will also offer orientation sessions to help them become comfortable with the digital format. Next, the recruitment of therapists within a research setting may not reflect therapists involved in nonresearch settings. In addition, although we have taken great care in selecting instruments that have been validated in our population and available in both English and French, some limitations may still exist with the instruments chosen. For instance, ceiling effects have been observed in the Cambridge Neuropsychological Test Automated Battery cognitive test. Finally, there may exist differences across 5 sites in training experiences and implementation fidelity. We plan to attend to this by standardizing training and fidelity checks conducted across all sites.
Conclusions
In summary, this project aims to address pressing clinical needs related to cognitive impairments and biases in SSDs. By leveraging digital technologies, it aims to facilitate the adoption of evidence-based interventions within clinical settings, support future implementation of similar initiatives, and validate an adaptable platform designed for widespread use.
Acknowledgments
The authors would like to thank the study staff, community and clinical partners, and study participants who support this work. SG and GS are supported by a Chercheur Boursier Junior 1 from the Fonds de Recherche du Québec en Santé. ML is supported by a James McGill Professorship from McGill University. ET receives postdoctoral fellowship funding from the Canadian Institutes of Health Research (#171198). The authors thank the following people for their support with this project: Shalini Lal, Michael Bodnar, Deji Ayonrinde, Alexandra Baines, Ridha Joober, Tania Lecomte, Jai Shah, Nicola Wright, David Attwood, Patricia Debergue, Alonso Montoya, Tim Pauley, and Phil Tibbo. This research was undertaken thanks in part to funding from the Canada First Research Excellence Fund awarded through the Healthy Brains, Healthy Lives initiative at McGill University and by a Canadian Institutes of Health Research Project (grant #180501).
Data Availability
The datasets generated or analyzed during this study are not publicly available due to ongoing data collection but are available from the corresponding author on reasonable request.
Authors' Contributions
CA-Y and HT were responsible for writing the original draft and conducting investigations. They will also handle future formal analysis. MB, SG, KML, ET, and GS developed the conceptual framework and methodologies and participated in writing and reviewing the manuscript. AES contributed to the visualization and also engaged in manuscript review and editing. CRB, MM, SM, MP, and TSW contributed to conceptualization and methodology, provided resources, and reviewed and edited the manuscript. ML played a multifaceted role including conceptualization, methodology development, drafting and reviewing the manuscript, as well as overseeing the project administration and securing funding. DR-C was involved in conceptualization, methodology development, writing the original draft, and reviewing the manuscript, in addition to supervising the entire project. All authors reviewed the final manuscript.
Conflicts of Interest
This project is carried out in collaboration with the company SBT, which provides licenses for the Happy Neuron platform. ML reports grants from Roche Canada, grants from Otsuka Lundbeck Alliance, and personal fees from Boehringer Ingelheim, Janssen, Lundbeck Canada, and Otsuka Canada outside the submitted work. SG has received financial compensation for consulting services from Boehringer Ingelheim outside the submitted work. MB has received consulting fees from Boehringer Ingelheim. KML reports consulting fees from Otsuka Canada, Lundbeck Canada, and Boehringer Ingelheim. SM has received consultant fees from Boehringer and ROVI and is developer of metacognitive training. DR-C reports consultations fees from Otsuka Canada Pharmaceutical, outside the submitted work. All other authors declare no conflicts of interest.
Multimedia Appendix 3
E-Cog training platform action-based cognitive remediation introduction page.
PNG File , 821 KBMultimedia Appendix 4
E-Cog training platform metacognitive training introduction page.
PNG File , 539 KBReferences
- Chong HY, Teoh SL, Wu DB, Kotirum S, Chiou CF, Chaiyakunapruk N. Global economic burden of schizophrenia: a systematic review. Neuropsychiatr Dis Treat. 2016;12:357-373. [FREE Full text] [CrossRef] [Medline]
- Goeree R, Farahati F, Burke N, Blackhouse G, O'Reilly D, Pyne J, et al. The economic burden of schizophrenia in Canada in 2004. Curr Med Res Opin. 2005;21(12):2017-2028. [CrossRef] [Medline]
- Carpenter WT, Kirkpatrick B. The heterogeneity of the long-term course of schizophrenia. Schizophr Bull. 1988;14(4):645-652. [CrossRef] [Medline]
- Carpenter WT, Strauss JS. The prediction of outcome in schizophrenia. IV: Eleven-year follow-up of the Washington IPSS cohort. J Nerv Ment Dis. 1991;179(9):517-525. [CrossRef] [Medline]
- Ciompi L. The natural history of schizophrenia in the long term. Br J Psychiatry. 1980;136:413-420. [CrossRef] [Medline]
- Ciompi L. Learning from outcome studies. Toward a comprehensive biological-psychosocial understanding of schizophrenia. Schizophr Res. 1988;1(6):373-384. [CrossRef] [Medline]
- Harding CM, Brooks GW, Ashikaga T, Strauss JS, Breier A. The Vermont longitudinal study of persons with severe mental illness, II: Long-term outcome of subjects who retrospectively met DSM-III criteria for schizophrenia. Am J Psychiatry. 1987;144(6):727-735. [CrossRef] [Medline]
- Lally J, Ajnakina O, Stubbs B, Cullinane M, Murphy KC, Gaughran F, et al. Remission and recovery from first-episode psychosis in adults: systematic review and meta-analysis of long-term outcome studies. Br J Psychiatry. 2017;211(6):350-358. [CrossRef] [Medline]
- Sauvé G, Lavigne KM, Pochiet G, Brodeur MB, Lepage M. Efficacy of psychological interventions targeting cognitive biases in schizophrenia: a systematic review and meta-analysis. Clin Psychol Rev. 2020;78:101854. [CrossRef] [Medline]
- Bora E, Pantelis C. Meta-analysis of cognitive impairment in first-episode bipolar disorder: comparison with first-episode schizophrenia and healthy controls. Schizophr Bull. 2015;41(5):1095-1104. [FREE Full text] [CrossRef] [Medline]
- Buck BE, Pinkham AE, Harvey PD, Penn DL. Revisiting the validity of measures of social cognitive bias in schizophrenia: additional results from the Social Cognition Psychometric Evaluation (SCOPE) study. Br J Clin Psychol. 2016;55(4):441-454. [FREE Full text] [CrossRef] [Medline]
- McCleery A, Nuechterlein KH. Cognitive impairment in psychotic illness: prevalence, profile of impairment, developmental course, and treatment considerations. Dialogues Clin Neurosci. 2019;21(3):239-248. [FREE Full text] [CrossRef] [Medline]
- Sauvé G, Malla A, Joober R, Brodeur MB, Lepage M. Comparing cognitive clusters across first- and multiple-episode of psychosis. Psychiatry Res. 2018;269:707-718. [CrossRef] [Medline]
- Vita A, Barlati S, Ceraso A, Nibbio G, Ariu C, Deste G, et al. Effectiveness, core elements, and moderators of response of cognitive remediation for schizophrenia: a systematic review and meta-analysis of randomized clinical trials. JAMA Psychiatry. 2021;78(8):848-858. [FREE Full text] [CrossRef] [Medline]
- Penney D, Sauvé G, Mendelson D, Thibaudeau É, Moritz S, Lepage M. Immediate and sustained outcomes and moderators associated with metacognitive training for psychosis: a systematic review and meta-analysis. JAMA Psychiatry. 2022;79(5):417-429. [FREE Full text] [CrossRef] [Medline]
- Vita A, Barlati S, Ceraso A, Nibbio G, Durante F, Facchi M, et al. Durability of effects of cognitive remediation on cognition and psychosocial functioning in schizophrenia: a systematic review and meta-analysis of randomized clinical trials. Am J Psychiatry. 2024;181(6):520-531. [CrossRef] [Medline]
- Lejeune JA, Northrop A, Kurtz MM. A meta-analysis of cognitive remediation for schizophrenia: efficacy and the role of participant and treatment factors. Schizophr Bull. 2021;47(4):997-1006. [FREE Full text] [CrossRef] [Medline]
- Hotte-Meunier A, Penney D, Mendelson D, Thibaudeau É, Moritz S, Lepage M, et al. Effects of metacognitive training (MCT) on social cognition for schizophrenia spectrum and related psychotic disorders: a systematic review and meta-analysis. Psychol Med. 2024;54(5):914-920. [CrossRef] [Medline]
- Moritz S, Veckenstedt R, Randjbar S, Vitzthum F, Woodward TS. Antipsychotic treatment beyond antipsychotics: metacognitive intervention for schizophrenia patients improves delusional symptoms. Psychol Med. 2011;41(9):1823-1832. [CrossRef] [Medline]
- Bowie CR, Bell MD, Fiszdon JM, Johannesen JK, Lindenmayer J, McGurk SR, et al. Cognitive remediation for schizophrenia: an expert working group white paper on core techniques. Schizophr Res. 2020;215:49-53. [CrossRef] [Medline]
- Dark F, Newman E, Harris M, Cairns A, Simpson M, Gore-Jones V, et al. Implementing cognitive remediation therapy (CRT) in a mental health service: staff training. Australas Psychiatry. 2016;24(2):185-189. [CrossRef] [Medline]
- Alvarez-Jimenez M, Alcazar-Corcoles MA, González-Blanch C, Bendall S, McGorry PD, Gleeson JF. Online, social media and mobile technologies for psychosis treatment: a systematic review on novel user-led interventions. Schizophr Res. 2014;156(1):96-106. [CrossRef] [Medline]
- Lal S, Dell'Elce J, Tucci N, Fuhrer R, Tamblyn R, Malla A. Preferences of young adults with first-episode psychosis for receiving specialized mental health services using technology: a survey study. JMIR Ment Health. 2015;2(2):e18. [FREE Full text] [CrossRef] [Medline]
- Medalia A, Saperstein AM, Stefancic A, Meyler S, Styke S, Qian M, et al. Feasibility and acceptability of remotely accessed cognitive remediation for schizophrenia in public health settings. Psychiatry Res. 2021;301:113956. [FREE Full text] [CrossRef] [Medline]
- Grossman MJ, Woolridge S, Lichtenstein S, McLaughlin D, Auther AM, Carrión RE, et al. Patterns and perceptions of face-to-face and digital communication in the clinical high risk and early stages of psychosis. Psychiatry Res. 2020;284:112667. [CrossRef] [Medline]
- Mendelson D, Thibaudeau É, Sauvé G, Lavigne KM, Bowie CR, Menon M, et al. Remote group therapies for cognitive health in schizophrenia-spectrum disorders: feasible, acceptable, engaging. Schizophr Res Cogn. 2022;28:100230. [FREE Full text] [CrossRef] [Medline]
- Bowie CR, Grossman M, Gupta M, Holshausen K, Best MW. Action-based cognitive remediation for individuals with serious mental illnesses: effects of real-world simulations and goal setting on functional and vocational outcomes. Psychiatr Rehabil J. 2017;40(1):53-60. [CrossRef] [Medline]
- Weiner BJ, Lewis CC, Stanick C, Powell BJ, Dorsey CN, Clary AS, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12(1):108. [FREE Full text] [CrossRef] [Medline]
- Bauer MS, Damschroder L, Hagedorn H, Smith J, Kilbourne AM. An introduction to implementation science for the non-specialist. BMC Psychol. 2015;3(1):32. [FREE Full text] [CrossRef] [Medline]
- Damschroder LJ, Reardon CM, Widerquist MAO, Lowery J. The updated Consolidated Framework for Implementation Research based on user feedback. Implement Sci. 2022;17(1):75. [FREE Full text] [CrossRef] [Medline]
- Wykes T, Huddy V, Cellard C, McGurk SR, Czobor P. A meta-analysis of cognitive remediation for schizophrenia: methodology and effect sizes. Am J Psychiatry. 2011;168(5):472-485. [CrossRef] [Medline]
- Wolfenden L, Williams CM, Wiggers J, Nathan N, Yoong SL. Improving the translation of health promotion interventions using effectiveness-implementation hybrid designs in program evaluations. Health Promot J Austr. 2016;27(3):204-207. [CrossRef] [Medline]
- Landes SJ, McBain SA, Curran GM. Reprint of: An introduction to effectiveness-implementation hybrid designs. Psychiatry Res. 2020;283:112630. [FREE Full text] [CrossRef] [Medline]
- Curran GM, Bauer M, Mittman B, Pyne JM, Stetler C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50(3):217-226. [FREE Full text] [CrossRef] [Medline]
- Luszczynska A. It's time for effectiveness-implementation hybrid research on behaviour change. Health Psychol Rev. 2020;14(1):188-192. [CrossRef] [Medline]
- Research Ethics. Quebec Montreal West Island Integrated University Health and Social Services Centre. URL: https://www.ciusss-ouestmtl.gouv.qc.ca/en/n-research-teaching-and-innovation/research/research-ethics [accessed 2025-03-19]
- Office of Research Ethics. The Royal Ottawa Health Care Group. URL: https://www.theroyal.ca/research/about-research/research-ethics-board [accessed 2025-03-19]
- Health Sciences and Affiliated Teaching Hospitals Research Ethics Board (HSREB). Queen's University. URL: https://www.queensu.ca/vpr/ethics/hsreb [accessed 2025-03-19]
- Joint Research Ethics Board (JREB). Ontario Shores Centre for Mental Health Sciences. URL: https://www.ontarioshores.ca/research/reb [accessed 2025-03-19]
- Office of Research Ethics. The University of British Columbia. URL: https://researchethics.ubc.ca/ [accessed 2025-03-19]
- Medical Research Council (Canada), Natural Sciences and Engineering Research Council of Canada, Social Sciences and Humananities Research Council of Canada. Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans—TCPS 2 (2022). 2022. URL: https://ethics.gc.ca/eng/policy-politique_tcps2-eptc2_2022.html [accessed 2025-03-27]
- Cognitive assessments. Cambridge Cognition Ltd. URL: https://cambridgecognition.com/ [accessed 2025-03-19]
- Wechsler D. Wechsler Memory Scale. Washington, DC. Psychological Corporation; 1945.
- Woodward TS, Moritz S, Cuttler C, Whitman JC. The contribution of a cognitive bias against disconfirmatory evidence (BADE) to delusions in schizophrenia. J Clin Exp Neuropsychol. 2006;28(4):605-617. [CrossRef] [Medline]
- Phillips LD, Edwards W. Conservatism in a simple probability inference task. J Exp Psychol. 1966;72(3):346-354. [CrossRef] [Medline]
- Stip E, Caron J, Renaud S, Pampoulova T, Lecomte Y. Exploring cognitive complaints in schizophrenia: the subjective scale to investigate cognition in schizophrenia. Compr Psychiatry. 2003;44(4):331-340. [CrossRef] [Medline]
- Tran T, Hillman JG, Hargadon DP, Cunningham S, Toubache R, Bowie CR. Approach and withdrawal from cognitively effortful activities: development, validation, and transdiagnostic clinical utility of a cognitive motivation scale. J Affect Disord. 2024;367:823-831. [CrossRef] [Medline]
- van der Gaag M, Schütz C, Ten Napel A, Landa Y, Delespaul P, Bak M, et al. Development of the Davos Assessment of Cognitive Biases Scale (DACOBS). Schizophr Res. 2013;144(1-3):63-71. [CrossRef] [Medline]
- Østergaard SD, Lemming OM, Mors O, Correll CU, Bech P. PANSS-6: a brief rating scale for the measurement of severity in schizophrenia. Acta Psychiatr Scand. 2016;133(6):436-444. [CrossRef] [Medline]
- Kirkpatrick B, Strauss GP, Nguyen L, Fischer BA, Daniel DG, Cienfuegos A, et al. The brief negative symptom scale: psychometric properties. Schizophr Bull. 2011;37(2):300-305. [FREE Full text] [CrossRef] [Medline]
- Haddock G, McCarron J, Tarrier N, Faragher EB. Scales to measure dimensions of hallucinations and delusions: the Psychotic Symptom Rating Scales (PSYRATS). Psychol Med. 1999;29(4):879-889. [CrossRef] [Medline]
- Lecomte T, Corbière M, Laisné F. Self-Esteem Rating Scale—Short Form. Psychiatry Res. 2006:99-108. [CrossRef]
- Russell D, Cutrona CE, Rose J, Yurko K. Social and emotional loneliness: an examination of Weiss's typology of loneliness. J Pers Soc Psychol. 1984;46(6):1313-1321. [CrossRef] [Medline]
- Vaingankar JA, Abdin E, Chong SA, Sambasivam R, Seow E, Jeyagurunathan A, et al. Psychometric properties of the Short Warwick Edinburgh Mental Well-Being Scale (SWEMWBS) in service users with schizophrenia, depression and anxiety spectrum disorders. Health Qual Life Outcomes. 2017;15(1):153. [FREE Full text] [CrossRef] [Medline]
- Williams J, Leamy M, Pesola F, Bird V, Le Boutillier C, Slade M. Psychometric evaluation of the Questionnaire about the Process of Recovery (QPR). Br J Psychiatry. 2015;207(6):551-555. [CrossRef] [Medline]
- Chen B, Vansteenkiste M, Beyers W, Boone L, Deci EL, Van der Kaap-Deeder J, et al. Basic psychological need satisfaction, need frustration, and need strength across four cultures. Motiv Emot. 2014;39(2):216-236. [CrossRef]
- Zuroff DC, Koestner R, Moskowitz DS, McBride C, Marshall M, Bagby MR. Autonomous motivation for therapy: a new common factor in brief treatments for depression. Psychother Res. 2007;17(2):137-147. [CrossRef]
- Nasrallah H, Morosini P, Gagnon DD. Reliability, validity and ability to detect change of the Personal and Social Performance scale in patients with stable schizophrenia. Psychiatry Res. 2008;161(2):213-224. [CrossRef] [Medline]
- Czajkowska Z, Wang H, Hall NC, Sewitch M, Körner A. Validation of the English and French versions of the brief Health Care Climate Questionnaire. Health Psychol Open. 2017;4(2):2055102917730675. [FREE Full text] [CrossRef] [Medline]
- Hansen MC, Jones BD, Eack SM, Glenthøj LB, Ikezawa S, Iwane T, et al. Validation of the MUSIC Model of Motivation Inventory for use with cognitive training for schizophrenia spectrum disorders: a multinational study. Schizophr Res. 2019;206:142-148. [FREE Full text] [CrossRef] [Medline]
- Beck AT, Wright FD, Newman CF, Liese BS, Stein DJ. Cognitive Therapy of Substance Abuse. New York, NY. The Guilford Press; 1993.
- Clough BA, Rowland DP, Casey LM. Development of the eTAP-T: a measure of mental health professionals' attitudes and process towards e-interventions. Internet Interv. 2019;18:100288. [FREE Full text] [CrossRef] [Medline]
- Luke DA, Calhoun A, Robichaux CB, Elliott MB, Moreland-Russell S. The program sustainability assessment tool: a new instrument for public health programs. Prev Chronic Dis. 2014;11:130184. [FREE Full text] [CrossRef] [Medline]
- Bowie CR, Grossman M, Gupta M, Oyewumi LK, Harvey PD. Cognitive remediation in schizophrenia: efficacy and effectiveness in patients with early versus long-term course of illness. Early Interv Psychiatry. 2014;8(1):32-38. [CrossRef] [Medline]
- Moritz S, Woodward TS. Metacognitive training in schizophrenia: from basic research to knowledge translation and intervention. Curr Opin Psychiatry. 2007;20(6):619-625. [CrossRef] [Medline]
- E-Cog. URL: https://e-cog.ca/ [accessed 2024-11-15]
- Gavarkovs AG, Blunt W, Petrella RJ. A protocol for designing online training to support the implementation of community-based interventions. Eval Program Plann. 2019;72:77-87. [CrossRef] [Medline]
- Farreny A, Aguado J, Ochoa S, Huerta-Ramos E, Marsà F, López-Carrilero R, et al. REPYFLEC cognitive remediation group training in schizophrenia: looking for an integrative approach. Schizophr Res. 2012;142(1-3):137-144. [CrossRef] [Medline]
- Köther U, Vettorazzi E, Veckenstedt R, Hottenrott B, Bohn F, Scheu F, et al. Bayesian analyses of the effect of metacognitive training on social cognition deficits and overconfidence in errors. J Exp Psychopathol. 2017;8(2):158-174. [CrossRef]
- Liu Y, Tang C, Hung T, Tsai P, Lin M. The efficacy of metacognitive training for delusions in patients with schizophrenia: a meta-analysis of randomized controlled trials informs evidence-based practice. Worldviews Evid Based Nurs. 2018;15(2):130-139. [CrossRef] [Medline]
- Bryce S, Sloan E, Lee S, Ponsford J, Rossell S. Cognitive remediation in schizophrenia: a methodological appraisal of systematic reviews and meta-analyses. J Psychiatr Res. 2016;75:91-106. [CrossRef] [Medline]
- Briki M, Monnin J, Haffen E, Sechter D, Favrod J, Netillard C, et al. Metacognitive training for schizophrenia: a multicentre randomised controlled trial. Schizophr Res. 2014;157(1-3):99-106. [CrossRef] [Medline]
- Fernández-García MP, Vallejo-Seco G, Livácic-Rojas P, Tuero-Herrero E. The (ir)responsibility of (under)estimating missing data. Front Psychol. 2018;9:556. [FREE Full text] [CrossRef] [Medline]
- Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG, et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls. BMJ. 2009;338:b2393. [FREE Full text] [CrossRef] [Medline]
- Braun V, Clarke V. Toward good practice in thematic analysis: avoiding common problems and be(com)ing a researcher. Int J Transgend Health. 2023;24(1):1-6. [FREE Full text] [CrossRef] [Medline]
- Venkatesh V, Brown S, Sullivan Y. Guidelines for conducting mixed-methods research: an extension and illustration. J Assoc Inf Syst. 2016;17(7):435-494. [CrossRef]
- iCogCA Communications Hub! URL: https://www.icogca.com/post/welcome [accessed 2024-11-15]
- Bowie CR, Gupta M, Holshausen K, Jokic R, Best M, Milev R. Cognitive remediation for treatment-resistant depression: effects on cognition and functioning and the role of online homework. J Nerv Ment Dis. 2013;201(8):680-685. [CrossRef] [Medline]
- Porter RJ, Bowie CR, Jordan J, Malhi GS. Cognitive remediation as a treatment for major depression: a rationale, review of evidence and recommendations for future research. Aust N Z J Psychiatry. 2013;47(12):1165-1175. [CrossRef] [Medline]
- Medalia A, Saperstein AM, Erlich MD, Sederer LI. Cognitive remediation in large systems of psychiatric care. CNS Spectr. 2019;24(1):163-173. [CrossRef] [Medline]
- Dark F, Harris M, Gore-Jones V, Newman E, Whiteford H. Implementing cognitive remediation and social cognitive interaction training into standard psychosis care. BMC Health Serv Res. 2018;18(1):458. [FREE Full text] [CrossRef] [Medline]
- Wykes T, Joyce E, Velikonja T, Watson A, Aarons G, Birchwood M, et al. The CIRCuiTS study (Implementation of cognitive remediation in early intervention services): protocol for a randomised controlled trial. Trials. 2018;19(1):183. [FREE Full text] [CrossRef] [Medline]
- Young AS, Cohen AN, Niv N, Nowlin-Finch N, Oberman RS, Olmos-Ochoa TT, et al. Mobile phone and smartphone use by people with serious mental illness. Psychiatr Serv. 2020;71(3):280-283. [FREE Full text] [CrossRef] [Medline]
- Miu AS, Vo HT, Palka JM, Glowacki CR, Robinson RJ. Teletherapy with serious mental illness populations during COVID-19: telehealth conversion and engagement. Couns Psychol Q. 2020;34(3-4):704-721. [CrossRef]
Abbreviations
| ABCR: action-based cognitive remediation |
| CFIR: Consolidated Framework for Implementation Research |
| CIUSSS: Centre intégré universitaire de santé et de services sociaux |
| CR: cognitive remediation |
| MCT: metacognitive training |
| REB: research ethics board |
| SSD: schizophrenia spectrum disorder |
Edited by A Schwartz, T Leung; submitted 14.06.24; peer-reviewed by T Wykes, P Harvey; comments to author 11.10.24; revised version received 02.12.24; accepted 26.02.25; published 15.04.25.
Copyright©Christy Au-Yeung, Helen Thai, Michael Best, Christopher R Bowie, Synthia Guimond, Katie M Lavigne, Mahesh Menon, Steffen Moritz, Myra Piat, Geneviève Sauvé, Ana Elisa Sousa, Elisabeth Thibaudeau, Todd S Woodward, Martin Lepage, Delphine Raucher-Chéné. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 15.04.2025.
This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.