This paper is in the following e-collection/theme issue:

RCTs - Protocols/Proposals (eHealth) (506) Games for Cognitive Assessment (56) Games and Gamification for Health (604) Memory (42)

Published on in Vol 12 (2023)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/47496, first published .
Working Memory Training for Children Using the Adaptive, Self-Select, and Stepwise Approaches to Setting the Difficulty Level of Training Activities: Protocol for a Randomized Controlled Trial

Working Memory Training for Children Using the Adaptive, Self-Select, and Stepwise Approaches to Setting the Difficulty Level of Training Activities: Protocol for a Randomized Controlled Trial

Working Memory Training for Children Using the Adaptive, Self-Select, and Stepwise Approaches to Setting the Difficulty Level of Training Activities: Protocol for a Randomized Controlled Trial

Authors of this article:

Regine Cassandra Lau1 Author Orcid Image ;   Peter John Anderson1, 2 Author Orcid Image ;   Joshua F Wiley1, 3 Author Orcid Image ;   Derek Huang4 Author Orcid Image ;   Faisha Surjatin4 Author Orcid Image ;   Paul McIntosh4 Author Orcid Image ;   Susan Gathercole5 Author Orcid Image ;   Megan Spencer-Smith1 Author Orcid Image
Article Authors Cited by Tweetations (4) Metrics

    Protocol

    1School of Psychological Sciences and Turner Institute for Brain and Mental Health, Monash University, Clayton, Australia

    2Victorian Infant Brain Studies (VIBeS), Clinical Sciences, Murdoch Children’s Research Institute, Melbourne, Australia

    3Peter MacCallum Cancer Centre, Melbourne, Australia

    4Virtual and Augmented Reality Services (VARS), eSolutions, Monash University, Clayton, Australia

    5MRC Cognition and Brain Sciences Unit, University of Cambridge, Cambridge, United Kingdom

    Corresponding Author:

    Megan Spencer-Smith, PhD

    School of Psychological Sciences and Turner Institute for Brain and Mental Health

    Monash University

    18 Innovation Walk

    Clayton, 3166

    Australia

    Phone: 61 399059148

    Email: megan.spencer-smith@monash.edu


    Background: A common yet untested assumption of cognitive training in children is that activities should be adaptive, with difficulty adjusted to the individual’s performance in order to maximize improvements on untrained tasks (known as transfer). Working memory training provides the ideal testbed to systematically examine this assumption as it is one of the most widely studied domains in the cognitive training literature, and is critical for children’s learning, including following instructions and reasoning.

    Objective: This trial aimed to examine children’s outcomes of working memory training using adaptive, self-select (child selects difficulty level), and stepwise (difficulty level increases incrementally) approaches to setting the difficulty of training activities compared to an active control condition immediately and 6-month postintervention. While the aim is exploratory, we hypothesized that children allocated to a working memory training condition would show greater improvements: (1) on near transfer measures compared to intermediate and far transfer measures and (2) immediately postintervention compared to 6-month postintervention.

    Methods: This double-blinded, active-controlled, parallel-group randomized trial aimed to recruit 128 children aged 7 to 11 years from 1 metropolitan primary school in Melbourne, Australia. Following baseline testing, children were randomized into 1 of 4 conditions: adaptive, self-select, or stepwise working memory training, or active control. An experimental intervention embedded in Minecraft was developed for teachers to deliver in class over 2 consecutive weeks (10 × 20-minute sessions). The working memory training comprised 2 training activities with processing demands similar to daily activities: backward span and following instructions. The control comprised creative activities. Pre- and postintervention, children completed a set of working memory tests (near and intermediate transfer) and the Raven’s Standard Progressive Matrices (far transfer) to determine training outcomes, as well as motivation questionnaires to determine if motivations toward learning and the intervention were similar across conditions. Caregivers completed the ADHD-Rating Scale-5 to measure their child’s attention (far transfer). Statistical analysis will include traditional null hypothesis significance testing and Bayesian methods to quantify evidence for both the null and alternative hypotheses.

    Results: Data collection concluded in December 2022. Data are currently being processed and analyzed.

    Conclusions: This trial will determine whether the adaptive approach to setting the difficulty of training activities maximizes cognitive training outcomes for children. This trial has several strengths: it adopts best practices for cognitive training studies (design, methods, and analysis plan); uses a range of measures to detect discrete levels of transfer; has a 6-month postintervention assessment; is appropriately powered; and uses an experimental working memory training intervention based on our current understanding of the cognitive mechanisms of training. Findings will inform future research and design of cognitive training interventions and highlight the value of the evidence-based principles of cognitive training.

    Trial Registration: Australian New Zealand Clinical Trials Registry, ACTRN12621000990820; https://www.anzctr.org.au/ACTRN12621000990820.aspx

    International Registered Report Identifier (IRRID): DERR1-10.2196/47496

    JMIR Res Protoc 2023;12:e47496

    doi:10.2196/47496

    Keywords

    children; working memory; memory training; adaptive training; cognitive training; transfer effects; training effects; cognitive outcomes; randomized controlled trial 


    Background

    Cognitive training interventions targeting core cognitive abilities such as working memory, attention, and problem-solving are in demand. Supporters of cognitive training explain that repeated practice on an activity will lead to performance gains that can transfer to improved performance on untrained tasks, an outcome known as transfer [1-4]. Transfer is the ultimate goal of cognitive training, and this intervention method is therefore of considerable interest to practitioners in health and education sectors seeking to prevent and ameliorate the disadvantage associated with low cognitive abilities, including inattention and learning related challenges (eg, psychologists, psychiatrists, and other mental health professionals; speech and language therapists; pediatricians; and teachers, special education needs coordinators, and specialist teachers). Transfer has been described on a continuum, ranging from near transfer to far transfer. Near transfer refers to improvement on measures structurally similar to the training activity, for example, performance on a measure using the same paradigm as the training activity but different stimulus, mode of presentation, or response modality. Far transfer focuses on performances on measures beyond the trained cognitive domain that rely on the trained skill. More recently, researchers have also been examining intermediate transfer, which describes improvement on measures that assess the same cognitive domain but use a different paradigm to the training activity. Working memory training benefits typically diminish with time, and with decreasing similarities between the training activity and outcome measure, providing consistent evidence for near transfer, less evidence for intermediate transfer, and little convincing evidence for far transfer [5-8].

    A key design feature of cognitive training that is assumed to maximize training outcomes is adaptivity of the training activities. In this approach, the difficulty of the training activity is adjusted to the trainee’s performance, thought to sufficiently challenge the trainee’s cognitive limits to induce plasticity [9-12]. Indeed, most commercially available cognitive training programs are adaptive. Initial support for adaptive cognitive training in children came from studies demonstrating greater training effects and transfer following adaptive training, in which training difficulty varied, compared to a nonadaptive control, in which training difficulty was fixed at a low level [11,13-16]. This design is problematic for evaluating the adaptive method, with adaptivity and difficulty of the training activities confounded: children in the adaptive condition were exposed to varying difficulty levels, but those in the nonadaptive condition only experienced a single difficulty level. To address this important limitation, von Bastian and Eschen [7] systematically examined in adults the outcomes of working memory training with difficulty of the training activities being adaptive, self-selected, or randomized compared to an active control. The adults in the adaptive condition outperformed those in the active control condition on the training activity, but this was also the finding for the self-selected and randomized conditions. While near transfer effects were not examined, none of the training conditions showed improvements on intermediate or far transfer tests. The authors postulated that varying the difficulty level of the training activities may be sufficient to induce training effects, and that training and transfer effects may not be modulated by a specific approach to setting the difficulty of a training activity. These findings question the common assumption that an adaptive approach is superior for maximizing cognitive training outcomes.

    Working memory training provides the ideal testbed to systematically examine the assumption that the adaptive approach maximizes cognitive training outcomes. It is one of the most widely studied domains in the cognitive training literature [17,18]. Working memory is a mental workspace capable of holding and processing information for brief periods in the course of ongoing cognitive activity [19]. This ability is critical for children’s learning as it facilitates a host of complex cognitive activities such as comprehension, reasoning, and problem-solving [20]. Indeed, low working memory is commonly associated with poor scholastic attainments [20,21]. Meta-analyses examining near transfer effects following cognitive training reveal the highest effect sizes for training targeting working memory (g=0.50) than either inhibitory control (g=0.24) or cognitive flexibility (g=0.37) [6,17]. Evidence from high-quality trials also demonstrates near transfer effects of working memory training programs in children, such as Cogmed [15,22] and Braintwister [16,23]. However, there is less support for intermediate transfer, and the evidence for far transfer is negligible when methodological issues, such as the lack of an active control group, are taken into account [6,8,17,22,24-26].

    Aims and Hypotheses

    This randomized controlled trial assessed the common assumption that cognitive training in children should be adaptive. The trial aimed to examine children’s outcomes of working memory training using adaptive, self-select, and stepwise approaches to setting the the difficulty of training activities compared to an active control condition immediately and 6-month postintervention. Although the aim is exploratory, we hypothesized that children allocated to the working memory training conditions would show greater improvements: (1) on near transfer compared to intermediate and far transfer measures and (2) immediately postintervention compared to 6-month postintervention.


    Ethics Approval

    This trial was approved by the Monash University Human Research Ethics Committee (24305) in January 2021 and Melbourne Archdiocese Catholic Schools (1066) in February 2021. It was registered with the Australian New Zealand Clinical Trials Registry (ACTRN12621000990820) on July 28, 2021. We note that due to time constraints associated with COVID-19 lockdowns in Melbourne, Australia, the first participant was enrolled in the trial on July 13, 2021, when approval from the Australian New Zealand Clinical Trials Registry was pending, resulting in the trial’s retrospective registration status.

    Design and Blinding

    This double-blinded, active-controlled, parallel-group, randomized superiority trial is part of the larger Common Assumptions of Cognitive Training (COMET) study. Researchers involved in child assessments were blinded to the child’s intervention allocation and past assessment results. Teachers, caregivers, and children were informed that the study was examining children’s thinking skills, and they were unaware of the different conditions. Thus, participants were unaware that this was a randomized controlled trial with different conditions, thereby preserving blinding. This trial was conducted and will be reported according to CONSORT (Consolidated Standards of Reporting Trials) guidelines. Figure 1 summarizes the trial design. This trial was expected to run for 15 months, with school selection and recruitment taking 3 months, and consent, data collection, and intervention carried out over 12 months.

    ‎
    Figure 1. Trial design.

    Setting

    The research was conducted in a primary school in metropolitan Melbourne in the state of Victoria (population of 6.6 million in 2022), Australia [27].

    School Recruitment

    Eligible primary schools were within 20 km of the Monash University Clayton campus (for practical reasons) and had at least 197 children in grades 2 to 5 (to achieve the required sample size based on power calculations). Schools were randomly invited until one agreed to participate.

    Participant Recruitment

    All children in grades 2 to 5 (7 to 11 years) at the participating school were invited to participate in this trial. The grade 2 to 5 teachers distributed the caregiver explanatory statement and caregiver consent form for the children to take home to their caregivers. Caregivers were asked to provide consent for completing questionnaires about themselves and their child, and for their child to participate in cognitive testing. The children returned the completed consent forms to their teachers to pass on to the research team. Child assent was obtained during the baseline child testing session. Consent was sought from the teachers for their involvement in obtaining caregiver consent.

    Eligibility Criteria

    Children were included if they had caregiver consent, and provided written assent to participate. They were excluded if they had caregiver-reported vision impairment that cannot be corrected by glasses, including color vision deficiency, hearing impairment that cannot be corrected by a hearing aid, fine motor impairment, or intellectual disability that would prevent participation in the intervention.

    Randomization

    Following baseline testing, children were randomized to 1 of the 4 number-coded conditions (adaptive, self-select, stepwise working memory training, or active control) using a randomization scheme generated in advance and set up in Research Electronic Data Capture (REDCap) hosted at Monash University by the trial statistician (JFW) who was not involved in data collection. Specifically, block sizes of variable size (4 or 8) were used to assure allocation concealment and preguessing of the allocation sequence at the end of each block. Randomization was stratified by age at baseline: 7 to 8 years, 9 to 10 years, and 11 years.

    Intervention

    Overview

    The Brain Space Program intervention was designed and developed by our team in Minecraft Education (Mojang Studios and Xbox Game Studios) for the COMET project (not an official Minecraft product, not approved by or associated with Mojang), and in this trial includes 3 working memory training conditions (adaptive, self-select, and stepwise) and an active control condition. All conditions were in the same Minecraft environment and had the same motivating features, including a “space mission” narrative for each session and the acquisition of experience points that could be used at the end of each training session for creative activities.

    The teacher delivered the intervention in class. Children completed a session lasting 20 minutes on each school day for 2 consecutive weeks (total 10 sessions, total dose 200 minutes). Children individually performed the intervention on an iPad with headphones (provided by the researchers) to reduce distraction. To account for missed sessions (eg, child absences, school events, and public holidays), children were allowed to complete up to 3 training sessions per day during the intervention period. Researchers were in the classroom during the intervention period to ensure protocol adherence and compliance.

    Working Memory Training Conditions

    The working memory training consisted of 2 activities that required the child to temporarily store and manipulate verbally presented information: a backward span activity and a following instructions activity (see Figure 2). These paradigms were selected based on the current understanding of the cognitive mechanisms of training. Importantly, first, processing demands are sufficiently unfamiliar so the child is required to generate novel cognitive routines (strategies) that improve efficiency in performance and can be applied to new activities with sufficient overlap in task structure [5,6]. Second, processing demands overlap with daily activities, such as following teacher directions, leading to potential practical benefits [2,28].

    ‎
    Figure 2. The working memory training activities: (A) backward span and (B) following instructions. ms = milliseconds, ISI = Interstimulus Interval.

    In the backward span training activity, a series of digits were presented auditorily, and the child had to remember and immediately recall in reverse order the sequence of items by tapping the digits on the iPad screen. The difficulty level was manipulated by increasing the number of digits to be recalled. The following instructions training activity is based on a classroom analog test of working memory developed by Gathercole et al [29]. In this training activity, the child was introduced to 3 objects, each object comes in 3 colors, and 2 actions that require different tapping responses on the iPad screen. The child had to remember and immediately implement an action-color-object sequence (eg, break the red wire, then place the blue button, reflecting a span of 2) by using the relevant tapping response on the screen of the iPad with the corresponding color-objects. Difficulty level was manipulated by increasing the number of action-color-objects in the sequence the child has to perform. Each activity comprised 5 blocks of trials per session, and each block had 4 trials of a span level (per session: total 20 backward span trials, then 20 following instructions trials).

    In this trial, there were three different working memory training conditions, each with a different approach to adjusting the difficulty level of the training activities: (1) adaptive condition, the difficulty level adapted to the child’s performance during each training session on a block-by-block basis for the first 4 blocks (adapted between block 1 to 2, block 2 to 3, and block 3 to 4, and block 5 retained the same difficulty as block 4) within the training sessions for each activity; (2) self-select condition, the child decided at the end of each session if they would like a training activity to be easier, the same, or more difficult in the next session; and (3) stepwise condition, the difficulty level increased incrementally across sessions and this progression was predetermined and stratified by age based on current understanding of the development of children’s working memory: 7 years, 8 to 9 years, and 10 to 11 years [30].

    Active Control Condition

    The active control involved the child completing a series of creative building and discovery activities each session. The activities were embedded in the same environment as the working memory training activities, and therefore the condition can be considered a placebo [1]. The activities were selected based on a review of activities available on the Minecraft Education website for school teachers to use with children aged 7 to 11 years in the classroom that do not require working memory [31].

    Measures

    Overview

    Table 1 summarizes the measures, respondents, administration time points, as well as the primary outcomes of this trial.

    Table 1. Trial measures.
    Domain or measureRespondentBaselineImmediately postintervention6 months postintervention
    Child characteristics

    		Participant information questionnaireCaregiver✓N/AaN/A
    Near transfer

    		Backward span digitsChild✓Primary outcome✓

    		Following instructions objectsChild✓Primary outcome✓

    		Backward span lettersChild✓✓✓

    		Following instructions lettersChild✓✓✓
    Intermediate transfer

    		2-back objectsChild✓✓✓
    Far transfer

    		Raven’s SPMb sets A to EChild✓N/AN/A

    		Raven’s SPM sets A and BChildN/A✓✓

    		ADHD-5-RScCaregiver✓✓✓
    Child motivation

    		Intrinsic Motivation ScaleChild✓N/AN/A

    		Intrinsic Motivation InventoryChildN/A✓N/A

    aN/A: Not applicable, test not performed at this time point.

    bSPM: Standard Progressive Matrices.

    cADHD-5-RS: ADHD-Rating Scale-5, home version.

    Child Characteristics

    Caregivers completed a participant information questionnaire that collected demographic information about their child, including their child’s sex, developmental and medical history, and family socioeconomic risk factors [32].

    Training Outcomes
    Near Transfer

    A set of four experimental working memory tests that have the same paradigms as the training activities, but different stimulus features, provided measures of near transfer (Table 2, Figures 3A to 3D): (1) backward span tests, digits and letters versions, and (2) following instructions tests, objects and letters versions [6]. For each test, there were 4 trials per block of a span level. Responses were self-paced with a cut-off duration for response time adjusted per span level. The span level increased by 1 if the child scored 3 or more out of the 4 trials in that block (≥75% accuracy); otherwise, the test ended. The tests, including instructions, practice items, and corrective feedback, were delivered via an iPad-based web app developed for this research. These multiple near transfer tests provide the opportunity to detect discrete levels of transfer by minimizing confounders associated with training effects [4].

    Table 2. Comparison of the working memory training activities and near transfer tests. Responses provided on an iPad by tapping on-screen buttons.
    Activity or testStimulus typeStimulus, presentationResponse modality (response layout)
    Backward span

    		Training activityDigitsAuditoryMotor (3×3 keypad)

    		Near transfer testDigitsVisualMotor (3×3 keypad)

    		Near transfer testLettersAuditoryMotor (3×3 keypad)
    Following instructions

    		Training activityItems: space objects; actions: place, breakAuditoryMotor (3×3 grid)

    		Near transfer testItems: common everyday objects and animals; actions: place, breakVisualMotor (3×3 grid)

    		Near transfer testItems: letter cards; actions: touch, flipAuditoryMotor (2-row keyboard)
    ‎
    Figure 3. The experimental working memory transfer tests delivered on an iPad: (A) backward span digits, (B) backward span letters, (C) following instructions objects, (D) following instructions letters, (E) 2-back objects. (A) to (D) depict each test at a span level of 2. ms = milliseconds, ISI = Interstimulus Interval.
    Intermediate Transfer

    An experimental n-back test measured intermediate transfer (Figure 3E). In this 2-back test, the child was presented with a continuous stream of 22 common objects and animals (eg, carrot, flower, and cat; 6 targets and 16 nontargets), and the child had to decide for each object if it matched the object that was presented 2 items ago [5,33,34]. The child responded by tapping the “same” or “not same” on-screen button within the time limit. This test was delivered via the same iPad-based web app as the near transfer tests.

    Far Transfer

    Far transfer measures assessed children’s reasoning and attention. Children’s reasoning ability was measured using the Raven’s Standard Progressive Matrices (SPM), standardized for individuals aged 6.5 to 80 years [35,36]. The Raven’s SPM contains 5 sets (A to E) of 12 items. Each item has a geometric design with a missing piece on the bottom right. The child had to select the correct figure that would complete the overall geometric design vertically and horizontally from 6 or 8 possible options. The items increased in difficulty within sets and across sets. Children were allowed as much time as they needed to complete the task. The Raven’s SPM has excellent split half-reliability and test-retest reliability (>0.80), and its concurrent validity is well established [37]. There is a moderately strong correlation (0.50 to 0.70) between SPM and conventional intelligence tests such as Stanford-Binet and Wechsler’s tests [38].

    Children’s inattention and hyperactivity-impulsivity behaviors were assessed using the caregiver-rated ADHD-Rating Scale-5 (ADHD-RS-5, home version), standardized for children and adolescents aged 5 to 17 years [39]. The ADHD-RS-5 has an inattention subscale (9 items) and a hyperactivity-impulsivity subscale (9 items) that closely follow the Diagnostic and Statistical Manual of Mental Disorders (Fifth Edition) criteria for attention-deficit/hyperactivity disorder. Caregivers rated their child’s behaviors at home based on the previous 6 months, and at the immediate postintervention time point based on the past 2 weeks to assess any changes observed during the intervention period. Internal consistency of the symptom subscales (0.61 to 0.83) and concurrent validity were established by high correlations (>0.78) with respective subscales from the Connors Parent Rating Scale [39].

    Child Motivation
    Overview

    Intrinsic motivation, an internal drive to seek out new and challenging experiences to learn and expand one’s abilities [40], has been associated with performances on cognitive training activities and outcomes in children [41-43] and adults [44]. We measured aspects of children’s motivation to rule out the potential influence of differences in expectations between the working memory training conditions and the active control [1,45].

    Motivation Toward Classroom Activities

    Children’s motivation toward typical classroom activities was measured using the Intrinsic Motivation Scale (IMS) [46]. The IMS consists of 3 subscales: challenge (6 items; eg, I like hard work because it’s difficult), curiosity (6 items; eg, I ask questions in class because I want to learn new things), and independent mastery (5 items; eg, I like to try to figure things out at school on my own). The child responded using a 3-point Likert scale (yes: 3 points; sometimes: 2 points; no: 1 point). The IMS has very high internal consistency (0.90) and high test-retest reliability with a 6-week interval (0.74) [46].

    Motivation Toward the Intervention

    Children’s motivations toward the intervention were measured using a modified Intrinsic Motivation Inventory (IMI) [47], which has been used in child cognitive training studies [41-43]. This modified IMI consists of 4 subscales: interest or enjoyment (7 items; eg, it was fun to do), perceived competence (6 items; eg, I think I am pretty good at the training), effort or importance (5 items; eg, I put a lot of effort into it), and value or usefulness (4 items; eg, I think the training could help me). Identical to the IMS, the child responded on a 3-point Likert scale. This modified IMI has been used in a previous randomized control trial of working memory training with similar-aged children and has high internal consistency (≥0.74) [41].

    Data Management and Confidentiality

    Personal and trial data were collected and managed using REDCap electronic data capture tools hosted and managed by Helix (Monash University) [48,49]. Personal data required to disseminate the caregiver questionnaires were held securely and will not be used for any other purpose. Given the sample size, the trial’s timeframe, and the minimal risks associated with participation, there is no data monitoring committee.

    Procedures

    Children participated in testing sessions, and caregivers completed behavioral questionnaires at baseline, immediately, and 6 months postintervention. The baseline and immediately postintervention measures were completed 1 week before and after the intervention, respectively. The 6-month postintervention measures were completed approximately 26 weeks postintervention (±8 weeks due to school term breaks).

    Children were tested in small groups of 6 to 8. The group child testing sessions were led by a doctoral provisional psychologist and conducted in a quiet room on the primary school campus. The test order and administration protocol was the same at each time point. Children first completed the working memory measures on an iPad with headphones to reduce distraction, the Raven’s SPM, and then motivation questionnaires. General task instructions were delivered to the children as a group. Children were encouraged to ask clarifying questions, and the researcher responded with standardized additional instructions and ensured the child understood how to complete the task before proceeding with testing.

    Power Calculations and Sample Size

    Calculations in G*Power (effect size f2=0.07; 80% power; α=.05) [50,51] indicated a sample size of 29 children per condition would be required to detect a small to moderate effect of each working memory training condition (adaptive, self-select, and stepwise) compared to the active control on the near transfer measures immediately postintervention (primary outcome). This effect size was based on previous research, which indicates working memory training programs produce a small to moderate effect compared to a control condition on near transfer measures [6,17]. The target sample size was increased to 32 children per condition to account for 10% attrition [22]. Our previous school-based trial of a working memory training program had a 65% child recruitment rate [22]. Thus, to achieve our target sample size, the participating primary school had to have at least 197 children enrolled in grades 2 to 5.

    Statistical Analysis

    Primary analyses will be intention to treat (ITT) with sensitivity analyses only with children who completed the allocated intervention. Primary analyses will be separate linear regressions for each primary outcome at the primary end point (immediately postintervention) as the dependent variable, the outcome at baseline and the stratification factor (age) as covariates, and conditions (dummy coded with the active control as the reference) as the independent variables. Statistical significance will be set at α=.05 (2-tailed). If outcomes do not meet the normality assumption, a nonparametric bootstrap will be used. Effect sizes and 95% CI will be reported to demonstrate the magnitude of differences in outcomes between each training condition relative to the active control. Bayesian methods will be used to quantify the strength of evidence in favor of the null hypothesis and the alternative hypothesis. Interpretation of results will consider the strength of effects and P values. The same analytic approach will be applied to secondary outcomes and to both primary and secondary outcomes at the 6-month postintervention time point.


    This trial was supported by internal research funding from the School of Psychological Sciences and the Turner Institute for Brain and Mental Health at Monash University, awarded in December 2019. Data collection was attempted in July 2021 but was ceased shortly after due to COVID-19 lockdowns in Melbourne. Data collection occurred from February 2022 to December 2022. Data are currently being processed and analyzed. It is anticipated that the results of this trial will be published by 2024.


    The potential transfer of skills learned from a cognitive training intervention to other untrained activities is of interest to practitioners in health and education sectors, and families. Cognitive training interventions are intensive and typically comprise adaptive training activities, yet whether this approach to setting the difficulty of training activities maximizes transfer effects has only been systematically evaluated once in adults [7]. This trial aims to examine outcomes of working memory training using adaptive, self-select, and stepwise approaches to setting the difficulty level of training activities compared to an active control condition. This will be the first trial to systematically examine methods of setting the difficulty of working memory training activities in children. Our design includes evaluation of nonadaptive approaches (self-select and stepwise) and thereby eliminates task difficulty as a confounder, which was an important limitation of initial studies in the field.

    This trial has important strengths. The trial design, methods, and analysis plan are in line with best-practice standards for cognitive training trials [1]. The trial was randomized, participants and assessors were blinded, and an active control condition was used. The active control condition (instead of a passive or no-contact control) was designed to be believable as an intervention, indistinguishable from the working memory training conditions in terms of the environment and motivating features, and thus was designed to ensure children experience a similar level of engagement, and experimenter and teacher attention to children allocated to the working memory training conditions. The set of outcome measures was designed to detect discrete levels of near transfer by minimizing confounders associated with training effects [4], and intermediate and far transfer effects are measured too. Training outcomes were assessed immediately postintervention and again at 6-month postintervention to determine any persisting training benefits. The trial is sufficiently powered to detect differences between the working memory training conditions and the active control condition. Importantly, our novel experimental working memory training interventions are designed specifically for this research and reflect the current understanding of the cognitive mechanisms of training: the processing demands of the training activities are (1) sufficiently unfamiliar such that the child needs to generate novel cognitive routines to do well, and the routines can then be applied to new activities that are similar but untrained; and (2) overlap with daily activities, such as following teacher directions, which overlap with daily activities. This is the first time the following instructions paradigm has been used as a working memory training activity. The nonadaptive approaches used in this trial reflect a similar approach to learning that children experience in daily activities, such as having autonomy over their learning (self-select), and mastering simpler concepts before complex concepts (stepwise). Motivation questionnaires will be used in analyses to determine if children in the training and active control conditions have similar motivation toward learning in the classroom and expectations of the intervention.

    It is acknowledged that this trial was not powered to detect potential differences between the adaptive, self-select, and stepwise working memory training conditions. The lack of research and effect sizes from nonadaptive approaches makes calculating appropriate power challenging. We anticipate the differences in effect sizes between the 3 conditions will be small, and therefore a particularly large sample size would be required and this may not be practical or feasible.

    This trial will determine whether the adaptive approach to setting the difficulty of working memory training activities is superior for maximizing training outcomes. It is our intention to present findings at national and international conferences and publish findings in peer-reviewed journal papers. Families and staff of the participating school will receive a summary of the findings. Findings will contribute to our understanding of the design of effective working memory training interventions, discrete levels of transfer of working memory training, and the malleability of cognitive functions in children. Knowledge gained could have implications for other types of cognitive training that target core domains such as attention and inhibitory control, and for other populations, including aging and dementia [3]. This work could help highlight the value of evidence-based principles of cognitive training and shift the focus from studying the effectiveness of different programs to more rigorous evaluations of the foundational principles of cognitive training. In summary, the results have the potential to influence the future direction and application of cognitive training.

    Acknowledgments

    RCL is supported by the Australian Government Research Training Program Scholarship, the Monash Graduate Excellence Scholarship, and the Tan Kah Kee Foundation postgraduate scholarship. JFW is supported by a National Health and Medical Research Council Fellowship (1178487). PJA is supported by a National Health and Medical Research Council Investigator Grant (#1176077). The project would not have been possible without the support of the School of Psychological Sciences and Turner Institute for Brain and Mental Health at Monash University.

    Data Availability

    The data sets generated and analyzed during this study are not publicly available due to the restricted consent from participants but may be available from the corresponding author on reasonable request.

    Authors' Contributions

    MSS, PJA, and SG conceived of the trial and initiated the trial design. RCL, PJA, SG, JFW, and MSS made substantial contributions to the conception and design of the trial. RCL, PJA, SG, and MSS designed the intervention. RCL, PJA, and MSS designed the near and intermediate outcome measures. PM, FS, and DH developed the intervention and outcome measures. JFW provided statistical expertise in clinical trial design. RCL and MSS drafted this paper. All authors have reviewed and contributed to the final version of this paper. All authors have read and approved the final paper.

    Conflicts of Interest

    None declared.

    1. Simons DJ, Boot WR, Charness N, Gathercole SE, Chabris CF, Hambrick DZ, et al. Do "Brain-Training" programs work? Psychol Sci Public Interest. 2016;17(3):103-186. [CrossRef] [Medline]
    2. Jaeggi SM, Buschkuehl M. Working memory training and transfer: theoretical and practical considerations. In: Toni B, editor. Springer Proceedings in Mathematics & Statistics: New Frontiers of Multidisciplinary Research in STEAM-H (Science, Technology, Engineering, Agriculture, Mathematics, and Health). Cham, Switzerland. Springer International Publishing; 2014;19-43.
    3. Gobet F, Sala G. Cognitive training: a field in search of a phenomenon. Perspect Psychol Sci. 2023;18(1):125-141. [FREE Full text] [CrossRef] [Medline]
    4. von Bastian CC, Belleville S, Udale RC, Reinhartz A, Essounni M, Strobach T. Mechanisms underlying training-induced cognitive change. Nat Rev Psychol. 2022;1(1):30-41. [CrossRef]
    5. Byrne EM, Ewbank MP, Gathercole SE, Holmes J. The effects of transcranial direct current stimulation on within- and cross-paradigm transfer following multi-session backward recall training. Brain Cogn. 2020;141:105552. [FREE Full text] [CrossRef] [Medline]
    6. Gathercole SE, Dunning DL, Holmes J, Norris D. Working memory training involves learning new skills. J Mem Lang. 2019;105:19-42. [FREE Full text] [CrossRef] [Medline]
    7. von Bastian CC, Eschen A. Does working memory training have to be adaptive? Psychol Res. 2016;80(2):181-194. [CrossRef] [Medline]
    8. Sala G, Gobet F. Working memory training in typically developing children: a multilevel meta-analysis. Psychon Bull Rev. 2020;27(3):423-434. [FREE Full text] [CrossRef] [Medline]
    9. Lövdén M, Bäckman L, Lindenberger U, Schaefer S, Schmiedek F. A theoretical framework for the study of adult cognitive plasticity. Psychol Bull. 2010;136(4):659-676. [CrossRef] [Medline]
    10. Klingberg T. Training and plasticity of working memory. Trends Cogn Sci. 2010;14(7):317-324. [CrossRef] [Medline]
    11. Holmes J, Gathercole SE, Dunning DL. Adaptive training leads to sustained enhancement of poor working memory in children. Dev Sci. 2009;12(4):F9-F15. [CrossRef] [Medline]
    12. Astle DE, Barnes JJ, Baker K, Colclough GL, Woolrich MW. Cognitive training enhances intrinsic brain connectivity in childhood. J Neurosci. 2015;35(16):6277-6283. [FREE Full text] [CrossRef] [Medline]
    13. Dunning DL, Holmes J, Gathercole SE. Does working memory training lead to generalized improvements in children with low working memory? A randomized controlled trial. Dev Sci. 2013;16(6):915-925. [FREE Full text] [CrossRef] [Medline]
    14. Klingberg T, Fernell E, Olesen PJ, Johnson M, Gustafsson P, Dahlström K, et al. Computerized training of working memory in children with ADHD--a randomized, controlled trial. J Am Acad Child Adolesc Psychiatry. 2005;44(2):177-186. [FREE Full text] [CrossRef] [Medline]
    15. Klingberg T, Forssberg H, Westerberg H. Training of working memory in children with ADHD. J Clin Exp Neuropsychol. 2002;24(6):781-791. [CrossRef] [Medline]
    16. Karbach J, Strobach T, Schubert T. Adaptive working-memory training benefits reading, but not mathematics in middle childhood. Child Neuropsychol. 2015;21(3):285-301. [CrossRef] [Medline]
    17. Kassai R, Futo J, Demetrovics Z, Takacs ZK. A meta-analysis of the experimental evidence on the near- and far-transfer effects among children's executive function skills. Psychol Bull. 2019;145(2):165-188. [FREE Full text] [CrossRef] [Medline]
    18. Luis-Ruiz S, Caldú X, Sánchez-Castañeda C, Pueyo R, Garolera M, Jurado MA. Is cognitive training an effective tool for improving cognitive function and real-life behaviour in healthy children and adolescents? A systematic review. Neurosci Biobehav Rev. 2020;116:268-282. [CrossRef] [Medline]
    19. Baddeley AD, Hitch G. Working memory. Psychol Learn Motiv. 1974;8:47-89. [CrossRef]
    20. Cowan N. Working memory underpins cognitive development, learning, and education. Educ Psychol Rev. 2014;26(2):197-223. [FREE Full text] [CrossRef] [Medline]
    21. Gathercole SE. Working memory: a system for learning. In: Wagner RK, Muse AE, Tannenbaum KR, editors. Vocabulary Acquisition: Implications for Reading Comprehension. New York, NY. Guilford Press; 2007;233-248.
    22. Roberts G, Quach J, Spencer-Smith M, Anderson PJ, Gathercole S, Gold L, et al. Academic outcomes 2 years after working memory training for children with low working memory: a randomized clinical trial. JAMA Pediatr. 2016;170(5):e154568. [FREE Full text] [CrossRef] [Medline]
    23. Loosli SV, Buschkuehl M, Perrig WJ, Jaeggi SM. Working memory training improves reading processes in typically developing children. Child Neuropsychol. 2012;18(1):62-78. [CrossRef] [Medline]
    24. Schwaighofer M, Fischer F, Bühner M. Does working memory training transfer? A meta-analysis including training conditions as moderators. Educ Psychol. 2015;50(2):138-166. [CrossRef]
    25. Spencer-Smith M, Quach J, Mensah F, Roberts G, Gathercole S, Wake M, et al. The effectiveness of working memory training for children with low working memory. Pediatrics. 2020;146(6):e20194028. [FREE Full text] [CrossRef] [Medline]
    26. Melby-Lervåg M, Redick TS, Hulme C. Working memory training does not improve performance on measures of intelligence or other measures of "far transfer": evidence from a meta-analytic review. Perspect Psychol Sci. 2016;11(4):512-534. [FREE Full text] [CrossRef] [Medline]
    27. National, state and territory population. Australian Bureau of Statistics. 2022. URL: https:/​/www.​abs.gov.au/​statistics/​people/​population/​national-state-and-territory-population/​sep-2022#states-and-territories [accessed 2023-06-11]
    28. von Bastian CC, Oberauer K. Effects and mechanisms of working memory training: a review. Psychol Res. 2014;78(6):803-820. [FREE Full text] [CrossRef] [Medline]
    29. Gathercole SE, Durling E, Evans M, Jeffcock S, Stone S. Working memory abilities and children's performance in laboratory analogues of classroom activities. Appl Cognit Psychol. 2008;22(8):1019-1037. [CrossRef]
    30. Gathercole SE, Pickering SJ, Ambridge B, Wearing H. The structure of working memory from 4 to 15 years of age. Dev Psychol. 2004;40(2):177-190. [CrossRef] [Medline]
    31. Lessons for minecraft education. Minecraft Education. URL: https://education.minecraft.net/en-us/resources/explore-lessons [accessed 2023-06-06]
    32. Roberts G, Howard K, Spittle AJ, Brown NC, Anderson PJ, Doyle LW. Rates of early intervention services in very preterm children with developmental disabilities at age 2 years. J Paediatr Child Health. 2008;44(5):276-280. [FREE Full text] [CrossRef] [Medline]
    33. Katz B, Jaeggi S, Buschkuehl M, Stegman A, Shah P. Differential effect of motivational features on training improvements in school-based cognitive training. Front Hum Neurosci. 2014;8:242. [FREE Full text] [CrossRef] [Medline]
    34. Jaeggi SM, Buschkuehl M, Perrig WJ, Meier B. The concurrent validity of the N-back task as a working memory measure. Memory. 2010;18(4):394-412. [CrossRef] [Medline]
    35. Raven JC. Standardization of progressive matrices, 1938. Br J Med Psychol. 1941;19(1):137-150. [CrossRef]
    36. Raven JC. Progressive Matrices: A Perceptual Test of Intelligence. In: Individual Form. Oxford, United Kingdom. Oxford Psychologists Press Ltd; 1938.
    37. Raven J, Raven JC, Court JH. Raven Manual: Section 3. Standard Progressive Matrices. Oxford, United Kingdom. Oxford Psychologists Press Ltd; 2000.
    38. Strauss E, Sherman EM, Spreen O. A Compendium of Neuropsychological Tests: Administration, Norms, and Commentary, Third Edition. Oxford, New York. Oxford University Press; 2006.
    39. DuPaul GJ, Power TJ, Anastopoulos AD, Reid R. ADHD Rating Scale-5 for Children and Adolescents: Checklists, Norms, and Clinical Interpretation. New York, London. Guilford Publications; 2016.
    40. Deci EL, Ryan RM. Intrinsic Motivation and Self-Determination in Human Behavior. Boston, MA. Springer US; 1985.
    41. Pascoe L, Spencer-Smith M, Wiley J, Lee K, Roberts G, Josev E, et al. Child motivation and family environment influence outcomes of working memory training in extremely preterm children. J Cogn Enhanc. 2019;3(4):396-404. [CrossRef]
    42. Jaeggi SM, Buschkuehl M, Jonides J, Shah P. Short- and long-term benefits of cognitive training. Proc Natl Acad Sci U S A. 2011;108(25):10081-10086. [FREE Full text] [CrossRef] [Medline]
    43. Söderqvist S, Matsson H, Peyrard-Janvid M, Kere J, Klingberg T. Polymorphisms in the dopamine receptor 2 gene region influence improvements during working memory training in children and adolescents. J Cogn Neurosci. 2014;26(1):54-62. [CrossRef] [Medline]
    44. Parong J, Seitz AR, Jaeggi SM, Green CS. Expectation effects in working memory training. Proc Natl Acad Sci U S A. 2022;119(37):e2209308119. [FREE Full text] [CrossRef] [Medline]
    45. Boot WR, Simons DJ, Stothart C, Stutts C. The pervasive problem with placebos in psychology: why active control groups are not sufficient to rule out placebo effects. Perspect Psychol Sci. 2013;8(4):445-454. [FREE Full text] [CrossRef] [Medline]
    46. Lepper MR, Corpus JH, Iyengar SS. Intrinsic and extrinsic motivational orientations in the classroom: age differences and academic correlates. J Educ Psychol. 2005;97(2):184-196. [CrossRef]
    47. Deci EL, Ryan RM. Intrinsic Motivation Inventory (IMI). Center for Self-Determination Theory. URL: https://selfdeterminationtheory.org/intrinsic-motivation-inventory/ [accessed 2020-11-13]
    48. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)--a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377-381. [FREE Full text] [CrossRef] [Medline]
    49. Harris PA, Taylor R, Minor BL, Elliott V, Fernandez M, O'Neal L, et al. REDCap Consortium. The REDCap consortium: building an international community of software platform partners. J Biomed Inform. 2019;95:103208. [FREE Full text] [CrossRef] [Medline]
    50. Faul F, Erdfelder E, Lang AG, Buchner A. G*Power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. Behav Res Methods. 2007;39(2):175-191. [CrossRef] [Medline]
    51. Faul F, Erdfelder E, Buchner A, Lang AG. Statistical power analyses using G*Power 3.1: tests for correlation and regression analyses. Behav Res Methods. 2009;41(4):1149-1160. [CrossRef] [Medline]

    ‎
    ADHD-RS-5: ADHD-Rating Scale-5, home version
    COMET: Common Assumptions of Cognitive Training
    CONSORT: Consolidated Standards of Reporting Trials
    IMI: Intrinsic Motivation Inventory
    IMS: Intrinsic Motivation Scale
    ITT: intention to treat
    REDCap: Research Electronic Data Capture
    SPM: Standard Progressive Matrices

    Edited by A Mavragani; submitted 24.03.23; peer-reviewed by Z Aghaei, S Wyver; comments to author 02.06.23; revised version received 15.06.23; accepted 05.07.23; published 19.09.23.

    Copyright

    ©Regine Cassandra Lau, Peter John Anderson, Joshua F Wiley, Derek Huang, Faisha Surjatin, Paul McIntosh, Susan Gathercole, Megan Spencer-Smith. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 19.09.2023.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.