This paper is in the following e-collection/theme issue:

Non-Randomized Studies (funded, eHealth) (102) Research Instruments, Questionnaires, and Tools (545)

Published on in Vol 13 (2024)

Preprints (earlier versions) of this paper are available at https://preprints.jmir.org/preprint/57663, first published .
Reliability and Sensitivity of a Virtual Assessment Developed for Workplace Concussions: Protocol for a Method-Comparison Study

Reliability and Sensitivity of a Virtual Assessment Developed for Workplace Concussions: Protocol for a Method-Comparison Study

Reliability and Sensitivity of a Virtual Assessment Developed for Workplace Concussions: Protocol for a Method-Comparison Study

Authors of this article:

Keely Barnes1, 2, 3 Author Orcid Image ;   Heidi Sveistrup1, 2, 4, 5 Author Orcid Image ;   Mark Bayley6, 7 Author Orcid Image ;   Michel Rathbone8 Author Orcid Image ;   Monica Taljaard3, 9 Author Orcid Image ;   Mary Egan1, 2 Author Orcid Image ;   Martin Bilodeau1, 2, 4 Author Orcid Image ;   Motahareh Karimijashni1, 3 Author Orcid Image ;   Shawn Marshall2, 3, 10 Author Orcid Image
Article Authors Cited by Tweetations Metrics

    Protocol

    1School of Rehabilitation Sciences, Faculty of Health Sciences, University of Ottawa, Ottawa, ON, Canada

    2Bruyère Research Institute, Ottawa, ON, Canada

    3Ottawa Hospital Research Institute, Ottawa, ON, Canada

    4School of Human Kinetics, Faculty of Health Sciences, University of Ottawa, Ottawa, ON, Canada

    5Systems and Computer Engineering Technology, Carleton University, Ottawa, ON, Canada

    6Kite Research Institute, Toronto Rehabilitation Institute, University Health Network, Toronto, ON, Canada

    7Division of Physical Medicine and Rehabilitation, Temerty Faculty of Medicine, University of Toronto, Toronto, ON, Canada

    8Department of Medicine, Division of Neurology, Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada

    9School of Epidemiology and Public Health, University of Ottawa, Ottawa, ON, Canada

    10Department of Medicine, University of Ottawa, Ottawa, ON, Canada

    Corresponding Author:

    Keely Barnes, MHK

    School of Rehabilitation Sciences

    Faculty of Health Sciences

    University of Ottawa

    Health Sciences Building (FHS) 200 Lees Avenue

    Ottawa, ON, K1N 6N5

    Canada

    Phone: 1 6136126127

    Email: kbarn076@uottawa.ca


    Background: Workplace mild traumatic brain injuries are frequently associated with persistent symptoms, leading to a reduction in productivity at work or even disability. People who sustain workplace injuries frequently need rehabilitation and support, and the challenges of delivering these services was heightened during the COVID-19 pandemic as injured workers had to be cared for remotely. Currently, clinicians are conducting both in-person and virtual (remote) concussion assessments; however, the measures that are being used to complete these assessments have undocumented psychometric properties.

    Objective: This study will document the psychometric properties of the clinical measures that are being used remotely and their ability to produce similar results to in-person assessments. Specifically, through this method-comparison study, we aim to (1) evaluate the sensitivity of the measures included in a virtual assessment toolkit when compared to an in-person assessment and (2) determine the interrater and intrarater reliabilities of the measures included in a virtual assessment toolkit.

    Methods: Patient participants (people living with acquired brain injuries) will attend two assessments (in person and virtual) at the Ottawa Hospital. The two assessments will be identical, consisting of the measures included in our previously developed virtual concussion assessment toolkit, which includes finger-to-nose testing, the Vestibular/Ocular Motor Screening tool, balance testing, cervical spine range of motion, saccades testing, and evaluation of effort. All virtual assessments will occur using the Microsoft Teams platform and will be audio/video-recorded. The clinician assessor and patient participant will complete a feedback form following completion of the assessments. A different clinician will also document the findings on observed videos of the virtual assessment shortly after completion of both in-person and virtual assessments and approximately 1 month later. Interrater reliability will be assessed by comparing the second clinician’s observation with the first clinician’s initial virtual assessment. Intrarater reliability will be evaluated by comparing the second clinician’s observation with their own assessment approximately 1 month later. Sensitivity will be documented by comparing the findings (identification of abnormality) of the in-person assessment completed by the initial clinician assessor with those of the second clinician assessor on the observation of the recording of the virtual assessment.

    Results: As of May 2024, we have recruited 7 clinician assessors and completed study assessments with 39 patient participants. The study recruitment is expected to be completed by September 2024.

    Conclusions: Currently, it is unknown if completing concussion assessments virtually produces similar results to the in-person assessment. This work will serve as a first step to determining the similarity of the virtual assessment to the matching in-person assessment and will provide information on the reliability of the virtual assessment.

    International Registered Report Identifier (IRRID): DERR1-10.2196/57663

    JMIR Res Protoc 2024;13:e57663

    doi:10.2196/57663

    Keywords

    telehealth; virtual care; concussion; mTBI; mild traumatic brain injury; assessment; examination 


    Background

    A direct or indirect head impact leading to a concussion is a form of mild traumatic brain injury [1], which results in an alteration of brain functioning [2,3]. Most individuals who sustain a concussion experience rapid recovery within the first 2 weeks to 1 month following the injury. There are, however, instances where individuals continue to experience persistent symptoms after a concussion, with a reported occurrence of approximately 15%-20% [4]. These symptoms may persist beyond 1 year following the injury [3,5]. Factors such as premorbid characteristics (age, sex, concussion history), psychological factors, and injury-related factors (ie, severity of injury) are likely contributors to the persistence of the symptoms [6]. Postconcussion symptom presentation can vary widely, including cognitive deficits such as memory issues, vestibular deficits such as balance issues or dizziness, and ocular deficits such as blurred vision, among others [1,7-9]. There is a need for a multifactorial assessment approach due to the diversity in symptom presentation. Specifically, it is recommended to incorporate measures targeting each of the commonly experienced symptoms into a battery of tests [10].

    There is evidence suggesting a growing documented incidence of concussion in the workplace context [11]. Kristman et al [12] reported that there are approximately 39-58 workplace concussions per 10,000 claims annually in Ontario, Canada [12]. Workplace concussions often lead to greater delays in return to work compared to injuries sustained outside of the workplace context [5,13]. It is estimated that roughly 60% to 90% of adults return to work within 6 months postconcussion injury, with the remainder continuing to be off work at this time point [5]. Compensation associated with a workplace injury may be a contributor to this delay in return to work [5,13].

    Several factors can contribute to delays in return to work following concussion, including a lower level of education, more severe immediate symptoms post injury, and other injuries sustained [5]. In general, workplace concussions pose significant challenges. These include financial burdens that are felt by the health care system, employers, the Workplace Safety and Insurance Board (WSIB), and the workers themselves [14]. For the patient, lives are disrupted due to prolonged recovery times [13], resulting in the need for additional support from the WSIB for individuals presenting for specialized assessments.

    The onset of the COVID-19 pandemic affected how specialized postconcussion assessments could be completed along with a need to switch to virtual care [15]. However, it is not known whether the measures being used by clinicians in virtual assessments are the same as those used for in-person assessments [16]. Additionally, the reliability of the virtual concussion assessments being conducted is unknown. Measures used in person that have shown strong psychometric properties may not produce the same or similar results when administered virtually [17]. Furthermore, the level of agreement between measures used in the in-person concussion assessment and those used in the virtual assessment has not yet been established. Currently, clinicians are continuing to complete both in-person and virtual concussion assessments; however, there is limited information on the measures used in the virtual assessment. It is important to gain an understanding of the properties of the clinical measures and their ability to produce similar results to the in-person assessment to inform the effectiveness of use in a virtual context [18].

    The preparatory work for this study was initiated by developing a virtual assessment toolkit. Clinician surveys were used to identify relevant measures for assessing the physical domains of concussion [19]. A working group refined the measures feasible for virtual administration based on expert opinion. Focus groups then identified barriers and facilitators for virtual concussion assessments. The resulting virtual assessment toolkit was then used in a feasibility study to evaluate recruitment rates, procedure acceptability, and time to assess; the manuscript reporting these findings is currently being prepared for submission.

    Objectives

    The objectives of this method-comparison study are two-fold: (1) to evaluate the sensitivity of the virtual assessment against the gold standard of the in-person assessment and (2) to determine interrater and intrarater reliabilities of the measures in the virtual assessment. This study will further contribute to the development of a virtual toolkit of assessments across four physical domains along with the evaluation of effort, specifically designed for the WSIB context.


    Design

    This study will follow a prospective method-comparison study design [17].

    Participants

    Patient Participants

    Eligible participants will be adults 18 years or older who have sustained an acquired brain injury (concussion or other form) and are being assessed by a physician, physician assistant, and/or physiotherapist at the Ottawa Hospital. In addition to people with concussions, people living with acquired brain injuries, including moderate to severe traumatic brain injury or hypoxic brain injury, will be recruited. In this study, we are assessing the consistency of physical examination findings; people with other forms of acquired brain injury such as severe traumatic brain injury more frequently have signs of central nervous system deficits, whereas these assessments are often normal during a postconcussion examination [20,21]. This inclusion is thus intended to ensure recruitment of participants with a broad range of identifiable deficits on all components of the assessment. Those unable to complete the study assessments and unable to speak English or French will be excluded from the study. Participant information will be documented, including demographics, functional status, environmental aids used, socioeconomic and work status, injury information, and information related to technology experience.

    Clinician Participants

    Clinicians with expertise in conducting assessments for people with acquired brain injuries will be invited to participate in this project. Demographic information for participating clinicians will be documented, including age, years practicing with patients who have an acquired brain injury, professional background (physiotherapist, physician assistant, physiatrist, other), and self-reports of competency with in-person and virtual neurological assessments.

    Sample Size

    The target sample size of this method-comparison study is 60 patient participants. For the objective of assessing the sensitivity of the virtual assessment against the gold standard of in-person assessment, with a total of 60 participants, 50 (83%) of whom are expected to have an abnormal finding, and an estimated sensitivity ranging from 77% to 96%, the two-sided 95% CI around the estimated sensitivity will have a total width ranging from 8.7% to 25.3% (or margin of error 4.4%-12.7%). For the objective of assessing interrater agreement between two clinicians conducting a virtual assessment using the κ statistic, a sample size of 60 participants achieves 80% power to detect a true κ value of 0.89 using a one-sided test at the 5% significance level, assuming that 90% of participants are classified as abnormal; κ under the null hypothesis was specified as 0.5.

    Recruitment and Consent

    Patient Participants

    Purposive sampling methods will be used to recruit individuals living with acquired brain injuries with impairments spanning all domains of interest. The participants will be recruited through the Ontario Workers Network and relevant rehabilitation clinics at the Ottawa Hospital. Electronic medical records will be screened to identify eligible patients, who will then be called or approached in person for consent.

    Clinician Participants

    Clinicians will be recruited from the Ontario Workers Network and relevant rehabilitation clinics at the Ottawa Hospital to participate as clinician assessors in the study. Consent will be obtained over the telephone or face to face.

    Outcomes

    The virtual toolkit of assessments will include the following outcome measures: the classification of abnormality (binary variable) on the finger-to-nose test, Vestibular/Ocular Motor Screening tool, balance testing, cervical spine range of motion, saccades testing, and evaluation of effort. The procedures outlined below provide details for these outcome measures, including the methods used for evaluating the sensitivity of the virtual assessment when compared to the in-person assessment and the interrater and intrarater reliabilities of the virtual assessment.

    Procedures

    Training

    Training of the assessing clinicians on the outcome measures will occur prior to commencement of patient-participant recruitment. A previously developed training manual [22] will be used to complete the training. Training will be uniform with the goal of standardizing clinician completion of the assessments.

    Assessments

    Data collection will occur in Research Electronic Data Capture (REDCap), a secure electronic data capture application. An assessing clinician will conduct two assessments: one in person and one virtual. Both assessments will be identical consisting of all measures from the virtual assessment toolkit (see Table 1). Both assessments will be completed at the Ottawa Hospital. When possible, assessments will occur on the same day with a brief rest period in between [23,24]. The order of assessment completion (in person vs virtual) will be randomized and counter-balanced. A random-numbers table will be generated and imported into REDCap and the randomization will occur through REDCap. In addition to the assessments identified in Table 1, clinicians will document adverse events (if any) and will record findings from the assessments on a paper copy, which will be directly input into REDCap by a research team member.

    For the virtual assessment, the patient participants will be located in a separate room from the assessing clinicians within the Ottawa Hospital. All patient participants will use the same computer and will be provided with technical support from a research team member as needed. The virtual assessments will be conducted in the clinical environment to standardize as many components of the assessment as possible, including computer screen, lighting, image size, and room setup. During the virtual assessment, several safety measures will be implemented: (1) a research team member will be present in the room throughout and (2) the patient participant will stand in front of a wall/bed/chair or with their back at a corner during the vestibular assessments in case of loss of balance. All the virtual assessments will occur through the Microsoft Teams platform. The study assessments will be audio/video-recorded using the record function on Microsoft Teams.

    Where possible, both assessments will be completed on the same day. It is estimated that a portion (~5%) of participants may be unable to complete both the in-person and virtual assessments on the same day, either due to lack of time or inability to tolerate a second assessment as a result of symptom aggravation (eg, headaches, dizziness, nausea, sensitivity to noise and light). For these participants, the second assessment (virtual or in person) will be scheduled for completion within 1 week of the initial assessment.

    Following completion of the assessments, the patient participants and clinician assessors will complete a feedback form. Participants will complete a feedback form (see Multimedia Appendix 1) related to completion of the virtual assessment, the environmental setup, and confidence in the assessment findings.

    Table 1. Outline of measures included in the virtual assessment toolkit to be completed in both in-person and virtual assessmentsa.
    DomainMeasuresDocumentation of findings by clinicians
    Neurological examinationCoordination: finger-to-nose testAbnormal (hesitation, tremor, over- or undershooting [25])/normal
    VestibularVOMSbChange in symptoms (≥2 point increase indicates abnormality [26]); average distance for 3 trials of near-point convergence in centimeters (≥5 cm convergence indicates abnormality [26])
    VestibularBalance (feet together, single-leg stance, tandem stance) with eyes open and eyes closed for 20 seconds eachAbnormal (unable to complete 20-second balance [22])/normal
    OculomotorSaccadesAbnormal (observe saccade initiation, range of motion and conjugacy, speed, accuracy, intrusions, or oscillations [27])/normal
    CervicalRange of motion (flexion, extension, right and left lateral flexion, right and left rotation)Estimated angles for each recorded abnormality (determined by comparison to norms in people aged 20-59 years: flexion=50°-72°, extension=58°-77°, lateral flexion=37°-47°, rotation=67°-81° [28])
    EffortRating scaleFull use of effort documented as yes or no (clinician discretion); rating scale documented from 0 (no effort) to 10 (maximum effort) (clinician discretion)

    aFor a complete presentation of the instructions used to administer the measures, see Johnston et al [22].

    bVOMS: Vestibular/Ocular Motor Screening.

    Observation and Rating of Audio-Video Recordings

    A second clinician (rater B in Figure 1 [17]) will complete two ratings of the recording of the virtual assessment (in a randomized order for participants) at 1-month intervals. Clinicians will record findings from the assessments on paper or electronic copies, which will be input directly into REDCap by a research team member. See Figure 1 [17] for an outline of study assessment procedures.

    ‎
    Figure 1. Assessment procedures (adapted from Russell et al [17]).

    Analysis

    Assessments Overview

    Patient and clinician participant characteristics will be descriptively analyzed. IBM SPSS (version 28) will be used to calculate the sensitivity of virtual assessments versus in-person assessments, along with the κ values (interrater and intrarater reliabilities) of the virtual assessments (see Figure 1 for comparisons).

    Method-Comparison (Sensitivity) Analysis

    The sensitivity of the virtual assessment compared to the gold-standard in-person assessment (rater A in person vs rater A virtual) will be determined, where the assessment of the uncertain classification of the measures used virtually will occur against the gold standard or “true” classification, which is the in-person assessment in this study. The findings of the initial clinician assessor (rater A) that completed the in-person and virtual assessments will be compared to determine the sensitivity of the measures administered virtually. In this case, sensitivity (true positives divided by the sum of true positives and false negatives multiplied by 100) refers to the ability of the virtual measure to identify a deficit or positive finding when the in-person measure identifies the deficit or positive finding. Values closer to 100% indicate greater sensitivity.

    Reliability

    To calculate the interrater reliability of the virtual assessment, the findings of rater A (virtual assessment) will be compared to the findings of rater B1 (initial observation of the virtual assessment recording). To determine the intrarater reliability of the virtual assessment, the findings of rater B1 (initial observation of the virtual assessment) will be compared to the findings of rater B2 (second observation of the virtual assessment) approximately 1 month after the initial observation. Estimates of reliability will be calculated using the κ statistic (classification of abnormal vs normal) for all measures included in the toolkit.

    Adverse Events

    Adverse events, including the type and severity, will be summarized by assessment type (in person or virtual).

    Ethical Considerations

    Ethics approval was obtained from the Ottawa Health Sciences Network Research Ethics Board (20230311-01H), the Bruyère Research Ethics Board (M16-22-006), and the University of Ottawa Board of Ethics (H-06-23-9348). All potential risks and adverse consequences will be relayed to participants verbally and all participants will provide verbal consent over the telephone or informed consent in person prior to their scheduled appointment. A wet-ink signature will be obtained from all participants on the day of their scheduled assessment. Privacy and confidentiality will be maintained by using deidentified data. Patient participants will be provided with a CAD $30 (~US $22) gift card and parking voucher as a token of appreciation for their participation in the study.


    As of May 2024, we enrolled and completed both the in-person and virtual assessments for 39 patient participants. We have recruited 7 clinician assessors. Recruitment and testing are expected to be completed by September 2024.


    Projected Significance

    This study will establish the reliability and sensitivity properties associated with 5 physical components of the virtual concussion assessment. There is an identified gap in current knowledge with limited research conducted on the reliability and comparability of virtual concussion assessments in comparison to in-person assessments. While virtual care has demonstrated to be a valuable approach to concussion assessment and management [29,30], there is a need for additional information regarding the psychometric properties of outcome measures. This information is essential for instilling confidence in the administration of these measures in a practical setting. Virtual approaches to care have been used for many years; however, the COVID-19 pandemic significantly increased their use in an effort to reduce hospital visits [15]. While the COVID-19 pandemic initiated this increased use of virtual care, some assessments are continuing to be offered virtually. Using measures with acceptable properties in a virtual context is important in practice to ensure accurate identification of problem areas on virtual assessments so that treatment interventions can be appropriately directed [31]. Despite measures showing adequate properties when administered in person, they may produce different results when administered in a virtual context [17,32], thereby providing a rationale to carry out this work. The aim of completing this method-comparison study is to obtain preliminary information on the virtual assessment in relation to the WSIB context.

    Strengths and Limitations of the Methodological Approach

    The strengths of this method-comparison approach include the use of commonly adopted technology and software for virtual brain injury assessments in a practical setting. Further, including a wide range of acquired brain injury severities in our sample ensures the inclusion of abnormalities spanning all relevant domains. This will make the findings not only applicable to current practice but also potentially generalizable to different types of acquired brain injury.

    We plan to assess the virtual evaluation in a controlled hospital setting. Moving forward, it would be beneficial to evaluate the virtual assessment in environments that are less standardized such as patients’ homes or community clinics where physician specialists are in a different location than the patient.

    Finally, due to constraints in clinic scheduling, a methodological decision was made to have the same clinician (rater A) conduct both the in-person and virtual assessments with the participants. In attempts to address this concern, the order of study assessments (in person and virtual) will be randomized and counter-balanced.

    Conclusion

    This method-comparison study is an important endeavor considering the limited exploration of virtual concussion assessments in the literature despite their increasing use in practice. This study aims to determine whether virtual and in-person assessments yield similar results in terms of the identification of abnormality in common physical domains of a concussion assessment. The study will also assess the interrater and intrarater reliabilities of the virtual assessment, contributing valuable insights to the field of concussion rehabilitation.

    Acknowledgments

    KB is supported by an admission scholarship from the University of Ottawa, the Ontario Graduate Scholarship, and the Early Professionals, Inspired Careers in AgeTech (EPIC-AT) Fellowship through AGE-WELL. This Project is funded by a grant provided by the Workplace Safety and Insurance Board (WSIB) of Ontario. The provision of grant support by the WSIB does not in any way infer or imply endorsement of the content by the WSIB. MK is supported by an admission scholarship, international doctoral scholarship; Special Merit Scholarship from the University of Ottawa; Hans K Uhthoff, MD FRCSC Graduate Fellowship; Ontario Physiotherapy Association Scholarship; and BMO Financial Group Graduate Scholarship.

    Authors' Contributions

    All authors have made substantial contributions to the work presented in this paper.

    Conflicts of Interest

    None declared.

    Multimedia Appendix 1

    Feedback forms.

    PDF File (Adobe PDF File), 180 KB
    Multimedia Appendix 2

    Peer-review reports from the Strategic Research - Workplace Safety and Insurance Board (WSIB) (Ontario, Canada).

    PDF File (Adobe PDF File), 164 KB
    1. Scorza KA, Raleigh MF, O'Connor FG. Current concepts in concussion: evaluation and management. Am Fam Physician. Jan 15, 2012;85(2):123-132. [FREE Full text] [Medline]
    2. Mullally WJ. Concussion. Am J Med. Aug 2017;130(8):885-892. [CrossRef] [Medline]
    3. Ventura R, Balcer L, Galetta S. The concussion toolbox: the role of vision in the assessment of concussion. Semin Neurol. Oct 6, 2015;35(5):599-606. [CrossRef] [Medline]
    4. Marshall S. Concussions Ontario. 2023. URL: https://concussionsontario.org [accessed 2024-06-28]
    5. Iverson GL, Gardner AJ, Terry DP, Ponsford JL, Sills AK, Broshek DK, et al. Predictors of clinical recovery from concussion: a systematic review. Br J Sports Med. Jun 31, 2017;51(12):941-948. [FREE Full text] [CrossRef] [Medline]
    6. Kontos A, Collins M. Concussion: A Clinical Profile Approach to Assessment and Treatment. Washington, DC. American Psychological Association; 2018.
    7. Greenberg MS, Wood NE, Spring JD, Gurvits TV, Nagurney JT, Zafonte RD, et al. Pilot study of neurological soft signs and depressive and postconcussive symptoms during recovery from mild traumatic brain injury (mTBI). J Neuropsychiatry Clin Neurosci. Jul 2015;27(3):199-205. [CrossRef] [Medline]
    8. Clugston J, Houck Z, Asken B, Boone J, Kontos A, Buckley T, et al. Relationship between the King-Devick test and commonly used concussion tests at baseline. J Athl Train. Dec 2019;54(12):1247-1253. [FREE Full text] [CrossRef] [Medline]
    9. Murray NG, Szekely B, Moran R, Ryan G, Powell D, Munkasy BA, et al. Concussion history associated with increased postural control deficits after subsequent injury. Physiol Meas. Feb 20, 2019;40(2):024001. [CrossRef] [Medline]
    10. Guskiewicz KM. Balance assessment in the management of sport-related concussion. Clin Sports Med. Jan 2011;30(1):89-102. [CrossRef] [Medline]
    11. Mollayeva T, Colantonio A. concUssion in ontario HealtH care WorKers: implications for prevention. OOHNA J. 2019;Spring/Summer 2019:20-28. [FREE Full text]
    12. Kristman VL, Côté P, Van Eerd D, Vidmar M, Rezai M, Hogg-Johnson S, et al. Prevalence of lost-time claims for mild traumatic brain injury in the working population: improving estimates using workers compensation databases. Brain Inj. Jan 03, 2008;22(1):51-59. [CrossRef] [Medline]
    13. Terry DP, Iverson GL, Panenka W, Colantonio A, Silverberg ND. Workplace and non-workplace mild traumatic brain injuries in an outpatient clinic sample: a case-control study. PLoS One. Jun 1, 2018;13(6):e0198128. [FREE Full text] [CrossRef] [Medline]
    14. Colantonio A, Mroczek D, Patel J, Lewko J, Fergenbaum J, Brison R. Examining occupational traumatic brain injury in Ontario. Can J Public Health. Mar 1, 2010;101(S1):S58-S62. [CrossRef]
    15. Womble M, Reynolds E, Kissinger-Knox A, Collins M, Kontos A, West R, et al. The emerging role of telehealth for concussion clinical care during the coronavirus (COVID-19) pandemic. J Head Trauma Rehabil. 2022;37(2):E49-E54. [CrossRef] [Medline]
    16. Montes J, Eichinger KJ, Pasternak A, Yochai C, Krosschell KJ. A post pandemic roadmap toward remote assessment for neuromuscular disorders: limitations and opportunities. Orphanet J Rare Dis. Jan 04, 2022;17(1):5. [FREE Full text] [CrossRef] [Medline]
    17. Russell TG, Martin-Khan M, Khan A, Wade V. Method-comparison studies in telehealth: study design and analysis considerations. J Telemed Telecare. Sep 11, 2017;23(9):797-802. [CrossRef]
    18. McLeod T, Leach C. Psychometric properties of self-report concussion scales and checklists. J Athl Train. 2012;47(2):221-223. [FREE Full text] [CrossRef] [Medline]
    19. Barnes K, Sveistrup H, Bayley M, Egan M, Rathbone M, Taljaard M, et al. Identification of clinical measures to use in a virtual concussion assessment: protocol for a mixed methods study. JMIR Res Protoc. Dec 22, 2022;11(12):e40446. [FREE Full text] [CrossRef] [Medline]
    20. Fuller G. Neurological Examination Made Easy E-Book. 6th edition. Amsterdam. Elsevier Health Sciences; 2019.
    21. Leddy JJ, Haider MN, Noble JM, Rieger B, Flanagan S, McPherson JI, et al. Clinical assessment of concussion and persistent post-concussive symptoms for neurologists. Curr Neurol Neurosci Rep. Nov 24, 2021;21(12):70. [CrossRef] [Medline]
    22. Johnston S, Leddy J, Nick RO, Cortel-LeBlanc A, Hafizi R, Laing S. Virtual Concussion Exam Training Manual. URL: https://cattonline.com/wp-content/uploads/2022/01/Virtual-Concussion-Exam-Manual.pdf [accessed 2024-06-28]
    23. Mani S, Sharma S, Singh DK. Concurrent validity and reliability of telerehabilitation-based physiotherapy assessment of cervical spine in adults with non-specific neck pain. J Telemed Telecare. Feb 2021;27(2):88-97. [CrossRef] [Medline]
    24. Stewart GW, McQueen-Borden E, Bell RA, Barr T, Juengling J. Comprehensive assessment and management of athletes with sport concussion. Int J Sports Phys Ther. Aug 2012;7(4):433-447. [FREE Full text] [Medline]
    25. Nishida K, Usami T, Matsumoto N, Nishikimi M, Takahashi K, Matsui S. The finger-to-nose test improved diagnosis of cerebrovascular events in patients presenting with isolated dizziness in the emergency department. Nagoya J Med Sci. Aug 2022;84(3):621-629. [FREE Full text] [CrossRef] [Medline]
    26. Mucha A, Collins MW, Elbin R, Furman JM, Troutman-Enseki C, DeWolf RM, et al. A brief Vestibular/Ocular Motor Screening (VOMS) assessment to evaluate concussions: preliminary findings. Am J Sports Med. Oct 08, 2014;42(10):2479-2486. [FREE Full text] [CrossRef] [Medline]
    27. Termsarasab P, Thammongkolchai T, Rucker JC, Frucht SJ. The diagnostic value of saccades in movement disorder patients: a practical guide and review. J Clin Mov Disord. Oct 15, 2015;2(1):14. [FREE Full text] [CrossRef] [Medline]
    28. Swinkels RAHM, Swinkels-Meewisse IEJCM. Normal values for cervical range of motion. Spine. Mar 01, 2014;39(5):362-367. [CrossRef] [Medline]
    29. Barnes K, Sveistrup H, Karimijashni M, Bayley M, Egan M, Bilodeau M, et al. Barriers and facilitators associated with virtual concussion physical assessments from the perspectives of clinicians and people living with workplace concussions. JMIR Preprints. Preprint posted online Jan 9, 2024 [CrossRef]
    30. van Ierssel J, O'Neil J, King J, Zemek R, Sveistrup H. Clinician perspectives on providing concussion assessment and management via telehealth: a mixed-methods study. J Head Trauma Rehabil. 2023;38(3):E233-E243. [CrossRef] [Medline]
    31. Fawcett A. Principles of Assessment and Outcome Measurement for Occupational Therapists and Physiotherapists: Theory, Skills and Application. Hoboken, NJ. John Wiley & Sons; 2013.
    32. O'Neil J, Barnes K, Morgan Donnelly E, Sheehy L, Sveistrup H. Identification and description of telerehabilitation assessments for individuals with neurological conditions: a scoping review. Digit Health. Jun 22, 2023;9:20552076231183233. [FREE Full text] [CrossRef] [Medline]

    ‎
    REDCap: Research Electronic Data Capture
    WSIB: Workplace Safety and Insurance Board

    Edited by D Khajeei; The proposal for this study was externally peer-reviewed by the Strategic Research - Workplace Safety and Insurance Board (WSIB) (Ontario, Canada). See the Multimedia Appendix for the peer-review report; submitted 04.03.24; accepted 14.06.24; published 26.07.24.

    Copyright

    ©Keely Barnes, Heidi Sveistrup, Mark Bayley, Michel Rathbone, Monica Taljaard, Mary Egan, Martin Bilodeau, Motahareh Karimijashni, Shawn Marshall. Originally published in JMIR Research Protocols (https://www.researchprotocols.org), 26.07.2024.

    This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Research Protocols, is properly cited. The complete bibliographic information, a link to the original publication on https://www.researchprotocols.org, as well as this copyright and license information must be included.