Reliability, Sensitivity, and Specificity Measures of the Comprehensive Assessment of Accentedness and Intelligibility (CAAI) Test Battery
DOI:
https://doi.org/10.14738/assrj.115.17016Abstract
Purpose: This paper aims to introduce and report on measures of validation of a diagnostic assessment, the Comprehensive Assessment of Accentedness and Intelligibility (CAAI) Test Battery that identifies communication issues related to a person’s spoken accent. Due to scant availability of standardized assessments of accents, this test battery was developed to enable evidence-based diagnoses and determine severity levels of dialect- and accent-related communication difficulties (Shah, 2024). The present paper reports on psychometric testing conducted on the test battery with the resulting measures of reliability, sensitivity, and specificity of the CAAI. Methods: Native and nonnative speakers of English (N = 61) were administered the CAAI Test Battery, and their communication was evaluated and scored. Descriptive statistics were used to examine test sensitivity, specificity, interrater reliability, inter-item (internal consistency) reliability, and test-retest reliability. Results: The CAAI Test Battery meets high standards for test sensitivity in identifying accent-related communication concerns. Test specificity was high for not falsely identifying native speakers as having accent-related concerns. The instrument had strong interrater reliability across all 20 sections of the test (Pearson correlation coefficients ranged from 0.68 to 1.00, all highly significant at p<0.01). The test showed strong inter-item reliability/internal consistency with a large number of sections yielding moderate to high Cronbach’s alpha coefficient range of 0.70 to 1.00. The test showed strong test-retest reliability with correlation coefficients ranging from 0.73 to 1.00, significant at p<0.05. Conclusions: With high sensitivity, specificity, and reliability, the CAAI Test Battery is found to be a stable and meaningful means to assess dialect- and accent-related communication concerns. The CAAI Test Battery helps fill a crucial gap in the area of accents/dialects that lacks other validated assessment measures. With a sensitive and reliable assessment, clinicians and teachers can identify an accurate baseline pattern of errors to address for accent management or pronunciation teaching and achieve effective outcomes in a short amount of time.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2024 Amee P. Shah
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors wishing to include figures, tables, or text passages that have already been published elsewhere are required to obtain permission from the copyright owner(s) for both the print and online format and to include evidence that such permission has been granted when submitting their papers. Any material received without such evidence will be assumed to originate from the authors.