Reliability, Sensitivity, and Specificity Measures of the Comprehensive Assessment of Accentedness and Intelligibility (CAAI) Test Battery

Authors

  • Amee P. Shah School of Health Sciences, Stockton University, Galloway, New Jersey

DOI:

https://doi.org/10.14738/assrj.115.17016

Abstract

Purpose: This paper aims to introduce and report on measures of validation of a diagnostic assessment, the Comprehensive Assessment of Accentedness and Intelligibility (CAAI) Test Battery that identifies communication issues related to a person’s spoken accent. Due to scant availability of standardized assessments of accents, this test battery was developed to enable evidence-based diagnoses and determine severity levels of dialect- and accent-related communication difficulties (Shah, 2024). The present paper reports on psychometric testing conducted on the test battery with the resulting measures of reliability, sensitivity, and specificity of the CAAI. Methods: Native and nonnative speakers of English (N = 61) were administered the CAAI Test Battery, and their communication was evaluated and scored. Descriptive statistics were used to examine test sensitivity, specificity, interrater reliability, inter-item (internal consistency) reliability, and test-retest reliability. Results: The CAAI Test Battery meets high standards for test sensitivity in identifying accent-related communication concerns. Test specificity was high for not falsely identifying native speakers as having accent-related concerns. The instrument had strong interrater reliability across all 20 sections of the test (Pearson correlation coefficients ranged from 0.68 to 1.00, all highly significant at p<0.01). The test showed strong inter-item reliability/internal consistency with a large number of sections yielding moderate to high Cronbach’s alpha coefficient range of 0.70 to 1.00. The test showed strong test-retest reliability with correlation coefficients ranging from 0.73 to 1.00, significant at p<0.05. Conclusions: With high sensitivity, specificity, and reliability, the CAAI Test Battery is found to be a stable and meaningful means to assess dialect- and accent-related communication concerns. The CAAI Test Battery helps fill a crucial gap in the area of accents/dialects that lacks other validated assessment measures. With a sensitive and reliable assessment, clinicians and teachers can identify an accurate baseline pattern of errors to address for accent management or pronunciation teaching and achieve effective outcomes in a short amount of time.

Downloads

Published

2024-05-31

How to Cite

Shah, A. P. (2024). Reliability, Sensitivity, and Specificity Measures of the Comprehensive Assessment of Accentedness and Intelligibility (CAAI) Test Battery. Advances in Social Sciences Research Journal, 11(5), 318–334. https://doi.org/10.14738/assrj.115.17016