A list of papers on automatic speech scoring.

ASR

Feature extraction

Grader (scoring model)

• Impact of ASR Performance on Free Speaking Language Assessment, K.M.Knill, Interspeech 2018 ⭐⭐⭐

This paper considered the impact of ASR performance on assessment of free speaking tests of non-native learners of English. Comparison of 2 state-of-the-art ASR systems with a significant difference in WER showed that the baseline auto-marking system based on audio and fluency related features was unaffected by the reduction in WER due to the system having been designed to mitigate ASR errors. The impact that ASR errors may have on the ability of the system to provide detailed feedback to the learner was also analysed.

• Neural Approaches to Automated Speech Scoring of Monologue and Dialogue Responses, Yao Qian et al, Interspeech 2019⭐⭐

This paper presented Neural Network (NN) approaches (attention, BLSTM-RNN, encoder, MemN2N, and conditional inputs) to the automated assessment of non-native spontaneous speech in a monologic task and a simulated dialogic task. Three attention-based Bidirectional Long Short-Term Memory (BLSTM) Recurrent Neural Networks (RNN) are employed to learn three dimensions (i.e., delivery, language use, and content) of scoring rubrics for the spoken responses. The three subscores are fused together to generate a holistic score. The experimental results show that the NN approaches significantly outperform the conventional approaches to speech scoring and the correlations of automatically predicted scores with the reference human scores are higher than human-human agreement levels for both tasks.

• A Comparison of ASR and Human Errors for Transcription of Non-native Spontaneous Speech, Matthew Mulholland et al, ICASSP 2016⭐

This paper compares ASR and human transcriptions of non-native speech to investigate to what extent the accuracy and the patterns of errors of a modern ASR system match those of human listeners in the context of automated assessment of L2 English language proficiency (how different are the patterns of errors made by human and the ASR system?). There is evidence the same words may present difficulties for both ASR and human transcribers.

• Bidirectional LSTM-RNN for Improving Automated Assessment Of Non-native Children’s Speech, Yao Qian et al, Interspeech 2017⭐⭐

This paper investigated different neural network architectures for improving non-native children's speech recognition and the impact of the features extracted from the corresponding ASR output on the automated assessment of speaking proficiency. Experimental results show that bidirectional LSTM-RNN can outperform feed-forward DNN in ASR and the improved speech recognition can then boost the language proficiency assessment performance.

• Automatic Text Scoring Using Neural Networks, 2016
• Investigating neural architectures for short answer scoring, 2017
• Automatic scoring of non-native spontaneous speech in tests of spoken English, 2009
• A three-stage approach to the automated scoring of spontaneous spoken responses, 2010
• Exploring Content Features for Automated Speech Scoring, 2012
• Using Deep Neural Networks to Improve Proficiency Assessment for Children English Language Learners, Interspeech 2014
• Self-Adaptive DNN for Improving Spoken Language Proficiency Assessment, Interspeech 2016
• Towards Automatic Assessment of Spontaneous Spoken English, 2017
• USING BIDIRECTIONAL LSTM RECURRENT NEURAL NETWORKS TO LEARN HIGH-LEVEL ABSTRACTIONS OF SEQUENTIAL FEATURES FOR AUTOMATED SCORING OF NON-NATIVE SPONTANEOUS SPEECH ASRU 2015
• Automatic Assessment of the Speech of Young English Learners, Jian Cheng et al, ACL 2014
• Validating automated speaking tests, Jared Bernstein et al, 2010
• Automatically Grading Learners’ English Using a Gaussian Process, Rogier C. van Dalen et al, 2015