Regional Indian English Accent Identification via Hybrid Acoustic-Contextual Speech Representations and Hierarchical HuBERT Layer Probing

Abstract

Regional Indian English exhibits intricate acoustic, phonological, and prosodic variations shaped by speakers\' native language (L1) backgrounds. Automated identification of these regional accents is essential for accent-adaptive automatic speech recognition (ASR), spoken language identification, and culturally nuanced voice interfaces. This paper presents a hybrid speech representation framework for six-class regional Indian English accent identification using the IndicAccentDB corpus (8,116 recordings across Andhra/Telugu, Gujarat/Gujarati, Jharkhand/Eastern Hindi, Karnataka/Kannada, Kerala/Malayalam, and Tamil). The proposed pipeline fuses low-level spectral summary statistics derived from 13 Mel-Frequency Cepstral Coefficients (MFCCs; 26 dimensions capturing temporal mean and standard deviation) with deep contextual embeddings extracted from the frozen, pretrained HuBERT-Base encoder (768 dimensions via global temporal average pooling), yielding a 794-dimensional hybrid representation. The fused representations are evaluated using multinomial Logistic Regression with L? regularization and a lightweight one-dimensional Convolutional Neural Network (1D CNN). Under a stratified 80/20 utterance-level evaluation protocol, the framework achieves 90.33% test accuracy with Logistic Regression and 90.83% with the 1D CNN. A complementary layer-wise probe across all 13 HuBERT representations on a 500-utterance diagnostic subset reveals that shallow layers (Layers 1–2) yield peak linear separability of 94.00%, consistent with low-level acoustic and vocal tract encoding, before declining to 88%–91% in deeper contextual layers. Temporal duration analysis shows a 25.48% accuracy drop when restricting audio to 0.7-second sub-utterance clips. The trained classifier is deployed in an interactive Gradio application coupling regional accent identification with a cultural culinary recommendation interface. We provide a thorough discussion of evaluation scope, including the distinction between utterance-level predictability and speaker-disjoint generalization.

Introduction

This study proposes a hybrid speech recognition framework for identifying six regional Indian English accents using acoustic features and self-supervised learning. Indian English varies across regions due to the influence of speakers’ first languages and local linguistic environments. Identifying these accent variations is important for improving automatic speech recognition (ASR), language learning systems, voice assistants, and other speech-based technologies.

Traditional accent classification methods rely on handcrafted acoustic features, such as Mel-Frequency Cepstral Coefficients (MFCCs), pitch, and formant frequencies. Although these features capture important speech characteristics, they have limited ability to represent complex temporal and phonetic patterns. Self-supervised learning models such as HuBERT and Wav2Vec 2.0 can learn richer speech representations from audio, but using their full networks for training can require substantial computational resources.

To address these limitations, the study introduces a hybrid feature extraction and classification approach that combines two types of speech representations:

• MFCC statistical features: A 26-dimensional vector containing the means and standard deviations of 13 MFCC coefficients, representing acoustic and spectral characteristics.

• HuBERT embeddings: A 768-dimensional representation extracted from a frozen pretrained HuBERT-Base model using temporal average pooling.

These features are concatenated to create a 794-dimensional hybrid representation. Two classifiers are then evaluated: multinomial logistic regression, which provides a linear baseline, and a one-dimensional convolutional neural network (1D CNN), which captures nonlinear patterns.

The experiments use the IndicAccentDB dataset, containing 8,116 audio recordings across six regional Indian English accent categories: Tamil, Andhra (Telugu), Karnataka (Kannada), Kerala (Malayalam), Jharkhand (Eastern Hindi/Bihari), and Gujarat (Gujarati). The recordings are standardized to mono audio at 16 kHz, and silence is trimmed before feature extraction. The dataset is divided into 80% training data and 20% testing data using a stratified split.

The dataset is imbalanced, with the four southern Indian accent categories accounting for approximately 86% of the recordings, while the Gujarat category represents only 3.67%. This imbalance may affect classification performance, particularly for underrepresented accents. Additionally, because the evaluation uses an utterance-level split without ensuring that speakers are separated between training and testing sets, the results may not fully reflect generalization to previously unseen speakers.

The study also includes layer-wise diagnostic probing to investigate how different HuBERT layers represent accent-related information, duration sensitivity analysis to examine the effect of recording length, and deployment of the classifier through an interactive Gradio application.

Conclusion

This paper presented a hybrid speech representation framework fusing 26-dimensional MFCC temporal statistics with 768-dimensional HuBERT-Base contextual embeddings into a 794-dimensional representation for six-class regional Indian English accent identification. Under stratified evaluation on 8,116 IndicAccentDB utterances, the framework achieves 90.33% accuracy with multinomial Logistic Regression and 90.83% with a 1D CNN. Hierarchical layer-wise probing across 13 HuBERT representations indicates that shallow layers (Layers 1–2) yield a peak linear separability of 94.00% by capturing low-level vocal tract and acoustic cues, before attenuating in deeper contextual layers. Temporal duration analysis demonstrates a 25.48% accuracy degradation on 0.7-second clips, confirming that regional accent discrimination relies critically on suprasegmental acoustic-prosodic context. Future work will focus on speaker-disjoint evaluation on expanded corpora [3, 13], multi-layer HuBERT representation aggregation [12], parameter-efficient fine-tuning with LoRA [6], and training on larger, speaker-diverse datasets.

References

[1] S. Gaikwad, B. Gawali, and K. V. Kale, \"Accent recognition for Indian English using acoustic feature approach,\" International Journal of Computer Applications, vol. 63, no. 7, pp. 25–32, 2013. [2] I. Patel, R. Kulkarni, and Y. S. Rao, \"Automatic non-native dialect and accent voice detection of South Indian English,\" Advances in Image and Video Processing, vol. 5, no. 1, pp. 15–26, 2017. DOI: 10.14738/aivp.51.2749. [3] A. Ahamad, A. Anand, and P. Bhargava, \"AccentDB: A database of non-native English accents to assist neural speech recognition,\" in Proc. 12th Language Resources and Evaluation Conference (LREC), Marseille, France, 2020, pp. 5353–5360. [4] S. Darshana, H. Theivaprakasham, G. Jyothish Lal, B. Premjith, V. Sowmya, and K. P. Soman, \"MARS: A hybrid deep CNN-based multi-accent recognition system for English language,\" in Proc. 1st International Conference on Artificial Intelligence Trends and Pattern Recognition (ICAITPR), IEEE, 2022, pp. 1–6. DOI: 10.1109/ICAITPR51569.2022.9844177. [5] S. Darshana, \"IndicAccentDb,\" Hugging Face Datasets, Repository ID: DarshanaS/IndicAccentDb, 2022. [Online]. Available: https://huggingface.co/datasets/DarshanaS/IndicAccentDb. [6] R. Jairam, G. Jyothish Lal, and B. Premjith, \"A few-shot multi-accented speech classification for Indian languages using transformers and LLM\'s fine-tuning approaches,\" in Proc. 4th Workshop on Speech, Vision, and Language Technologies for Dravidian Languages (DravidianLangTech), Malta: ACL Anthology, 2024, pp. 1–7. DOI: 10.18653/v1/2024.dravidianlangtech-1.1. [7] W.-N. Hsu, B. Bolte, Y.-H. H. Tsai, K. Lakhotia, R. Salakhutdinov, and A. Mohamed, \"HuBERT: Self-supervised speech representation learning by masked prediction of hidden units,\" IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 29, pp. 3451–3460, 2021. [8] A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, \"wav2vec 2.0: A framework for self-supervised learning of speech representations,\" in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020, pp. 12449–12460. [9] S. Chen et al., \"WavLM: Large-scale self-supervised pre-training for full stack speech processing,\" IEEE Journal of Selected Topics in Signal Processing, vol. 16, no. 6, pp. 1505–1518, 2022. [10] A. Pasad, J.-C. Chou, and K. Livescu, \"Layer-wise analysis of a self-supervised speech representation model,\" in Proc. IEEE ASRU Workshop, 2021, pp. 914–921. [11] A. Pasad, B. Bowen, and K. Livescu, \"Comparative layer-wise analysis of self-supervised speech models,\" IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 31, pp. 3855–3868, 2023. [12] S.-W. Yang et al., \"SUPERB: Speech processing universal performance benchmark,\" in Proc. INTERSPEECH, 2021, pp. 1194–1198. [13] G. Zhao et al., \"L2-ARCTIC: A non-native English speech corpus for accented speech research,\" in Proc. INTERSPEECH, Hyderabad, India, 2018, pp. 2783–2787. [14] K. Prahallad et al., \"The IIIT-H Indic speech databases,\" in Proc. INTERSPEECH, Portland, OR, USA, 2012, pp. 2546–2549. [15] S. Chopra, A. Sharma, and S. Sakti, \"Svarah: Evaluating Indian English speech recognition across diverse accents and domains,\" in Proc. INTERSPEECH, Dublin, Ireland, 2023, pp. 2018–2022. [16] A. Abid et al., \"Gradio: Hassle-free sharing and testing of ML models in the wild,\" arXiv:1906.02569, 2019.

Copyright

Copyright © 2026 Shivapuja Sai Kiran, Varsha Thakur, Sompalli Naveen Kumar, Vadeghar Nitin, Shivakumar Nethani. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.