MIMIX-AI Interview Analyzer

Abstract

Interviews are essential for assessing a candidate\'s communication abilities, technical expertise, and behavioral characteristics. Nonetheless, manual assessment frequently results in bias, variability, and restricted scalability. To tackle these issues, this article suggests an AI-based Interview Analyzer that integrates Natural Language Processing (NLP), voice stress evaluation, and resume–answer comparison to generate objective, systematic, and data-informed interview evaluations. The system evaluates textual replies through TF-IDF vectorization, cosine similarity, grammar assessment, and keyword identification. At the same time, audio replies are transcribed via OpenAI Whisper and analysed with librosa/pyAudioAnalysis to obtain MFCCs, pitch variation, jitter, energy levels, and various prosodic characteristics to identify stress and confidence. Resume–answer alignment verifies authenticity by evaluating the skills listed on resumes against candidate answers through TF-IDF similarity analysis. Scores from NLP, audio evaluation, and resume verification are combined to produce a final interview score. The Gemini API is utilized to create individualized feedback according to scoring trends. Experimental findings indicate a significant relationship between scores produced by the system and ratings from human evaluators, with a discrepancy of under 10%. The suggested system offers a scalable, impartial, and automated approach to interview evaluation ideal for universities, training platforms, and hiring systems

Introduction

Interviews remain vital in academic and recruitment evaluations, assessing communication, confidence, and overall candidate fit. Traditional interviews, however, are subjective, inconsistent, and resource-intensive, especially for large-scale recruitment. Human biases, variability in judgment, and manual effort limit fairness and efficiency.

Advances in AI, NLP, and speech processing enable automated, multi-modal evaluation systems that analyze linguistic content, emotional cues, and resume alignment. The AI Interview Analyzer leverages these technologies to provide objective, scalable, and comprehensive interview assessments.

Objectives

1. Match candidate responses to resumes using TF-IDF to verify claimed skills.

2. Create a weighted scoring system combining text, audio, and resume data.

3. Generate automated feedback using rule-based logic and Gemini API.

4. Provide a scalable, impartial system that standardizes interview evaluation.

Limitations of Existing Systems

• Rely on human evaluators → subjective and inconsistent.

• Lack deep linguistic, semantic, or audio analysis.

• Fail to verify resume claims against candidate answers.

• Provide generic feedback instead of actionable insights.

Proposed Solution

The AI Interview Analyzer is a multi-modal system integrating:

1. Text Analysis:

   • TF-IDF vectorization for semantic relevance.

   • Grammar evaluation via Language Tool.

   • Keyword extraction to check coverage of essential concepts.

2. Audio Analysis:

   • Audio preprocessing (noise reduction, normalization).

   • Transcription using OpenAI Whisper.

   • Extraction of acoustic features (MFCCs, pitch, jitter, energy) via Librosa and pyAudioAnalysis.

   • Machine learning classifiers detect confidence or stress levels.

3. Resume-Answer Matching:

   • TF-IDF comparison of resume and candidate responses.

   • Ensures honesty and consistency of claimed skills.

4. Scoring & Feedback:

   • Weighted scoring across text, audio, and resume modules.

   • Rule-based feedback highlights strengths and improvement areas.

Methodology

• Text: Tokenization, stopword removal, lemmatization, TF-IDF, grammar check, keyword extraction.

• Audio: Feature extraction, stress/confidence classification.

• Resume: Cosine similarity to evaluate alignment with responses.

• Final Output: Unified score + structured feedback on linguistic clarity, confidence, semantic relevance, and honesty.

Implementation

• Frontend: React.js with Tailwind CSS for responsive, interactive UI; audio recording and submission supported.

• Backend: FastAPI handles NLP/audio processing, scoring, and database interactions; SVM/Logistic Regression used for stress classification.

• NLP Module: Preprocessing, TF-IDF vectorization, grammar evaluation, keyword extraction.

• Speech Module: Whisper transcription, Librosa/pyAudioAnalysis feature extraction, ML-based stress detection.

• Database: SQLite stores user data, responses, transcripts, features, and scores locally, supporting offline, academic-friendly deployment.

Key Takeaway:
The AI Interview Analyzer provides a holistic, automated, and unbiased approach to interview evaluation, combining linguistic analysis, emotional assessment, and resume verification. It addresses limitations of conventional interviews by delivering scalable, consistent, and actionable insights into candidate performance.

If you want, I can also make a visual flowchart showing the multi-modal

Conclusion

The AI Interview Analyzer presented in this work offers a complete and automated framework for assessing interview responses through textual, acoustic, and resume-based analysis. By integrating efficient NLP techniques, dependable speech-processing workflows, and machine learning classifiers, the system is capable of evaluating grammatical quality, semantic depth, confidence indicators, and factual consistency with notable precision. Whisper-driven transcription and MFCC-based stress analysis allow the system to capture subtle vocal cues that traditional interviewers may overlook. The resume-matching feature further strengthens the evaluation process by validating the authenticity of a candidate’s claimed skills. Experimental findings show that the system’s scoring aligns closely with human judgments while being free from bias and evaluator variability.

References

[1] S. Bird, E. Klein and E. Loper, Natural Language Processing with Python, O’Reilly Media, 2009. [2] F. Pedregosa et al., “Scikit-learn: Machine Learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825– 2830, 2011. [3] M. Abadi et al., “Whisper: Robust Speech Recognition via Large-Scale Weak Supervision,” OpenAI, 2022. [4] B. McFee et al., “librosa: Audio and Music Signal Analysis in Python,” in Proc. SciPy Conference, 2015. [5] T. Giannakopoulos, “pyAudioAnalysis: An Open-Source Python Library for Audio Analysis,” PLOS ONE, vol. 10, no. 12, pp. 1–17, 2015. [6] C. Cortes and V. Vapnik, “Support Vector Networks,” Machine Learning, vol. 20, pp. 273–297, 1995. [7] R. Tomlinson, “RAKE: Rapid Automatic Keyword Extraction,” 2010. [8] LanguageTool Developers, “LanguageTool: Open-Source Grammar Checker,” 2023. [9] Google Cloud, “Speech-to-Text API Documentation,” Google Developers, 2023. [10] SQLite Consortium, “SQLite Documentation,” 2023. [Online]. Available: https://sqlite.org [11] FastAPI Documentation, “FastAPI: High-Performance Web Framework,” 2023. [12] React Team, “React Documentation,” Meta, 2023. [Online]. Available: https://reactjs.org [13] Shadcn, “Shadcn UI Component Library,” 2023. [Online]. Available: https://ui.shadcn.com [14] Gemini Team, “Google Gemini API,” Google DeepMind, 2024. [Online]. Available: https://ai.google.dev

Copyright

Copyright © 2025 Nethra S., Ananya K C, Poorvi S. Hebbal, Spoorthi M. V., Sujana K. G.. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.