A Study on Facial Emotion Based Song Recommendations Systems

Abstract

With the rapid growth of intelligent systems and personalized user experiences, emotion-aware applications have gained significant attention. This project presents a Facial Emotion-Based Song Recommendation System that automatically detects a user\'s emotional state through facial expressions and recommends music accordingly. The system utilizes computer vision techniques and deep learning models to analyze real-time facial inputs captured via a webcam. The proposed model employs algorithms based on Computer Vision and Deep Learning, particularly Convolutional Neural Networks (CNNs), to classify emotions such as happiness, sadness, anger, surprise, fear, and neutrality. Once the emotion is identified, the system maps it to a curated music database and suggests songs that align with the detected mood, enhancing user engagement and emotional well-being. The system integrates facial detection frameworks like OpenCV and machine learning libraries such as TensorFlow or Keres for model training and deployment. The recommendation engine may use content-based filtering or emotion-tagged playlists to provide relevant suggestions. This approach demonstrates how affective computing can be leveraged to create adaptive and intuitive music recommendation systems. The implementation aims to improve user satisfaction by delivering a seamless and personalized music experience based on real-time emotional analysis.

Introduction

The text describes a Facial Emotion-Based Song Recommendation System that uses AI to suggest music based on a user’s real-time facial expressions.

Traditional music recommendation systems rely on listening history and ratings, but they often fail to capture a user’s current emotional state. To solve this, the proposed system uses affective computing, computer vision, and deep learning to detect emotions from facial expressions and recommend suitable songs accordingly.

The system captures a user’s face through a webcam, processes it using OpenCV, and uses a CNN-based deep learning model to classify emotions such as happy, sad, angry, surprise, fear, and neutral. Based on the detected emotion, it selects songs from a playlist or external platforms like Spotify or YouTube that match the user’s mood.

The system has clear objectives: real-time face capture, facial feature extraction, and accurate emotion classification using deep learning.

The literature review shows that similar systems use CNNs, LSTM models, valence-arousal mapping, and APIs for emotion detection and music recommendation. Some also consider temporal mood changes or combine facial and musical emotion matching to improve accuracy.

The system architecture includes:

• A frontend (HTML, CSS, JavaScript) for user interaction
• A backend (Python Flask) for processing
• OpenCV for face detection
• A CNN model for emotion recognition
• A recommendation engine that maps emotions to songs
• A database (SQLite) for storing categorized music data
• Optional integration with music platforms like Spotify

Conclusion

The Facial Emotion-Based Song Recommendation System presents an intelligent and user-centric approach to music recommendation by integrating computer vision and deep learning techniques. The system successfully captures real-time facial expressions and analyzes them using models such as DeepFace to identify user emotions accurately. Based on the detected emotion, the system recommends appropriate songs, thereby enhancing personalization and improving overall user experience. Unlike traditional systems that rely on user history, this approach focuses on real-time emotional states, making the recommendations more relevant and dynamic. The system demonstrates the practical application of affective computing by bridging the gap between human emotions and automated digital services. Although there are certain limitations such as dependency on environmental conditions and accuracy challenges, the proposed system provides a strong foundation for future improvements. In conclusion, the project highlights the potential of combining Artificial Intelligence, computer vision, and real-time processing to develop smarter and more adaptive entertainment systems.

References

[1] Tripathi et al., “Facial Emotion-Based Song Recommender Using CNN,” 2024. [2] H. Nguyen et al., “Song Recommendation via Facial & Musical Emotion,” 2024. [3] A. Kahan et al., “Facial Emotion-Based Song Recommendation,” 2022. [4] P. Pardhi et al., “Emotion-Based Music Recommendation Using ML & AI,” 2024. [5] J. Malhotra et al., “Emotion-Based Music Recommendation Using LSTM,” 2024.

Copyright

Copyright © 2026 Prof. Asha Gaikar, Vaishnavi Khaire, Snehal Gorad, Payal Chavan, Sambodhi Gaikwad. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.