Explainable Movie Recommendation Framework Using Collaborative Filtering, Clustering, and Generative AI-Based

Abstract

Recommender systems have become a core part of how people discover content online, yet most still fall short in one important area: they tell you what to watch, but not why. This paper describes a movie recommendation and rating analysis platform we built to address that gap. The system combines collaborative filtering, K-Means clustering, and generative AI to produce recommendations that come with clear, human-readable explanations. On the infrastructure side, we use PySpark for batch model training, AWS Lambda to handle backend requests without managing servers, and S3 for storing precomputed data at scale. The most distinctive part of the system is what we call the Collaborative Explainer Agent — a multi-agent component that goes beyond ranking movies and instead identifies latent patterns in how users engage with films, covering aspects like narrative structure, visual style, and emotional tone. User taste profiles are kept current through exponential moving averages applied to interaction embeddings, which means the system adapts as preferences shift rather than waiting for a scheduled retraining cycle. We tested the platform on 100,000 ratings spanning 9,742 movies from the MovieLens dataset. The collaborative filtering model reached an RMSE of 0.8723, and silhouette analysis pointed to 40 as the best cluster count. Taken together, the system manages to balance prediction quality, transparency, and real-time responsiveness — three goals that are rarely achieved at once in production recommender systems.

Introduction

The text describes the development of an AI-powered movie analytics and recommendation platform designed to address the growing problem of content overload on streaming services. Traditional recommendation systems often provide suggestions without explaining why a movie was recommended, making them difficult for users to trust or understand. Existing methods such as content-based filtering rely heavily on simple genre tags, while collaborative filtering identifies behavioral patterns but functions largely as a black box. The proposed system aims to improve both recommendation accuracy and explainability.

The platform combines collaborative filtering, clustering, generative AI, and a serverless cloud architecture. Its core objective is to process large-scale movie rating data, generate personalized recommendations, and explain those recommendations in human-readable language. The backend is designed using serverless AWS technologies such as Lambda, DynamoDB, and S3 to minimize infrastructure management, while PySpark is used for distributed machine learning and data processing.

The system architecture consists of three main layers:

1. Offline ML Layer – PySpark performs collaborative filtering using ALS (Alternating Least Squares) on the MovieLens 100K dataset to generate 50-dimensional user and movie embeddings. K-Means clustering groups movies into thematic clusters, and generative AI labels these clusters with meaningful cinematic themes.
2. Cloud Backend Layer – AWS Lambda functions handle authentication, recommendations, interaction tracking, onboarding, and taste profiling. DynamoDB stores user embeddings and preferences, while S3 stores precomputed movie vectors.
3. Frontend Layer – A lightweight HTML/CSS/JavaScript interface allows users to interact with recommendations efficiently without heavy API dependency.

A key contribution of the system is the Collaborative Explainer Agent, which organizes recommendations into understandable thematic groups such as “Slow-Burn Character Studies” or “Visual Poetry Sci-Fi.” Instead of simply recommending movies, the system explains why a user may enjoy them based on viewing history and inferred cinematic preferences.

The recommendation engine also supports real-time personalization. User embeddings are updated dynamically using exponential moving averages whenever new interactions occur, ensuring recommendations reflect both long-term interests and recent behavior. The use of generative AI (Groq’s Llama 3.3 70B model) further improves explainability by generating descriptive cluster labels and summaries that go beyond traditional genre-based categorization.

Conclusion

This paper has described a movie recommendation and rating analysis system built around four key ideas: a hybrid vector scoring engine that blends long-term embeddings with time-decayed recent interactions; GenAI-driven cluster discovery that produces descriptive thematic labels rather than generic genre tags; a Collaborative Explainer Agent that structures recommendations around latent cinematic patterns and explains them in plain language; and a serverless AWS deployment that keeps the system scalable and operationally straightforward. Each of these components addresses a specific shortcoming of existing approaches — accuracy without explainability, personalization without real-time adaptation, or sophisticated ML without practical deployment. There are several natural directions to extend this work. Incorporating multi-modal embeddings — pulling in poster images, trailer clips, or audio — would give the model richer signals to work with and could improve both cluster quality and explanation depth. Adding temporal awareness to the recommendation logic, so that suggestions shift with seasonal viewing trends, is another promising direction. Further out, the embedding infrastructure could support cross-domain transfer, recommending music playlists or books based on a user’s film preferences. The modular design of the system means none of these extensions would require rebuilding the core architecture from scratch.

References

[1] Koren, Y., Bell, R., & Volinsky, C. (2009). Matrix Factorization Techniques for Recommender Systems. IEEE Computer, 42(8), 30-37. [2] Mnih, A., & Salakhutdinov, R. (2008). Probabilistic Matrix Factorization. NIPS 2007. [3] Kunaver, M., & Porl, T. (2017). Diversity in Recommender Systems. Journal of Electronic Commerce Research, 18(2), 68-93. [4] Zaharia, M., et al. (2016). Apache Spark: A Unified Engine for Big Data Processing. Communications of the ACM, 59(11), 56-65. [5] Huang, X., et al. (2020). Towards a Better Match in Semantic-Enhanced Information Retrieval. SIGIR 2020. [6] Brown, T., et al. (2020). Language Models are Few-Shot Learners. NeurIPS 2020. [7] Harper, F. M., & Konstan, J. A. (2015). The MovieLens Datasets: History and Context. ACM Transactions on Interactive Intelligent Systems, 5(4), 19:1-19:19.

Copyright

Copyright © 2026 Nitish M, Prabhanjan J, Venkatesh S, A. Ilakkia. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.