Student Performance Prediction Using Machine Learning: A Comparative Analysis of Decision Tree, Random Forest, and Optimized XGBoost

Abstract

Student performance prediction has become an important application of Machine Learning in educational data mining, enabling institutions to identify academically weak students at an early stage and provide timely academic support. Accurate prediction of student performance helps educators implement personalized learning strategies, reduce failure rates, and improve overall educational outcomes. This paper presents a Machine Learning-based Student Performance Prediction System that compares the performance of two existing classification algorithms, Decision Tree and Random Forest, with a proposed Optimized XGBoost algorithm. The proposed system utilizes the UCI Student Performance dataset, which combines Mathematics and Portuguese student records into a single dataset containing 1,044 student instances. The dataset undergoes comprehensive preprocessing, including duplicate removal, categorical encoding, feature scaling, feature selection, and performance category generation. Three supervised Machine Learning models are trained and evaluated using Accuracy, Precision, Recall, F1-Score, Log Loss, Cohen\'s Kappa, and Matthews Correlation Coefficient (MCC). Experimental results demonstrate that the proposed Optimized XGBoost model achieves the highest prediction accuracy of 67.94%, outperforming Random Forest (65.55%) and Decision Tree (61.72%). The trained model is deployed using the Streamlit framework to provide a user-friendly web application capable of predicting student performance and generating academic recommendations in real time. The proposed system offers an efficient decision-support tool for educational institutions, enabling proactive academic intervention and data-driven educational management.

Introduction

The text presents a Machine Learning-based Student Performance Prediction System designed to identify students who may need academic support at an early stage. Traditional evaluation methods often detect problems only after poor examination results, whereas Machine Learning can analyze historical student data and predict future academic performance proactively.

The system considers factors such as study time, attendance, parental education, family background, previous grades, health, demographic characteristics, and social factors. Several algorithms can be used for prediction, including Decision Tree, Random Forest, Support Vector Machine, Logistic Regression, and Artificial Neural Networks. The study focuses on Decision Tree, Random Forest, and Optimized XGBoost, with XGBoost selected as the proposed model because of its strong performance, regularization, scalability, and ability to reduce overfitting.

Dataset and Methodology

The study combines the Mathematics and Portuguese UCI Student Performance datasets, producing a dataset of 1,044 student records. The Machine Learning pipeline includes:

1. Dataset collection and merging.
2. Data cleaning and preprocessing.
3. Duplicate and missing-value checking.
4. Categorical encoding and feature scaling.
5. Feature selection to identify important factors.
6. Training and testing of Machine Learning models.
7. Hyperparameter optimization of XGBoost.
8. Evaluation using multiple performance metrics.
9. Deployment of the best model through a Streamlit web application.

Students are classified into four categories: Poor, Average, Good, and Excellent.

Algorithms Compared

The study evaluates:

• Decision Tree – Used as an existing baseline model.
• Random Forest – Uses multiple decision trees to improve generalization and reduce overfitting.
• Optimized XGBoost – The proposed model, enhanced through hyperparameter optimization, gradient boosting, and regularization.

Experimental Results

The models were trained using 80% of the data and tested using the remaining 20%.

The Optimized XGBoost model achieved the best overall performance, with an accuracy of 67.94%, outperforming Random Forest and Decision Tree. Its improved performance is attributed to gradient boosting, regularization, and hyperparameter optimization, which help the model capture complex relationships in student data.

Deployment and Practical Use

The best-performing model is deployed through a Streamlit-based web application. Users can enter student information through the interface, after which the system predicts the student's performance category, provides confidence scores, and generates academic recommendations.

Conclusion

The increasing availability of educational data has created opportunities for applying Machine Learning techniques to improve academic decision-making. This paper presented a Student Performance Prediction System that compares Decision Tree, Random Forest, and Optimized XGBoost algorithms using the combined UCI Student Performance dataset. The proposed system employed comprehensive preprocessing, feature selection, and hyperparameter optimization before training the Machine Learning models. Experimental evaluation demonstrated that the Optimized XGBoost algorithm achieved the highest prediction accuracy of 67.94%, outperforming Random Forest (65.55%) and Decision Tree (61.72%). The trained model was successfully deployed using the Streamlit framework, enabling real-time prediction of student performance and generation of academic recommendations. The proposed system provides an effective decision-support tool for teachers and educational administrators by facilitating early identification of students requiring academic assistance. Future work will focus on integrating larger educational datasets, applying Explainable Artificial Intelligence (XAI) techniques such as SHAP and LIME, and exploring Deep Learning models to further improve prediction accuracy and interpretability.

References

[1] P. Cortez and A. Silva, \"Using Data Mining to Predict Secondary School Student Performance,\" University of Minho, Portugal, 2008. [2] D. Dua and C. Graff, \"UCI Machine Learning Repository,\" University of California, Irvine, 2019. [3] L. Breiman, \"Random Forests,\" Machine Learning, vol. 45, no. 1, pp. 5–32, 2001. [4] J. R. Quinlan, \"Induction of Decision Trees,\" Machine Learning, vol. 1, no. 1, pp. 81–106, 1986. [5] T. Chen and C. Guestrin, \"XGBoost: A Scalable Tree Boosting System,\" in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016. [6] F. Pedregosa et al., \"Scikit-learn: Machine Learning in Python,\" Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011. [7] W. McKinney, \"Data Structures for Statistical Computing in Python,\" in Proceedings of the Python in Science Conference, 2010. [8] C. R. Harris et al., \"Array Programming with NumPy,\" Nature, vol. 585, pp. 357–362, 2020. [9] J. D. Hunter, \"Matplotlib: A 2D Graphics Environment,\" Computing in Science & Engineering, vol. 9, no. 3, pp. 90–95, 2007. [10] Streamlit Inc., \"Streamlit Documentation,\" 2024.

Copyright

Copyright © 2026 Kasukurthi Soni Sangeetha, Tulasi Miryala. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.