Bridging Scalability and Interpretability in AutoML Via Feature Engineering

Abstract

Feature engineering plays a key role in determining the performance of machine learning models, but manual feature design is time-consuming and requires strong domain knowledge. This work presents an automated feature engineering framework integrated with an interpretable AutoML pipeline, built around the BigFeat methodology. The system automatically generates new features from existing data using mathematical and logical operators and selects the most stable and relevant features for learning, while preserving interpretability by maintaining traceable mappings between original and engineered features. The framework is designed to handle large, high-dimensional datasets with manageable computational overhead. Automated model selection and hyperparameter tuning across Random Forest, Logistic Regression, and Decision Tree classifiers are incorporated to optimize predictive performance without manual intervention. The proposed system is intended to reduce human effort and development time in the feature engineering process while remaining scalable and adaptable to different datasets. Experimental evaluation on the Madelon dataset, a high-dimensional synthetic benchmark for feature selection, indicates that the automated and interpretable pipeline performs comparably to, and in some respects favourably against, baseline feature engineering approaches, demonstrating the practical effectiveness of combining scalable feature generation with interpretable AutoML.

Introduction

This study presents an automated feature engineering system that leverages Automated Machine Learning (AutoML) to simplify and improve the machine learning pipeline. Feature engineering is a critical step in machine learning, involving the creation, transformation, and selection of relevant features from raw data. Traditionally, this process requires extensive domain expertise, manual effort, and time. The proposed system automates these tasks to improve model accuracy, scalability, and development speed while reducing human intervention and making machine learning more accessible to non-experts.

The motivation for the research stems from the limitations of manual feature engineering, which is labour-intensive, difficult to scale for large and high-dimensional datasets, and prone to human bias and error. AutoML addresses these challenges by automating feature generation, feature selection, model selection, and hyperparameter optimisation, enabling faster and more consistent model development. The primary objective is to build an interpretable and scalable automated feature engineering framework that minimises dependence on domain experts while maintaining high predictive performance.

The literature review examines existing automated feature engineering approaches, including AutoFE, AutoFeat-Hybrid, SAFE, and meta-learning frameworks. While these methods reduce manual effort and improve model performance, they often suffer from high computational costs, limited interpretability, or scalability issues. These limitations motivate the development of a unified framework that integrates feature engineering, model selection, and optimisation within a single AutoML pipeline.

The proposed system introduces BigFeat-FE for automated feature engineering and BigFeat-AutoML for automated model selection and optimisation. BigFeat-FE dynamically generates new features using mathematical operations, feature interactions, and transformations, while applying correlation analysis, variance thresholding, and feature importance ranking to remove redundant and irrelevant features. This approach improves both model interpretability and computational efficiency by retaining only the most informative features.

The methodology follows a sequential pipeline consisting of data collection, preprocessing, automated feature engineering, feature selection, AutoML model training, hyperparameter optimisation, model evaluation, and prediction. The system is evaluated using the Madelon dataset, a high-dimensional benchmark designed for feature selection tasks. During preprocessing, missing values, duplicates, inconsistencies, and outliers are handled, and numerical features are normalised to improve model performance.

For model training, the AutoML framework automatically evaluates Random Forest, Logistic Regression, and Decision Tree classifiers. Hyperparameters are optimised using Grid Search and Random Search, and models are assessed using accuracy, precision, recall, and F1-score. The best-performing model is automatically selected, with optional refinement through feature adjustment or model ensembling. The final prediction is generated using a voting mechanism that combines the outputs of multiple classifiers, producing more robust and reliable predictions.

Overall, the proposed BigFeat-based AutoML framework provides an efficient, scalable, and interpretable solution for automated feature engineering by integrating feature generation, feature selection, model optimisation, and prediction into a unified pipeline. This significantly reduces manual effort while improving machine learning performance and usability across large, high-dimensional datasets.

Conclusion

This work presented an AutoML-based feature engineering system designed to bridge scalability and interpretability using the BigFeat-FE and BigFeat-AutoML methodologies. The system automates the complete feature engineering process-feature creation, transformation, and selection-substantially reducing the need for manual intervention. By integrating automated feature engineering with machine learning models such as Random Forest, Logistic Regression, and Decision Tree, the system is designed to improve model accuracy and overall performance while remaining efficient and scalable for large, high-dimensional datasets such as Madelon. The use of BigFeat-FE enhances feature quality while preserving model interpretability, and BigFeat-AutoML automates model selection and training, together achieving a balance between predictive performance and explainability. The resulting end-to-end pipeline, from data preprocessing through to prediction, offers a scalable and user-friendly machine learning solution that reduces development time and effort, supporting its suitability for real-world deployment.

References

[1] H. Eldeeb and R. Elshawi, “Empowering Machine Learning With Scalable Feature Engineering and Interpretable AutoML,” IEEE Transactions on Artificial Intelligence, vol. 6, no. 2, pp. 432–442, 2025. [2] H. Eldeeb and R. Elshawi, “BigFeat: A Scalable and Interpretable Automated Feature Engineering Framework,” 2023. [3] “Automated Feature Engineering for Machine Learning Using Hybrid Approaches (AutoFeat-Hybrid),” Knowledge-Based Systems, 2024. [4] D. Gil et al., “SAFE: Scalable Automatic Feature Engineering Framework,” arXiv preprint arXiv:2003.02556, 2020. [5] “Automated Feature Engineering and Machine Learning Applications,” Scientific Reports, 2023. [6] L. Breiman, “Random Forests,” Machine Learning, vol. 45, no. 1, pp. 5–32, 2001. [7] R. Olson et al., “TPOT: A Tree-Based Pipeline Optimization Tool for Automated Machine Learning,” 2016. [8] M. Feurer et al., “Efficient and Robust Automated Machine Learning,” Advances in Neural Information Processing Systems, 2015. [9] M. Horn et al., “AutoFeat: Automatic Feature Engineering for Linear Models,” 2019. [10] S. Lundberg and S. Lee, “A Unified Approach to Interpreting Model Predictions (SHAP),” Advances in Neural Information Processing Systems, 2017. [11] F. Pedregosa et al., “Scikit-learn: Machine Learning in Python,” Journal of Machine Learning Research, vol. 12, pp. 2825–2830, 2011. [12] J. Bergstra and Y. Bengio, “Random Search for Hyper-Parameter Optimization,” Journal of Machine Learning Research, vol. 13, pp. 281–305, 2012. [13] J. Snoek, H. Larochelle, and R. P. Adams, “Practical Bayesian Optimization of Machine Learning Algorithms,” Advances in Neural Information Processing Systems, 2012. [14] I. Guyon and A. Elisseeff, “An Introduction to Variable and Feature Selection,” Journal of Machine Learning Research, vol. 3, pp. 1157–1182, 2003. [15] R. Kohavi, “A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection,” IJCAI, 1995.

Copyright

Copyright © 2026 Ms. CH. Vasavi, Ms. SK. Raqeeba. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.