Optimizing Stacking Ensemble Models for Customer Churn Prediction in the Telecommunications Industry

Rofik Rofik; Jumanto Unjung; Dwika Ananda Agustina Pertiwi; Much Aziz Muslim

doi:10.15575/join.v11i1.1783

Authors

Rofik Rofik Department of Computer Science, Universitas Negeri Semarang, Indonesia
Jumanto Unjung Department of Computer Science, Universitas Negeri Semarang, Indonesia
Dwika Ananda Agustina Pertiwi Faculty Technology Management and Business, Universiti Tun Hussein Onn Malaysia, Malaysia
Much Aziz Muslim Department of Computer Science, Universitas Negeri Semarang, Indonesia

DOI:

https://doi.org/10.15575/join.v11i1.1783

Keywords:

Churn prediction, Logistic regression, Machine learning, Stacking ensemble, Telecommunications

Abstract

One of the biggest challenges in the telecommunications industry is predicting churn, which is the condition when a customer unsubscribes and switches to another service provider. In an era of competitive market conditions, retaining customers is much more efficient than acquiring new customers. Conventional prediction models are often unable to capture the complexity of customer behavior patterns, resulting in a lower accuracy than optimal. This study aims to optimize customer churn prediction performance by developing a stacking ensemble model that combines several classification algorithms to improve model performance. Fourteen algorithms were tested, and the six algorithms with the best accuracy were selected as base learners, while Logistic Regression was selected as the meta-learner. The stacking model testing was carried out sequentially through a combination of 6 algorithms with the same meta-learner algorithm. Testing was also carried out with and without using the SMOTE data balancing method to evaluate the effect of data balancing on the prediction results. The results of this study show that the combination of the Adaboost, Ridge Classifier, and Logistic Regression algorithms can produce the highest accuracy of 82.97%, which exceeds the prediction performance of a single algorithm. This research contributes to demonstrating an effective stacking ensemble configuration for predicting customer churn in the telecommunications industry and emphasizes that the selection of the right algorithm combination has a greater impact on model performance than the number of algorithms used.

References

[1] Y. Chen, R. Calabrese, and B. Martin-Barragan, “Interpretable machine learning for imbalanced credit scoring datasets,” Eur. J. Oper. Res., vol. 312, no. 1, pp. 357–372, 2024.

[2] A. Amin, F. Al-Obeidat, B. Shah, A. Adnan, J. Loo, and S. Anwar, “Customer churn prediction in telecommunication industry using data certainty,” J. Bus. Res., vol. 94, pp. 290–301, 2019.

[3] B. Prabadevi, R. Shalini, and B. R. Kavitha, “Customer churning analysis using machine learning algorithms,” Int. J. Intell. Networks, vol. 4, pp. 145–154, 2023.

[4] R. Rofik, J. Unjung, and B. Prasetiyo, “Enhancing Customer Churn Prediction with Stacking Ensemble and Stratified K-Fold,” Bull. Electr. Eng. Informatics, vol. 14, no. 1, pp. 398–408, Feb. 2025.

[5] M. Reppmann, S. Harms, L. M. Edinger-Schons, and J. N. Foege, “Activating the Sustainable Consumer: The Role of Customer Involvement in Corporate Sustainability,” J. Acad. Mark. Sci., vol. 53, no. 2, pp. 310–340, Mar. 2025.

[6] A. Amin, F. Al-Obeidat, B. Shah, A. Adnan, J. Loo, and S. Anwar, “Cross-Company Customer Churn Prediction in Telecommunication: A Comparison of Data Transformation Methods,” Int. J. Inf. Manage., vol. 46, pp. 304–319, Jun. 2019.

[7] Y. Li, B. Hou, Y. Wu, D. Zhao, A. Xie, and P. Zou, “Giant Fight: Customer Churn Prediction in Traditional Broadcast Industry,” J. Bus. Res., vol. 131, pp. 630–639, Jul. 2021.

[8] E. Sivasankar and J. Vijaya, “Hybrid PPFCM-ANN Model: An Efficient System for Customer Churn Prediction through Probabilistic Possibilistic Fuzzy Clustering and Artificial Neural Network,” Neural Comput. Appl., vol. 31, no. 11, pp. 7181–7200, Nov. 2019.

[9] S. M. Keaveney, “Customer Switching Behavior in Service Industries: An Exploratory Study,” J. Mark., vol. 59, no. 2, 1995.

[10] J. H. Ahn, S. P. Han, and Y. S. Lee, “Customer churn analysis: Churn determinants and mediation effects of partial defection in the Korean mobile telecommunications service industry,” Telecomm. Policy, vol. 30, no. 10–11, pp. 552–568, 2006.

[11] A. Al-Refaie, M. Al-Tarawneh, N. Bata, and others, “Study Of Customer Churn In The Telecom Industry Using Structural Equation Modelling,” Econ. Bus. J., vol. 12, no. 1, pp. 393–411, 2018.

[12] A. Alam, “Define Machine Learning and Describe the Main Types of Machine Learning.” Aug-2023.

[13] Y. Chen, R. Calabrese, and B. Martin-Barragan, “Interpretable machine learning for imbalanced credit scoring datasets,” Eur. J. Oper. Res., no. xxxx, 2023.

[14] D. K. Sharma, S. Lohana, S. Arora, A. Dixit, M. Tiwari, and T. Tiwari, “E-Commerce product comparison portal for classification of customer data based on data mining,” Mater. Today Proc., vol. 51, pp. 166–171, 2021.

[15] A. Ishaq et al., “Improving the Prediction of Heart Failure Patients’ Survival Using SMOTE and Effective Data Mining Techniques,” IEEE Access, vol. 9, pp. 39707–39716, 2021.

[16] B. Prasetiyo, Alamsyah, M. A. Muslim, Subhan, and N. Baroroh, “Artificial Neural Network Model for Bankruptcy Prediction,” J. Phys. Conf. Ser., vol. 1567, no. 3, p. 32022, Jun. 2020.

[17] Y. Dasril, M. A. Muslim, M. F. Al Hakim, Jumanto, and B. Prasetiyo, “Credit Risk Assessment in P2P Lending Using LightGBM and Particle Swarm Optimization,” Regist. J. Ilm. Teknol. Sist. Inf., vol. 9, no. 1, pp. 18–28, Feb. 2023.

[18] S. Wu, W.-C. Yau, T.-S. Ong, and S.-C. Chong, “Integrated Churn Prediction and Customer Segmentation Framework for Telco Business,” IEEE Access, vol. 9, pp. 62118–62136, 2021.

[19] A. Amin, F. Al-Obeidat, B. Shah, A. Adnan, J. Loo, and S. Anwar, “Customer churn prediction in telecommunication industry under uncertain situation,” J. Bus. Res., 2018.

[20] C. Wang, C. Rao, F. Hu, X. Xiao, and M. Goh, “Risk Assessment of Customer Churn in Telco Using FCLCNN-LSTM Model,” Expert Syst. Appl., vol. 248, p. 123352, Jan. 2024.

[21] S. J. Haddadi, A. Farshidvard, F. dos S. Silva, J. C. dos Reis, and M. da Silva Reis, “Customer Churn Prediction in Imbalanced Datasets with Resampling Methods: A Comparative Study,” Expert Syst. Appl., vol. 246, p. 123086, Sep. 2024.

[22] S. D. Team, “6. Underfitting and Overfitting.” .

[23] S. Guo, H. He, and X. Huang, “A Multi-Stage Self-Adaptive Classifier Ensemble Model With Application in Credit Scoring,” IEEE Access, vol. 7, pp. 78549–78559, 2019.

[24] R. Jayapermana, A. Aradea, and N. I. Kurniati, “Implementation of Stacking Ensemble Classifier for Multi-class Classification of COVID-19 Vaccines Topics on Twitter,” Sci. J. Informatics, vol. 9, no. 1, pp. 8–15, 2022.

[25] X. Yin, Q. Liu, Y. Pan, X. Huang, J. Wu, and X. Wang, “Strength of Stacking Technique of Ensemble Learning in Rockburst Prediction with Imbalanced Data: Comparison of Eight Single and Ensemble Models,” Nat. Resour. Res., vol. 30, no. 2, pp. 1795–1815, 2021.

[26] R. Rofik and J. Unjung, “Evaluation of Ridge Classifier and Logistic Regression for Customer Churn Prediction on Imbalanced Telecommunication Data,” Sci. J. Informatics, vol. 12, no. 2, pp. 311–326, 2025.

[27] N. Mehrabi, F. Morstatter, N. Saxena, K. Lerman, and A. Galstyan, “A Survey on Bias and Fairness in Machine Learning,” ACM Comput. Surv., vol. 54, no. 6, 2021.

[28] H. Guan, L. Zhao, X. Dong, and C. Chen, “Extended natural neighborhood for SMOTE and its variants in imbalanced classification,” Eng. Appl. Artif. Intell., vol. 124, no. March, p. 106570, 2023.

[29] Y. Ortakci and H. Seker, “Optimising Customer Retention: An AI-driven Personalised Pricing Approach,” Comput. & Ind. Eng., vol. 188, p. 109920, 2024.

[30] N. G. Ramadhan, “Comparative Analysis of ADASYN-SVM and SMOTE-SVM Methods on the Detection of Type 2 Diabetes Mellitus,” Sci. J. Informatics, vol. 8, no. 2, pp. 276–282, 2021.

[31] Y. Yustikasari, H. Mubarok, and R. Rianto, “Comparative Analysis Performance of K-Nearest Neighbor Algorithm and Adaptive Boosting on the Prediction of Non-Cash Food Aid Recipients,” Sci. J. Informatics, vol. 9, no. 2, pp. 205–217, 2022.

[32] K. I. Sundus, B. H. Hammo, M. B. Al-Zoubi, and A. Al-Omari, “Solving the Multicollinearity Problem to Improve the Stability of Machine Learning Algorithms Applied to a Fully Annotated Breast Cancer Dataset,” Informatics Med. Unlocked, vol. 33, p. 101088, Sep. 2022.

[33] V. Lugat, “Pima Indians Diabetes - EDA & Prediction (0.906),” 2019. .

[34] D. H. Wolpert, “Stacked generalization,” Neural Networks, vol. 5, no. 2, p. 241259, 1992.

[35] J. Martinez-gil, “Machine Learning with Applications,” Mach. Learn. with Appl., vol. 10, no. October, p. 100423, 2022.

[36] E. K. Sahin and S. Demir, “Greedy-AutoML: A novel greedy-based stacking ensemble learning framework for assessing soil liquefaction potential,” Eng. Appl. Artif. Intell., vol. 119, no. December 2022, p. 105732, 2023.

[37] J. Unjung and others, “Soft Voting Ensemble Model to Improve Parkinson’s Disease Prediction with SMOTE,” Int. J. Adv. Intell. Informatics, vol. 11, no. 1, p. 120, Feb. 2025.

[38] W. Yin, B. Kirkulak-Uludag, D. Zhu, and Z. Zhou, “Stacking ensemble method for personal credit risk assessment in Peer-to-Peer lending,” Appl. Soft Comput., vol. 142, p. 110302, 2023.

[39] M. A. Muslim et al., “New model combination meta-learner to improve accuracy prediction P2P lending with stacking ensemble learning,” Intell. Syst. with Appl., vol. 18, no. December 2022, p. 200204, 2023.

[40] Y. Yang, L. Wei, Y. Hu, Y. Wu, L. Hu, and S. Nie, “Classification of Parkinson’s disease based on multi-modal features and stacking ensemble learning,” J. Neurosci. Methods, vol. 350, no. May 2020, p. 109019, 2021.

[41] F. A. Rafrastara, C. Supriyanto, C. Paramita, and Y. P. Astuti, “Deteksi Malware Menggunakan Metode Stacking Berbasis Ensemble,” J. Inform. J. Pengemb. IT, vol. 8, no. 1, pp. 11–16, 2023.

[42] C. A. Hafsath and A. S. Jereesh, “A Stacked Ensemble Approach for Enhancing Anti Cancer Drug Synergy Prediction,” Procedia Comput. Sci., vol. 235, pp. 2567–2576, 2024.

[43] T. Yan, S. L. Shen, A. Zhou, and X. Chen, “Prediction of geological characteristics from shield operational parameters by integrating grid search and K-fold cross validation into stacking classification algorithm,” J. Rock Mech. Geotech. Eng., vol. 14, no. 4, pp. 1292–1303, 2022.

[44] P. Lalwani, M. K. Mishra, J. S. Chadha, and P. Sethi, “Customer Churn Prediction System: A Machine Learning Approach,” Computing, vol. 104, no. 2, pp. 271–294, 2022.

[45] W. Shan, D. Li, S. Liu, M. Song, S. Xiao, and H. Zhang, “A Random Feature Mapping Method Based on the AdaBoost Algorithm and Results Fusion for Enhancing Classification Performance,” Expert Syst. Appl., vol. 256, p. 124902, Jul. 2024.

[46] T. S. Lee, C. C. Chiu, Y. C. Chou, and C. J. Lu, “Mining the Customer Credit Using Classification and Regression Tree and Multivariate Adaptive Regression Splines,” Comput. Stat. & Data Anal., vol. 50, no. 4, pp. 1113–1130, 2006.

[47] G. Nie, W. Rowe, L. Zhang, Y. Tian, and Y. Shi, “Credit Card Churn Forecasting by Logistic Regression and Decision Tree,” Expert Syst. Appl., vol. 38, no. 12, pp. 15273–15285, 2011.

[48] H. Jain, A. Khunteta, and S. Srivastava, “Churn Prediction in Telecommunication Using Logistic Regression and Logit Boost,” in Procedia Computer Science (Proceedings of ICCCA 2016?), 2020, vol. 167, pp. 101–112.

[49] A. Singh, B. S. Prakash, and K. Chandrasekaran, “A comparison of linear discriminant analysis and ridge classifier on Twitter data,” Proceeding - IEEE Int. Conf. Comput. Commun. Autom. ICCCA 2016, pp. 133–138, 2017.