ENHANCING PREDICTIVE ANALYTICS WITH STACKING AND SYNTHETIC MINORITY OVER-SAMPLING BALANCING
Abstract
In the sphere of education, the assessment of student advancement is of utmost importance for educational establishments. These institutions sometimes encounter hurdles such as appraising student performance, conducting academic scrutiny, and foreseeing prospective learning requisites. To tackle these impediments, academic intervention schemes have been put into operation. Accurate prediction regarding student performance not only facilitate the early identification of struggling learners but also assist higher education institutions in their decision-making processes. Consequently, this study introduces a predictive model for the evaluation of scholastic accomplishment. It utilizes XGBoost for the selection of pertinent attributes and employs Synthetic Minority Oversampling Technique (SMOTE) to rectify class imbalances. Furthermore, it employs a stacking ensemble methodology encompassing base models like K-Nearest Neighbors, Support Vector Machines, Random Forest, Decision Trees, Multi-Layer Perceptron, and Logistic Regression to enhance predictive precision. This investigation provides an exhaustive theoretical assessment of models, showcasing practical applications of machine learning in education. With an exceptional 98% precision in the prediction of student performance, this research underscores the capacity of educational institutions to elevate outcomes. This study substantially propels predictive modeling in the realm of education, making a valuable contribution to the field of research.
Key words: Predictive analytics, stacking model, educational data mining, SMOTE, ensemble learning
Full Text:
PDFReferences
Akgün, E. (2020). Science Mapping Research on Educational Data Mining: A Bibliometric Review of International Publications. Future Visions Journal, 4(14), 1–17. https://doi.org/10.29345/futvis.160
Alamgir, Z., Akram, H., Karim, S., & Wali, A. (2023). Enhancing Student Performance Prediction via Educational Data Mining on Academic data. Informatics in Education. https://doi.org/10.15388/infedu.2024.04
Albreiki, B., Zaki, N., &Alashwal, H. (2021). A Systematic Literature Review of Student‘ Performance Prediction Using Machine Learning Techniques. Education Sciences, 11(9), 552. https://doi.org/10.3390/educsci11090552
Atif, A. (2013). Learning Analytics in Higher Education: A Summary of Tools and Approaches. Learning & Technology Library (LearnTechLib).
Aulakh, K., Roul, R. K., & Kaushal, M. (2023). E-learning enhancement through educational data mining with Covid-19 outbreak period in backdrop: A review. International Journal of Educational Development, 101, 102814. https://doi.org/10.1016/j.ijedudev.2023.102814
Banda, L. O. L., Liu, J., Banda, J. T., & Zhou, W. (2023). Impact of ethnic identity and geographical home location on student academic performance. Heliyon, 9(6), e16767. https://doi.org/10.1016/j.heliyon.2023.e16767
Begum, S., &Padmannavar, S. S. (2023). Student Performance Analysis using Bayesian Optimized Random Forest Classifier and KNN. International Journal of Engineering Trends and Technology, 71(5), 132–
https://doi.org/10.14445/22315381/ijett-v71i5p213
Chawla, N. V., Bowyer, K. W., Hall, L. O., &Kegelmeyer, W. P. (2002). SMOTE: Synthetic Minority Over- sampling Technique. Journal of Artificial Intelligence Research, 16, 321–357. https://doi.org/10.1613/jair.953
Cheng, S. T., & Kaplowitz, S. A. (2016). Family economic status, cultural capital, and academic achievement: The case of Taiwan. International Journal of Educational Development, 49, 271–278. https://doi.org/10.1016/j.ijedudev.2016.04.002
Dervenis, C., Kyriatzis, V., Stoufis, S., &Fitsilis, P. (2022). Predicting Students‘ Performance Using Machine Learning Algorithms. Proceedings of the 6th International Conference on Algorithms, Computing and Systems. https://doi.org/10.1145/3564982.3564990
Desjardins, S., & Grandbois, M. (2022). Sleep parameters associated with university students‘ grade point average and dissatisfaction with academic performance. Sleep Epidemiology, 2, 100038. https://doi.org/10.1016/j.sleepe.2022.100038
González-Brambila, S. B., Sánchez-Guerrero, L., Ardón-Pulido, I., Figueroa-González, J., & González- Beltrán, B. (2018). Predicting Academic Performance of Engineering Students After Approving a Mathematics Leveling Course using Decision Trees. Research in Computing Science, 147(12), 171–
https://doi.org/10.13053/rcs-147-12-16
H. Alamri, L., S. Almuslim, R., S. Alotibi, M., K. Alkadi, D., Ullah Khan, I., & Aslam, N. (2020). Predicting Student Academic Performance using Support Vector Machine and Random Forest. 2020 3rd International Conference on Education Technology Management. https://doi.org/10.1145/3446590.3446607
Hanaysha, J. R., Shriedeh, F. B., &In‘airat, M. (2023). Impact of classroom environment, teacher competency, information and communication technology resources, and university facilities on student engagement and academic performance. International Journal of Information Management Data Insights, 3(2), 100188. https://doi.org/10.1016/j.jjimei.2023.100188
Haryawan, C., &Sebatubun, M. M. (2020). Implementation of multilayer perceptron for student failure prediction. JUTI: JurnalIlmiahTeknologiInformasi, 18(2), 125.
https://doi.org/10.12962/j24068535.v18i2.a990
Howard, W.R. (2007), "Pattern Recognition and Machine Learning", Kybernetes, Vol. 36 No. 2, pp. 275-275. https://doi.org/10.1108/03684920710743466
Kim, Y., Li, T., Kim, H. K., Oh, W., & Wang, Z. (2023). Socioeconomic status and school adjustment trajectories among academically at-risk students: The mediating role of parental school-based involvement. Journal of Applied Developmental Psychology, 87, 101561.
https://doi.org/10.1016/j.appdev.2023.101561
King, R. D., Orhobor, O. I., & Taylor, C. C. (2021). Cross-validation is safe to use. Nature Machine Intelligence, 3(4), 276–276. https://doi.org/10.1038/s42256-021-00332-z
Kleinkorres, R., Stang-Rabrig, J., & McElvany, N. (2023). Comparing parental and school pressure in terms of their relations with students‘ well-being. Learning and Individual Differences, 104, 102288. https://doi.org/10.1016/j.lindif.2023.102288
Kumar, B., & Pal, S. (2011). Mining Educational Data to Analyze Students Performance. International Journal of Advanced Computer Science and Applications, 2(6). https://doi.org/10.14569/ijacsa.2011.020609
Lagman, A. C. (2015). Embedding Logistic Regression Model in Decision Support Software for Student Graduation Prediction. Proceedings Journal of Interdisciplinary Research, 2, 104–110. https://doi.org/10.21016/irrc.2015.au05ef81o
Lang, C., Siemens, G., Wise, A., &Gasevic, D. (Eds.). (2017). Handbook of Learning Analytics. https://doi.org/10.18608/hla17
Lehti, H. (2023). Parental Unemployment and Children‘s Educational Outcomes – A Literature Review. International Encyclopedia of Education (Fourth Edition), 118–127. https://doi.org/10.1016/b978-0- 12-818630-5.01019-8
Li, Z., & Qiu, Z. (2018). How Does Family Background Affect Children‘s Educational Achievement? Evidence from Contemporary China. The Journal of Chinese Sociology, 5(1). https://doi.org/10.1186/s40711- 018-0083-8
Liu, R. (2015). Variations in Learning Rate: Student Classification Based on Systematic Residual Error Patterns across Practice Opportunities. Retrieved from https://api.semanticscholar.org/CorpusID:11413105
Liu, R., & Hannum, E. (2023). Parental Absence and Student Academic Performance in Cross-National Perspective: Heterogeneous Forms of Family Separation and the Buffering Possibilities of Grandparents. International Journal of Educational Development, 103, 102898. https://doi.org/10.1016/j.ijedudev.2023.102898
Lucey, S., & Grydaki, M. (2022). University Attendance and Academic Performance: Encouraging Student Engagement. Scottish Journal of Political Economy, 70(2), 180–199. https://doi.org/10.1111/sjpe.12334
Márquez-Vera, C., Cano, A., Romero, C., & Ventura, S. (2012). Predicting Student Failure at School Using Genetic Programming and Different Data Mining Approaches with High Dimensional and Imbalanced Data. Applied Intelligence, 38(3), 315–330. https://doi.org/10.1007/s10489-012-0374-8
Mastour, H., Dehghani, T., Moradi, E., & Eslami, S. (2023). Early Prediction of Medical Students‘ Performance in High-Stakes Examinations Using Machine Learning Approaches. Heliyon, 9(7), e18248. https://doi.org/10.1016/j.heliyon.2023.e18248
Mativo, J. M., & Huang, S. (2014). Prediction of Students‘ Academic Performance: Adapt a Methodology of Predictive Modeling for a Small Sample Size. 2014 IEEE Frontiers in Education Conference (FIE) Proceedings. https://doi.org/10.1109/fie.2014.7044287
Mengash, H. A. (2020). Using Data Mining Techniques to Predict Student Performance to Support Decision Making in University Admission Systems. IEEE Access, 8, 55462–55470. https://doi.org/10.1109/access.2020.2981905
Mulyana, A. F., Puspita, W., &Jumanto, J. (2023). Increased Accuracy in Predicting Student Academic Performance Using Random Forest Classifier. Journal of Student Research Exploration, 1(2), 94–
https://doi.org/10.52465/josre.v1i2.169
Nawang, H., Makhtar, M., & Hamza, W. M. A. F. W. (2021). A Systematic Literature Review on Student Performance Predictions. International Journal of Advanced Technology and Engineering Exploration, 8(84). https://doi.org/10.19101/ijatee.2021.874521
Nugroho, A., Riady, O. R., Calvin, A., &Suhartono, D. (2020). Identification of Student Academic Performance Using the KNN Algorithm. Engineering, MAthematics and Computer Science (EMACS) Journal, 2(3), 115–122. https://doi.org/10.21512/emacsjournal.v2i3.6537
Priscilla, C. V., & Prabha, D. P. (2021). A two-phase feature selection technique using mutual information and XGB-RFE for credit card fraud detection. International Journal of Advanced Technology and Engineering Exploration, 8(85). https://doi.org/10.19101/ijatee.2021.874615
QIN, F., ZHU, L. Q., CHENG, Z. K., & ZHANG, Q. (2017). Research on Student Performance Evaluation Based on Random Forest. DEStech Transactions on Engineering and Technology Research. https://doi.org/10.12783/dtetr/eeta2017/7762
Rabelo, A., Rodrigues, M. W., Nobre, C., Isotani, S., & Zárate, L. (2023). Educational data mining and learning analytics: a review of educational management in e-learning. Information Discovery and Delivery. https://doi.org/10.1108/idd-10-2022-0099
Rahman, L., Setiawan, N. A., &Permjereari, A. E. (2017). Feature selection methods in improving accuracy of classifying students‘ academic performance. 2017 2nd International Conferences on Information Technology, Information Systems and Electrical Engineering (ICITISEE). https://doi.org/10.1109/icitisee.2017.8285509
Rajendran, S., Chamundeswari, S., & Sinha, A. A. (2022). Predicting the academic performance of middle- and high-school students using machine learning algorithms. Social Sciences & Humanities Open, 6(1), 100357. https://doi.org/10.1016/j.ssaho.2022.100357
Romero, C., & Ventura, S. (2010). Educational Data Mining: A Review of the State of the Art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6), 601– 618. https://doi.org/10.1109/tsmcc.2010.2053532
Sawangarreerak, S., &Thanathamathee, P. (2020). Random Forest with Sampling Techniques for Handling Imbalanced Prediction of University Student Depression. Information, 11(11), 519. https://doi.org/10.3390/info11110519
Schalk, P. D., Wick, D. P., Turner, P. R., & Ramsdell, M. W. (2011). Predictive assessment of student performance for early strategic guidance. 2011 Frontiers in Education Conference (FIE). https://doi.org/10.1109/fie.2011.6143086
Tantoh, M. C. (2023). Parental Level of Education and its Implications of their Expectations towards their Children Academic Performance. International Journal of Psychology and Cognitive Education, 2(1), 1–20. https://doi.org/10.58425/ijpce.v2i1.109
Uylaş, N. (2018). Semi-Supervised Classification in Educational Data Mining: Students‘ Performance Case Study. International Journal of Computer Applications, 179(26), 13–17. https://doi.org/10.5120/ijca2018916549
Wang, Y., & Wang, Y. (2023). Exploring the relationship between educational ICT resources, student engagement, and academic performance: A multilevel structural equation analysis based on PISA 2018 data. Studies in Educational Evaluation, 79, 101308.
https://doi.org/10.1016/j.stueduc.2023.101308
Xu, J., Han, Y., Marcu, D., & Van der Schaar, M. (2017). Progressive Prediction of Student Performance in College Programs. Proceedings of the AAAI Conference on Artificial Intelligence, 31(1). https://doi.org/10.1609/aaai.v31i1.10713
Yohannes, E., & Ahmed, S. (2018). Prediction of Student Academic Performance using Neural Network, Linear Regression and Support Vector Regression: A Case Study. International Journal of Computer Applications, 180(40), 39–47. https://doi.org/10.5120/ijca2018917057
Zaffar, M., Hashmani, M. A., & Savita, K. S. (2017). Performance analysis of feature selection algorithm for educational data mining. 2017 IEEE Conference on Big Data and Analytics (ICBDA). https://doi.org/10.1109/icbdaa.2017.8284099
Zhang, Y., & Liu, Y. (2021). The Research of Predicting Student‘s Academic Performance Based on Educational Data. 2021 5th International Conference on Computer Science and Artificial Intelligence. https://doi.org/10.1145/3507548.3507578
Refbacks
- There are currently no refbacks.