METAHEURISTIC HYPERPARAMETER OPTIMIZATION FOR DEEP LEARNING: A COMPARATIVE STUDY
DOI:
https://doi.org/10.33480/jitk.v12i1.8269Keywords:
Deep Learning, Dwarf Mongoose Optimization, Hyperparameter Optimization, Model Performance, Particle Swarm OptimizationAbstract
One of the primary challenges in developing effective deep learning models lies in identifying optimal hyperparameter configurations, particularly within high-dimensional and complex search spaces. Traditional tuning strategies, including grid search and random search, are widely known to be computationally inefficient and frequently yield suboptimal outcomes. To address this limitation, this study presents a comparative analysis of two metaheuristic algorithms — Particle Swarm Optimization (PSO) and Dwarf Mongoose Optimization (DMO) — as advanced alternatives for hyperparameter tuning in deep learning models trained on the CIFAR-10 dataset. Both algorithms were rigorously assessed using a comprehensive set of performance metrics, namely accuracy, precision, recall, and F1-score, supplemented by confusion matrix analysis to capture class-level behavior. Experimental findings confirm that both approaches yield substantial improvements in model performance. Notably, PSO demonstrated superior results, achieving a validation accuracy of 89.15% and a test accuracy of 87.82%, while DMO reached a final accuracy of 86.38%. In terms of optimization behavior, PSO exhibited greater convergence stability and more uniform class-level performance, whereas DMO proved more effective in broadly exploring the search space. Overall, this study reinforces the potential of metaheuristic-based optimization as a robust framework for hyperparameter tuning and underscores the critical role of optimization stability in achieving reliable model generalization.
Downloads
References
[1] T. Talaei Khoei, H. Ould Slimane, and N. Kaabouch, “Deep Learning: Systematic Review, Models, Challenges, and Research Directions,” Neural Comput. Appl., vol. 35, pp. 23103–23124, 2023, doi: 10.1007/s00521-023-08957-4.
[2] X. Zhao, L. Wang, Y. Zhang, X. Han, M. Deveci, and M. Parmar, “A Review of Convolutional Neural Networks in Computer Vision,” Artif. Intell. Rev., vol. 57, p. 99, 2024, doi: 10.1007/s10462-024-10721-6.
[3] A. Chen, Z. Ren, M. Wang, H. Chen, H. Leng, and S. Liu, “A Surrogate-Assisted Highly Cooperative Coevolutionary Algorithm for Hyperparameter Optimization in Deep Convolutional Neural Networks,” Appl. Soft Comput., vol. 147, p. 110794, 2023, doi: 10.1016/j.asoc.2023.110794.
[4] B. Bischl et al., “Hyperparameter Optimization: Foundations, Algorithms, Best Practices and Open Challenges,” Wiley Interdiscip. Rev. Data Min. Knowl. Discov., vol. 13, no. 2, p. e1484, 2023, doi: 10.1002/widm.1484.
[5] I. Amerini, and L. Palagi, “Tuning parameters of deep neural network training algorithms pays off: a computational study,” TOP, vol. 32, no. 3, pp. 579–620, Oct. 2024, doi: 10.1007/s11750-024-00683-x.
[6] T. Maumela, F. Nelwamondo, and T. Marwala, “Population based training and federated learning frameworks for hyperparameter optimisation and ML unfairness using Ulimisana Optimisation Algorithm,” Inf. Sci. (N. Y)., vol. 612, pp. 132–150, 2022, doi: 10.1016/j.ins.2022.08.091.
[7] A. Dushatskiy, A. Chebykin, T. Alderliesten, and P. Bosman, “Multi-Objective Population Based Training,” in Proceedings of the 40th International Conference on Machine Learning, PMLR, 2023, pp. 8969–8989.
[8] F. Zito, E.-G. Talbi, C. Cavallaro, V. Cutello, and M. Pavone, “Metaheuristics in Automated Machine Learning: Strategies for Optimization,” Intelligent Systems with Applications, vol. 26, p. 200532, 2025, doi: 10.1016/j.iswa.2025.200532.
[9] D. Gutiérrez-Avilés, M. J. Jiménez-Navarro, J. F. Torres, and F. Martínez-Álvarez, “MetaGen: A Framework for Metaheuristic Development and Hyperparameter Optimization in Machine and Deep Learning,” Neurocomputing, vol. 637, p. 130046, 2025, doi: 10.1016/j.neucom.2025.130046.
[10] T. M. Shami, A. A. El-Saleh, M. Alswaitti, Q. Al-Tashi, M. A. Summakieh, and S. Mirjalili, “Particle Swarm Optimization: A Comprehensive Survey,” IEEE Access, vol. 10, pp. 10031–10061, 2022, doi: 10.1109/ACCESS.2022.3142859.
[11] T.-Y. Hong and C.-C. Chen, “Hyperparameter Optimization for Convolutional Neural Network by Opposite-Based Particle Swarm Optimization and an Empirical Study of Photomask Defect Classification,” Appl. Soft Comput., vol. 148, p. 110904, 2023, doi: 10.1016/j.asoc.2023.110904.
[12] A. G. Gad, “Particle Swarm Optimization Algorithm and Its Applications: A Systematic Review,” Archives of Computational Methods in Engineering, vol. 29, pp. 2531–2561, 2022, doi: 10.1007/s11831-021-09694-4.
[13] J. O. Agushaka and others, “Dwarf mongoose optimization algorithm: A new bio-inspired metaheuristic,” Expert Syst. Appl., 2022.
[14] J. O. Agushaka, A. E. Ezugwu, and L. Abualigah, “Dwarf Mongoose Optimization Algorithm,” Comput. Methods Appl. Mech. Eng., vol. 391, p. 114570, 2022, doi: 10.1016/j.cma.2022.114570.
[15] I. Al-Shourbaji et al., “Artificial Ecosystem-Based Optimization with Dwarf Mongoose Optimization for Feature Selection and Global Optimization Problems,” International Journal of Computational Intelligence Systems, vol. 16, p. 102, 2023, doi: 10.1007/s44196-023-00279-6.
[16] S. Fu, H. Huang, C. Ma, J. Wei, Y. Li, and Y. Fu, “Improved Dwarf Mongoose Optimization Algorithm Using Novel Nonlinear Control and Exploration Strategies,” Expert Syst. Appl., vol. 233, p. 120904, 2023, doi: 10.1016/j.eswa.2023.120904.
[17] G. Moustafa et al., “An Enhanced Dwarf Mongoose Optimization Algorithm for Solving Engineering Problems,” Mathematics, vol. 11, no. 15, p. 3297, 2023, doi: 10.3390/math11153297.
[18] S. Aslam and A. B. Nassif, “Deep Learning Based CIFAR-10 Classification,” in 2023 Advances in Science and Engineering Technology International Conferences (ASET), IEEE, 2023. doi: 10.1109/ASET56582.2023.10180767.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Zulfahmi Syahputra, Romi Fadillah Rahmat
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
-a.jpg)
-b.jpg)
