PERFORMANCE COMPARISON OF CLASSICAL ALGORITHMS AND DEEP NEURAL NETWORKS FOR TUBERCULOSIS PREDICTION
Abstract
This study compares the performance of several classical machine learning algorithms and deep neural networks for the prediction of tuberculosis in the Democratic Republic of Congo (DRC), using a sample of 1000 cases including clinical and demographic data. The sample is divided into two sets: 80% for training and 20% for testing. The algorithms evaluated include Support Vector Machine (SVM), K-Nearest Neighbors (KNN), Decision Tree, Random Forest and Convolutional Neural Networks (CNN). The results show that the CNN has the best overall performance with an accuracy of 94%, an AUC of the ROC curve of 93%, an accuracy of 90%, an accuracy of 95%, a sensitivity of 88%, an F1-score of 91.3% and a Log Loss of 0.0386. The Random Forest follows closely behind with an accuracy of 92% and an AUC of 86%. The SVM and KNN models also performed strongly, but slightly less well. The Decision Tree obtained acceptable results, but inferior to the other algorithms evaluated. These results indicate that deep neural networks, and in particular the CNN, are superior for predicting tuberculosis compared with conventional machine learning algorithms. This superiority is particularly marked in terms of accuracy, sensitivity and reliability of predictions, as shown by the performance metrics obtained.
References
Ağbulut, Ü., Gürel, A. E., & Biçen, Y. (2021). Prediction of daily global solar radiation using different machine learning algorithms: Evaluation and comparison. Renewable and Sustainable Energy Reviews, 135, 110114. https://doi.org/10.1016/J.RSER.2020.110114
Auld, A. F., Kerkhoff, A. D., Hanifa, Y., Wood, R., Charalambous, S., Liu, Y., Agizew, T., Mathoma, A., Boyd, R., Date, A., Shiraishi, R. W., Bicego, G., Mathebula-Modongo, U., Alexander, H., Serumola, C., Rankgoane-Pono, G., Pono, P., Finlay, A., Shepherd, J. C., … Fielding, K. (2021). Derivation and external validation of a risk score for predicting HIV-associated tuberculosis to support case finding and preventive therapy scale-up: A cohort study. PLOS Medicine, 18(9), e1003739. https://doi.org/10.1371/JOURNAL.PMED.1003739
Chinagudaba, S. N., Gera, D., Dasu, K. K. V., S, U. S., K, K., Singarajpure, A., U, S., N, S., Chadda, V. K., & N, S. B. (2024). Predictive Analysis of Tuberculosis Treatment Outcomes Using Machine Learning: A Karnataka TB Data Study at a Scale. https://arxiv.org/abs/2403.08834v1
Hrizi, O., Gasmi, K., Ben Ltaifa, I., Alshammari, H., Karamti, H., Krichen, M., Ben Ammar, L., & Mahmood, M. A. (2022). Tuberculosis Disease Diagnosis Based on an Optimized Machine Learning Model. Journal of Healthcare Engineering, 2022(1), 8950243. https://doi.org/10.1155/2022/8950243
Huang, J. C., Ko, K. M., Shu, M. H., & Hsu, B. M. (2020). Application and comparison of several machine learning algorithms and their integration models in regression problems. Neural Computing and Applications, 32(10), 5461–5469. https://doi.org/10.1007/S00521-019-04644-5/METRICS
Jha, K. K., Jha, R., Jha, A. K., Hassan, M. A. M., Yadav, S. K., & Mahesh, T. (2021). A Brief Comparison on Machine Learning Algorithms Based on Various Applications: A Comprehensive Survey. CSITSS 2021 - 2021 5th International Conference on Computational Systems and Information Technology for Sustainable Solutions, Proceedings. https://doi.org/10.1109/CSITSS54238.2021.9683524
Kazemzadeh, S., Yu, J., Jamshy, S., Pilgrim, R., Nabulsi, Z., Chen, C., Beladia, N., Lau, C., McKinney, S. M., Hughes, T., Kiraly, A. P., Kalidindi, S. R., Muyoyeta, M., Malemela, J., Shih, T., Corrado, G. S., Peng, L., Chou, K., Cameron Chen, P. H., … Prabhakara, S. (2023). Deep Learning Detection of Active Pulmonary Tuberculosis at Chest Radiography Matched the Clinical Performance of Radiologists. Radiology, 306(1), 124–137. https://doi.org/10.1148/RADIOL.212213/ASSET/IMAGES/LARGE/RADIOL.212213.FIG7.JPEG
Khanam, J. J., & Foo, S. Y. (2021). A comparison of machine learning algorithms for diabetes prediction. ICT Express, 7(4), 432–439. https://doi.org/10.1016/J.ICTE.2021.02.004
Kratsch, W., Manderscheid, J., Röglinger, M., & Seyfried, J. (2021). Machine Learning in Business Process Monitoring: A Comparison of Deep Learning and Classical Approaches Used for Outcome Prediction. Business and Information Systems Engineering, 63(3), 261–276. https://doi.org/10.1007/S12599-020-00645-0/TABLES/5
La RDC mise sur les campagnes de dépistage actif gratuit des cas au sein de la population pour réduire l’infection tuberculeuse latente | OMS | Bureau régional pour l’Afrique. (n.d.). Retrieved August 22, 2024, from https://www.afro.who.int/fr/countries/democratic-republic-of-congo/news/la-rdc-mise-sur-les-campagnes-de-depistage-actif-gratuit-des-cas-au-sein-de-la-population-pour
Lane, T. R., Foil, D. H., Minerali, E., Urbina, F., Zorn, K. M., & Ekins, S. (2021). Bioactivity Comparison across Multiple Machine Learning Algorithms Using over 5000 Datasets for Drug Discovery. Molecular Pharmaceutics, 18(1), 403–415. https://doi.org/10.1021/ACS.MOLPHARMACEUT.0C01013/SUPPL_FILE/MP0C01013_SI_001.PDF
Lane, T. R., Urbina, F., Rank, L., Gerlach, J., Riabova, O., Lepioshkin, A., Kazakova, E., Vocat, A., Tkachenko, V., Cole, S., Makarov, V., & Ekins, S. (2022). Machine Learning Models for Mycobacterium tuberculosis in Vitro Activity: Prediction and Target Visualization. Molecular Pharmaceutics, 19(2), 674–689. https://doi.org/10.1021/ACS.MOLPHARMACEUT.1C00791/SUPPL_FILE/MP1C00791_SI_002.ZIP
Liu, Y., Liang, Z., Yang, J., Yuan, S., Wang, S., Huang, W., & Wu, A. (2024). Diagnostic and comparative performance for the prediction of tuberculous pleural effusion using machine learning algorithms. International Journal of Medical Informatics, 182, 105320. https://doi.org/10.1016/J.IJMEDINF.2023.105320
Nabulsi, Z., Sellergren, A., Jamshy, S., Lau, C., Santos, E., Kiraly, A. P., Ye, W., Yang, J., Pilgrim, R., Kazemzadeh, S., Yu, J., Kalidindi, S. R., Etemadi, M., Garcia-Vicente, F., Melnick, D., Corrado, G. S., Peng, L., Eswaran, K., Tse, D., … Shetty, S. (2021). Deep learning for distinguishing normal versus abnormal chest radiographs and generalization to two unseen diseases tuberculosis and COVID-19. Scientific Reports 2021 11:1, 11(1), 1–15. https://doi.org/10.1038/s41598-021-93967-2
Olmez, E., Orhan, E., & Hiziroglu, A. (2021). Deep Learning in BioMedical Applications : Detection of Lung Disease with Convolutional Neural Networks. Deep Learning in Biomedical and Health Informatics, 97–115. https://doi.org/10.1201/9781003161233-5
Pathak, K. C., Kundaram, S. S., Sarvaiya, J. N., & Darji, A. D. (2022). Diagnosis and Analysis of Tuberculosis Disease Using Simple Neural Network and Deep Learning Approach for Chest X-Ray Images. Intelligent Systems Reference Library, 206, 77–102. https://doi.org/10.1007/978-3-030-76732-7_4
Rashidi, H. H., Dang, L. T., Albahra, S., Ravindran, R., & Khan, I. H. (2021). Automated machine learning for endemic active tuberculosis prediction from multiplex serological data. Scientific Reports 2021 11:1, 11(1), 1–12. https://doi.org/10.1038/s41598-021-97453-7
RDC : en finir avec la tuberculose grâce au dépistage précoce | OMS | Bureau régional pour l’Afrique. (n.d.). Retrieved August 22, 2024, from https://www.afro.who.int/fr/node/19369
Robinson, M. C., Glen, R. C., & Lee, A. A. (2020). Validating the validation: reanalyzing a large-scale comparison of deep learning and machine learning models for bioactivity prediction. Journal of Computer-Aided Molecular Design, 34(7), 717–730. https://doi.org/10.1007/S10822-019-00274-0/FIGURES/8
Sahlol, A. T., Elaziz, M. A., Jamal, A. T., Damaševičius, R., & Hassan, O. F. (2020). A Novel Method for Detection of Tuberculosis in Chest Radiographs Using Artificial Ecosystem-Based Optimisation of Deep Neural Network Features. Symmetry 2020, Vol. 12, Page 1146, 12(7), 1146. https://doi.org/10.3390/SYM12071146
Sathitratanacheewin, S., Sunanta, P., & Pongpirul, K. (2020). Deep learning for automated classification of tuberculosis-related chest X-Ray: dataset distribution shift limits diagnostic performance generalizability. Heliyon, 6(8). https://doi.org/10.1016/j.heliyon.2020.e04614
Sekeroglu, B., Hasan, S. S., & Abdullah, S. M. (2020). Comparison of Machine Learning Algorithms for Classification Problems. Advances in Intelligent Systems and Computing, 944, 491–499. https://doi.org/10.1007/978-3-030-17798-0_39
Sharifani, K., & Amini, M. (2023). Machine Learning and Deep Learning: A Review of Methods and Applications. https://papers.ssrn.com/abstract=4458723
Sharma, A., Machado, E., Lima, K. V. B., Suffys, P. N., & Conceição, E. C. (2022). Tuberculosis drug resistance profiling based on machine learning: A literature review. Brazilian Journal of Infectious Diseases, 26(1), 102332. https://doi.org/10.1016/J.BJID.2022.102332
Thapa, N., Liu, Z., Kc, D. B., Gokaraju, B., & Roy, K. (2020). Comparison of Machine Learning and Deep Learning Models for Network Intrusion Detection Systems. Future Internet 2020, Vol. 12, Page 167, 12(10), 167. https://doi.org/10.3390/FI12100167
Wang, P., Fan, E., & Wang, P. (2021). Comparative analysis of image classification algorithms based on traditional machine learning and deep learning. Pattern Recognition Letters, 141, 61–67. https://doi.org/10.1016/J.PATREC.2020.07.042
Ye, Q., Chai, X., Jiang, D., Yang, L., Shen, C., Zhang, X., Li, D., Cao, D., & Hou, T. (2021). Identification of active molecules against Mycobacterium tuberculosis through machine learning. Briefings in Bioinformatics, 22(5). https://doi.org/10.1093/BIB/BBAB068
Copyright (c) 2024 Gilgen Mate Landry, Rodolphe Nsimba Malumba, Fiston Chrisnovi Balanganayi Kabutakapua, Bopatriciat Boluma Mangata
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
The copyright of any article in the TECHNO Nusa Mandiri Journal is fully held by the author under the Creative Commons CC BY-NC license.
- The copyright in each article belongs to the author.
- Authors retain all their rights to published works, not limited to the rights set out on this page.
- The author acknowledges that Techno Nusa Mandiri: Journal of Computing and Information Technology (TECHNO Nusa Mandiri) is the first to publish with a Creative Commons Attribution 4.0 International license (CC BY-NC).
- Authors can enter articles separately, manage non-exclusive distribution, from manuscripts that have been published in this journal into another version (for example: sent to author affiliation respository, publication into books, etc.), by acknowledging that the manuscript was published for the first time in Techno Nusa Mandiri: Journal of Computing and Information Technology (TECHNO Nusa Mandiri);
- The author guarantees that the original article, written by the stated author, has never been published before, does not contain any statements that violate the law, does not violate the rights of others, is subject to the copyright which is exclusively held by the author.
- If an article was prepared jointly by more than one author, each author submitting the manuscript warrants that he has been authorized by all co-authors to agree to copyright and license notices (agreements) on their behalf, and agrees to notify the co-authors of the terms of this policy. Techno Nusa Mandiri: Journal of Computing and Information Technology (TECHNO Nusa Mandiri) will not be held responsible for anything that may have occurred due to the author's internal disputes.
.png){kind=logo}
