OPTIMIZING TRANSFORMER-BASED LEARNING MODEL WITH TABTRANSFORMER FOR PREDICTING ANTIBIOTIC SUSCEPTIBILITY FROM MICROBIOLOGY MEDICAL RECORDS
DOI:
https://doi.org/10.33480/jitk.v11i3.7582Keywords:
Antibiotic Susceptibility Prediction, Antimicrobial Resistance (AMR), SHAP Interpretability Analysis, TabTransformer, Transformer-Based Deep LearningAbstract
Antimicrobial Resistance (AMR) has become a growing threat due to the increase in infections that are unresponsive to conventional therapies. Therefore, the development and optimization of Transformer-based Deep Learning using TabTransformer was employed to model the complex interactions between categorical features. This model was trained to predict antibiotic susceptibility at the individual culture level using the Antibiotic Resistance Microbiology Dataset (ARMD). To address the challenge of highly imbalanced data, the methodology applied includes extensive feature engineering to create historical and clinical variables, as well as the use of Focal Loss during training. After optimization, the final model demonstrated excellent discriminatory ability, with an Area Under the ROC Curve (AUC-ROC) of 0.93 and balanced classification performance, yielding a macro average F1-score of 0.82. Interpretability analysis using SHAP confirmed that patient clinical history and prior drug exposure were the most dominant predictive factors. These findings suggest that the Transformer-based Deep Learning architecture using TabTransformer, combined with clinically relevant feature engineering, can produce a reliable and evidence-based predictive tool
Downloads
References
[1] Y. Li, X. Cui, X. Yang, G. Liu, and J. Zhang, “Artificial intelligence in predicting pathogenic microorganisms’ antimicrobial resistance: challenges, progress, and prospects,” Front Cell Infect Microbiol, vol. 14, Nov. 2024, doi: 10.3389/fcimb.2024.1482186.
[2] F. Farhat, M. T. Athar, S. Ahmad, D. Ø. Madsen, and S. S. Sohail, “Antimicrobial resistance and machine learning: past, present, and future,” Front Microbiol, vol. Volume 14-2023, 2023, doi: 10.3389/fmicb.2023.1179312.
[3] A. Jain, G. R. Dabburu, B. Samanta, N. Singhal, and M. Kumar, “An explainable machine learning pipeline for prediction of antimicrobial resistance in Pseudomonas aeruginosa,” Bioinformatics Advances, vol. 5, no. 1, Dec. 2024, doi: 10.1093/bioadv/vbaf190.
[4] X. A. López-Cortés, J. M. Manríquez-Troncoso, R. Hernández-García, and D. Peralta, “MSDeepAMR: antimicrobial resistance prediction based on deep neural networks and transfer learning,” Front Microbiol, vol. 15, Apr. 2024, doi: 10.3389/fmicb.2024.1361795.
[5] J. S. Inda-Díaz et al., “Prediction of antibiotic resistance at the patient level using deep learning,” bioRxiv, p. 2023.05.09.539832, Jan. 2024, doi: 10.1101/2023.05.09.539832.
[6] S. Cannella et al., “Evaluation of EUCAST Rapid Antimicrobial Susceptibility Testing for Gram-Negative ESKAPEEc Pathogens in Blood Cultures, with a Focus on Carbapenemase-Producing Klebsiella pneumoniae in a University Hospital in Palermo, Italy,” Antibiotics, vol. 14, no. 12, p. 1251, Dec. 2025, doi: 10.3390/antibiotics14121251.
[7] A. Elbehiry et al., “Detection of antimicrobial resistance via state-of-the-art technologies versus conventional methods,” Front Microbiol, vol. Volume 16-2025, 2025, doi: 10.3389/fmicb.2025.1549044.
[8] N. Koenigstein, Transformers in Action. Manning Online, 2025.
[9] Y. Sun et al., “Efficient Attention Mechanisms for Large Language Models: A Survey,” Aug. 2025.
[10] G. Badaro, M. Saeed, and P. Papotti, “Transformers for Tabular Data Representation: A Survey of Models and Applications,” Trans Assoc Comput Linguist, vol. 11, pp. 227–249, Mar. 2023, doi: 10.1162/tacl_a_00544.
[11] F. , A. F. , & M. M. Nateghi Haredasht, “Antibiotic Resistance Microbiology Dataset (ARMD): A de-identified resource for studying antimicrobial resistance using electronic health records. ,” 2025.
[12] D. Rifaldi, Abdul Fadlil, and Herman, “Teknik Preprocessing Pada Text Mining Menggunakan Data Tweet ‘Mental Health,’” Decode: Jurnal Pendidikan Teknologi Informasi, vol. 3, no. 2, pp. 161–171, Apr. 2023, doi: 10.51454/decode.v3i2.131.
[13] P. Kamath B., G. Sharma, A. Bongale, D. Dharrao, and M. Seitshiro, “Exploratory Data Analysis and Water Potability Classification using Supervised Machine Learning Algorithms,” Engineering, Technology & Applied Science Research, vol. 15, no. 2, pp. 20898–20903, Apr. 2025, doi: 10.48084/etasr.8904.
[14] K. Maharana, S. Mondal, and B. Nemade, “A review: Data pre-processing and data augmentation techniques,” Global Transitions Proceedings, vol. 3, no. 1, pp. 91–99, Jun. 2022, doi: 10.1016/j.gltp.2022.04.020.
[15] J. , P. J. , & K. M. Han, Data mining: Concepts and techniques, 4th ed. Morgan Kaufmann, 2022.
[16] F. Xie et al., “Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies,” J Biomed Inform, vol. 126, p. 103980, Feb. 2022, doi: 10.1016/j.jbi.2021.103980.
[17] J. Xu, X. Xi, J. Chen, V. S. Sheng, J. Ma, and Z. Cui, “A Survey of Deep Learning for Electronic Health Records,” Applied Sciences, vol. 12, no. 22, p. 11709, Nov. 2022, doi: 10.3390/app122211709.
[18] G. S. Hida and A. C. Alves Do Nascimento, “Overview of machine learning in class imbalance scenarios: Trends, challenges, and approaches,” Expert Syst Appl, vol. 298, p. 129592, Mar. 2026, doi: 10.1016/j.eswa.2025.129592.
[19] Y. Shi, P. Wei, K. Feng, D.-C. Feng, and M. Beer, “A survey on machine learning approaches for uncertainty quantification of engineering systems,” Machine Learning for Computational Science and Engineering, vol. 1, no. 1, p. 11, Jun. 2025, doi: 10.1007/s44379-024-00011-x.
[20] X. Qian, S. Gao, W. Deng, and W. Wang, “Improving Oriented Object Detection by Scene Classification and Task-Aligned Focal Loss,” Mathematics, vol. 12, no. 9, p. 1343, Apr. 2024, doi: 10.3390/math12091343.
[21] Y. Shao et al., “An Improved BGE-Adam Optimization Algorithm Based on Entropy Weighting and Adaptive Gradient Strategy,” Symmetry (Basel), vol. 16, no. 5, p. 623, May 2024, doi: 10.3390/sym16050623.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Feri Sulianta, Endang Amalia, Rosalin Samiharjo, Noval Eka Herdinata
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
-a.jpg)
-b.jpg)
