KNOWLEDGE DISCOVERY OF PUBLIC SATISFACTION ON E-HEALTH PLATFORM THROUGH ENSEMBLE LEARNING AND THEMATIC ANALYSIS
DOI:
https://doi.org/10.33480/jitk.v12i1.7035Keywords:
Ensemble Learning, ISO/IEC 25010, SATUSEHAT, Sentiment Analysis, Thematic AnalysisAbstract
As Indonesia’s national digital health platform, the SATUSEHAT application is central to the country's healthcare integration. However, maintaining user satisfaction remains a challenge. This study evaluates public sentiment by analyzing over 30,000 Google Play Store reviews using an ensemble approach of machine learning and thematic analysis. To address substantial data imbalance, five base classifiers were compared with three ensemble models using Random Under-Sampling and SMOTE. Results indicate that the Stacking Ensemble with SMOTE outperformed all other models, achieving an Accuracy of 91.1% and an AUC-ROC of 0.919. Sentiment analysis reveals a critical dissatisfaction rate, with 83.3% of reviews being negative. By mapping user complaints to the ISO/IEC 25010 software quality framework, it was identified that functional suitability and reliability account for 80.7% of the reported issues. This research contributes a robust methodology for Indonesian-language sentiment analysis and demonstrates the utility of ISO/IEC 25010 in translating raw user feedback into actionable software engineering requirements. Practically, the findings provide the Ministry of Health with an evidence-based roadmap to resolve critical barriers in authentication, OTP delivery, and certificate retrieval.
Downloads
References
[1] Kementerian Kesehatan Republik Indonesia, “Keputusan Menteri Kesehatan Republik Indonesia Nomor HK.01.07/MENKES/33/2025 tentang Petunjuk Teknis Pemeriksaan Kesehatan Gratis Hari Ulang Tahun,” Jakarta, 2025.
[2] Presiden Republik Indonesia, “Peraturan Presiden Republik Indonesia Nomor 95 Tahun 2018 tentang Sistem Pemerintahan Berbasis Elektronik,” Jakarta, 2018.
[3] R. Filip, R. Gheorghita Puscaselu, L. Anchidin-Norocel, M. Dimian, and W. K. Savage, “Global Challenges to Public Health Care Systems during the COVID-19 Pandemic: A Review of Pandemic Measures and Problems,” J. Pers. Med., vol. 12, no. 8, p. 1295, Aug. 2022, doi: 10.3390/jpm12081295.
[4] Kementerian Kesehatan Republik Indonesia, Cetak Biru Strategi Transformasi Digital Kesehatan, vol. 1. Jakarta, 2021.
[5] Kementerian Kesehatan Republik Indonesia, “Peraturan Menteri Kesehatan Nomor 21 Tahun 2020 tentang Rencana Strategis Kementerian Kesehatan Tahun 2020-2024,” Indonesia, 2020.
[6] Direktorat Promosi Kesehatan dan Pemberdayaan Masyarakat Kemenkes RI, “PeduliLindungi Resmi Berubah Menjadi SATUSEHAT,” Promkes.kemkes.go.id. Accessed: Jun. 23, 2025. [Online]. Available: https://promkes.kemkes.go.id/pedulilindungi-resmi-berubah-menjadi-satusehat
[7] H. Melani Puspasari, I. Zharif Mustaqim, A. Tri Utami, R. Syalevi, and Y. Ruldeviyani, “Evaluation of Indonesia’s police public service platforms through sentiment and thematic analysis,” IAES International Journal of Artificial Intelligence (IJ-AI), vol. 13, no. 2, p. 1596, Jun. 2024, doi: 10.11591/ijai.v13.i2.pp1596-1607.
[8] N. Rizun, A. Revina, and N. Edelmann, “Application of Text Analytics in Public Service Co-Creation: Literature Review and Research Framework,” in Proceedings of the 24th Annual International Conference on Digital Government Research, New York, NY, USA: ACM, Jul. 2023, pp. 12–22. doi: 10.1145/3598469.3598471.
[9] H. D. Sharma and P. Goyal, “An Analysis of Sentiment: Methods, Applications, and Challenges,” in RAiSE-2023, Basel Switzerland: MDPI, Dec. 2023, p. 68. doi: 10.3390/engproc2023059068.
[10] V. Novalia, K. Ditha Tania, A. Meiriza, and A. Wedhasmara, “Knowledge Discovery of Application Review Using Word Embedding’s Comparison with CNN-LSTM Model on Sentiment Analysis,” in 2024 International Conference on Electrical Engineering and Computer Science (ICECOS), IEEE, Sep. 2024, pp. 234–238. doi: 10.1109/ICECOS63900.2024.10791113.
[11] D. Khoiriyah Harahap, K. Ditha Tania, and P. Eka Sevtiyuni, “Topic Mining-Based Knowledge Discovery of User Health Information Needs,” Indonesian Journal of Electronics, Electromedical Engineering, and Medical Informatics, vol. 7, no. 4, pp. 641–653, Oct. 2025, doi: 10.35882/ijeeemi.v7i4.270.
[12] D. A. Wulandari, F. A. Bachtiar, and I. Indriati, “Aspect Based Sentiment Analysis on Shopee Application Reviews Using Support Vector Machine,” Lontar Komputer : Jurnal Ilmiah Teknologi Informasi, vol. 15, no. 02, p. 99, Jan. 2025, doi: 10.24843/LKJITI.2024.v15.i02.p03.
[13] Raksaka Indra Alhaqq, I Made Kurniawan Putra, and Yova Ruldeviyani, “Analisis Sentimen terhadap Penggunaan Aplikasi MySAPK BKN di Google Play Store,” Jurnal Nasional Teknik Elektro dan Teknologi Informasi, vol. 11, no. 2, pp. 105–113, May 2022, doi: 10.22146/jnteti.v11i2.3528.
[14] B. Andrian, T. Simanungkalit, I. Budi, and A. F. Wicaksono, “Sentiment Analysis on Customer Satisfaction of Digital Banking in Indonesia,” International Journal of Advanced Computer Science and Applications, vol. 13, no. 3, 2022, doi: 10.14569/IJACSA.2022.0130356.
[15] E. Junianto, M. Puspitasari, S. I. Zakaria, T. Arifin, and I. W. P. Agung, “Klasifikasi Emosi pada Teks Berbahasa Inggris Menggunakan Pendekatan Ensemble Bagging,” Jurnal Nasional Teknik Elektro dan Teknologi Informasi, vol. 13, no. 4, pp. 272–281, Nov. 2024, doi: 10.22146/jnteti.v13i4.14440.
[16] H. Herianto, “Machine Learning Algorithm Optimization using Stacking Technique for Graduation Prediction,” Journal of Applied Data Sciences, vol. 5, no. 3, pp. 1272–1285, Sep. 2024, doi: 10.47738/jads.v5i3.316.
[17] T. A. Putra, V. Ariandi, and S. Defit, “Enhancing Accuracy by Using Boosting and Stacking Techniques on the Random Forest Algorithm on Data from Social Media X,” ILKOM Jurnal Ilmiah, vol. 16, no. 2, pp. 184–189, Aug. 2024, doi: 10.33096/ilkom.v16i2.2058.184-189.
[18] I. Zharif Mustaqim, H. Melani Puspasari, A. Tri Utami, R. Syalevi, and Y. Ruldeviyani, “Assessing public satisfaction of public service application using supervised machine learning,” IAES International Journal of Artificial Intelligence (IJ-AI), vol. 13, no. 2, p. 1608, Jun. 2024, doi: 10.11591/ijai.v13.i2.pp1608-1618.
[19] P. Lee et al., “Digital Health COVID-19 Impact Assessment: Lessons Learned and Compelling Needs,” NAM Perspectives, Jan. 2022, doi: 10.31478/202201c.
[20] J. H. Joloudari, A. Marefat, M. A. Nematollahi, S. S. Oyelere, and S. Hussain, “Effective Class-Imbalance Learning Based on SMOTE and Convolutional Neural Networks,” Applied Sciences, vol. 13, no. 6, p. 4006, Mar. 2023, doi: 10.3390/app13064006.
[21] V. Lumumba, D. Kiprotich, M. Mpaine, N. Makena, and M. Kavita, “Comparative Analysis of Cross-Validation Techniques: LOOCV, K-folds Cross-Validation, and Repeated K-folds Cross-Validation in Machine Learning Models,” American Journal of Theoretical and Applied Statistics, vol. 13, no. 5, pp. 127–137, Oct. 2024, doi: 10.11648/j.ajtas.20241305.13.
[22] C. Dewi, R.-C. Chen, H. J. Christanto, and F. Cauteruccio, “Multinomial Naïve Bayes Classifier for Sentiment Analysis of Internet Movie Database,” Vietnam Journal of Computer Science, vol. 10, no. 04, pp. 485–498, Nov. 2023, doi: 10.1142/S2196888823500100.
[23] S. K. Ahmed et al., “Using thematic analysis in qualitative research,” Journal of Medicine, Surgery, and Public Health, vol. 6, p. 100198, Aug. 2025, doi: 10.1016/j.glmedi.2025.100198.
[24] B. I. Rumabar and E. Maria, “Evaluasi Kualitas Shopeepay Menggunakan ISO/IEC 25010,” Jurnal Sistem Informasi Bisnis, vol. 14, no. 1, pp. 54–61, Jan. 2024, doi: 10.21456/vol14iss1pp54-61.
[25] J. Kazmaier and J. H. van Vuuren, “The power of ensemble learning in sentiment analysis,” Expert Syst. Appl., vol. 187, p. 115819, Jan. 2022, doi: 10.1016/j.eswa.2021.115819.
[26] K. Madatov, S. Sattarova, and J. Vičič, “TF-IDF-Based Classification of Uzbek Educational Texts,” Applied Sciences, vol. 15, no. 19, p. 10808, Oct. 2025, doi: 10.3390/app151910808.
[27] S. F. Taskiran, B. Turkoglu, E. Kaya, and T. Asuroglu, “A comprehensive evaluation of oversampling techniques for enhancing text classification performance,” Sci. Rep., vol. 15, no. 1, p. 21631, Jul. 2025, doi: 10.1038/s41598-025-05791-7.
[28] N. Nasaruddin, N. Masseran, W. M. R. Idris, and A. Z. Ul-Saufie, “A SMOTE PCA HDBSCAN approach for enhancing water quality classification in imbalanced datasets,” Sci. Rep., vol. 15, no. 1, p. 13059, Apr. 2025, doi: 10.1038/s41598-025-97248-0.
[29] Hafidz. Firdaus and Azizah. Zakiah, “Implementation of Usability Testing Methods to Measure the Usability Aspect of Management Information System Mobile Application (Case Study Sukamiskin Correctional Institution),” International Journal of Modern Education and Computer Science, vol. 13, no. 5, pp. 58–67, Oct. 2021, doi: 10.5815/ijmecs.2021.05.06.
[30] Y. Xu, J. Zhang, R. Chi, and G. Deng, “Enhancing customer satisfaction with chatbots: the influence of anthropomorphic communication styles and anthropomorphised roles,” Nankai Business Review International, vol. 14, no. 2, pp. 249–271, Jun. 2023, doi: 10.1108/NBRI-06-2021-0041.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Syakillah Nachwa, Ken Ditha Tania, Naretha Kawadha Pasemah Gumay
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
-a.jpg)
-b.jpg)
