ENHANCING SENTIMENT ANALYSIS ACCURACY IN DELIVERY SERVICES THROUGH SYNTHETIC MINORITY OVERSAMPLING TECHNIQUE

  • Setia Sri Anggraeni (1) Universitas Nasional
  • Septi Andryana (2*) Universitas Nasional

  • (*) Corresponding Author
Keywords: random forest, sentiment analysis, smote, SVM

Abstract

Paxel is one of the delivery services that use the application. On Google Play, there are more than 10 thousand users leaving reviews. From this review data, a sentiment analysis was then carried out to determine the level of user satisfaction with Paxel's services. The methods used in this study are Random Forest (RF) and Support Vector Machine (SVM), as well as applying Synthetic Minority Oversampling Technique (SMOTE) to overcome data imbalance. The results showed that the method testing by dividing the data into two, namely training data and testing data by 80:20, stated that by applying the SMOTE, a higher accuracy value was obtained, where the accuracy of the RF method reached 91%, and the SVM method reached 87%. The level of user satisfaction with Paxel services tends to be neutral. This can be seen in the classification of the RF method with F1-Score values for the Positive class 89%, Neutral class 93%, and Negative class 92%.

Downloads

Download data is not yet available.

References

R. Nariswari, T. S. Prakoso, N. Hafiz, and H. Pudjihastuti, "Biplot analysis: a study of the change of customer behavior on e-commerce," Procedia Comput Sci, vol. 216, pp. 524–530, 2023, doi: 10.1016/j.procs.2022.12.165.

M. Kaplan, S. Hotle, and K. Heaslip, “How has COVID-19 impacted customer perceptions and demand for delivery services: An exploratory analysis,” Transp Policy (Oxf), vol. 134, pp. 217–230, Apr. 2023, doi: 10.1016/j.tranpol.2023.02.020.

F. J. Damanik and D. B. Setyohadi, "Analysis of public sentiment about COVID-19 in Indonesia on Twitter using multinomial naive Bayes and support vector machine," in IOP Conference Series: Earth and Environmental Science, IOP Publishing Ltd, Apr. 2021. doi: 10.1088/1755-1315/704/1/012027.

W. F. Satrya, R. Aprilliyani, and E. H. Yossy, “Sentiment analysis of Indonesian police chief using multi-level ensemble model,” Procedia Comput Sci, vol. 216, pp. 620–629, 2023, doi: 10.1016/j.procs.2022.12.177.

E. Hirata and T. Matsuda, “Examining logistics developments in post-pandemic Japan through sentiment analysis of Twitter data,” Asian Transport Studies, vol. 9, Jan. 2023, doi: 10.1016/j.eastsj.2023.100110.

S. Andryana, B. Rahman, A. Gunaryati, “Predicting Market Segments from Twitter Data Using Arima Time Series Analysis" JITK, vol. 9, August. 2023, doi: DOI: 10.33480/jitk.v9i1.4275.

K. M. Hasib, M. A. Habib, N. A. Towhid, and M. I. H. Showrov, “A Novel Deep Learning based Sentiment Analysis of Twitter Data for US Airline Service,” in 2021 International Conference on Information and Communication Technology for Sustainable Development, ICICT4SD 2021 - Proceedings, Institute of Electrical and Electronics Engineers Inc., Feb. 2021, pp. 450–455. doi: 10.1109/ICICT4SD50815.2021.9396879.

I. Spada, F. Chiarello, S. Barandoni, G. Ruggi, A. Martini, G. Fantoni, “Are universities ready to deliver digital skills and competences? A text mining-based case study of marketing courses in Italy”, Technological Forecasting and Social Change, Volume 182, 2022, 121869, ISSN 0040-1625, doi:10.1016/j.techfore.2022.121869.

Y. Hai Hoang, V. Minh Ngo, N. Bich Vu, “Central bank digital currency: A systematic literature review using text mining approach”, Research in International Business and Finance, Volume 64, 2023, 101889, ISSN 0275-5319,doi:10.1016/j.ribaf.2023.101889.

C. Villavicencio, J. J. Macrohon, X. A. Inbaraj, J. H. Jeng, and J. G. Hsieh, “Twitter sentiment analysis towards covid-19 vaccines in the Philippines using naïve bayes,” Information (Switzerland), vol. 12, no. 5, May 2021, doi: 10.3390/info12050204.

Hermanto, A. Y. Kuntoro, T. Asra, E. B. Pratama, L. Effendi, and R. Ocanitra, “Gojek and Grab User Sentiment Analysis on Google Play Using Naive Bayes Algorithm and Support Vector Machine Based Smote Technique,” in Journal of Physics: Conference Series, IOP Publishing Ltd, Nov. 2020. doi: 10.1088/1742-6596/1641/1/012102.

D. Sudigyo, A. A. Hidayat, R. Nirwantono, R. Rahutomo, J. P. Trinugroho, and B. Pardamean, “Literature study of stunting supplementation in Indonesian utilizing text mining approach,” Procedia Comput Sci, vol. 216, pp. 722–729, 2023, doi: 10.1016/j.procs.2022.12.189.

W. L. M. Leurs, L. A. S. Lammers, W. N. Compagner, M. Groeneveld, E. H. H. M. Korsten, and C. M. J. van der Linden, “Text mining in nursing notes for text characteristics associated with in-hospital falls in older adults: A case-control pilot study,” Aging Health Res, vol. 2, no. 2, p. 100078, Jun. 2022, doi: 10.1016/j.ahr.2022.100078.

H. Malik, E. M. Shakshuki, and A. U. H. Yasar, “Approximating Viewership of Streaming T.V Programs Using Social Media Sentiment Analysis,” in Procedia Computer Science, Elsevier B.V., 2021, pp. 94–101. doi: 10.1016/j.procs.2021.12.215.

D. Wang and Y. Zhao, “Using News to Predict Investor Sentiment: Based on SVM Model,” in Procedia Computer Science, Elsevier B.V., 2020, pp. 191–199. doi: 10.1016/j.procs.2020.06.074.

M. Demircan, A. Seller, F. Abut, and M. F. Akay, “Developing Turkish sentiment analysis models using machine learning and e-commerce data,” International Journal of Cognitive Computing in Engineering, vol. 2, pp. 202–207, Jun. 2021, doi: 10.1016/j.ijcce.2021.11.003.

M. Qorib, T. Oladunni, M. Denis, E. Ososanya, and P. Cotae, “Covid-19 vaccine hesitancy: Text mining, sentiment analysis and machine learning on COVID-19 vaccination Twitter dataset,” Expert Syst Appl, vol. 212, Feb. 2023, doi: 10.1016/j.eswa.2022.118715.

N. Putu Yulika Trisna Wijayanti1, E. N Kencana, I Wayan Sumarjaya, “SMOTE: Potensi dan Kekurangannya Pada Survei”, E-Jurnal Matematika Vol. 10(4), November 2021, DOI:https://doi.org/10.24843/MTK.2021.v10.i04.p348.

Published
2024-02-12
How to Cite
[1]
S. Sri Anggraeni and S. Andryana, “ENHANCING SENTIMENT ANALYSIS ACCURACY IN DELIVERY SERVICES THROUGH SYNTHETIC MINORITY OVERSAMPLING TECHNIQUE”, jitk, vol. 9, no. 2, pp. 257-262, Feb. 2024.
Article Metrics

Abstract viewed = 53 times
PDF downloaded = 23 times