DEEP BELIEF NETWORK (DBN) IMPLEMENTATION FOR MULTIMODAL CLASSIFICATION OF SENTIMENT ANALYSIS
DOI:
https://doi.org/10.33480/jitk.v10i3.6257Keywords:
deep belief network, multimodal, sentiment analysisAbstract
In sentiment analysis, the use of multimodal data, consisting of a combination of images and text, is becoming increasingly important for understanding digital context. However, the main challenge lies in effectively integrating these two types of data into a single learning model. Deep Belief Network (DBN), with its capability to learn hierarchical data representations, is utilized to explore optimal strategies for multimodal sentiment analysis. The dataset includes 34,034 images from the FERPlus dataset to train the model in classifying emotions based on facial expressions, as well as 999 text and image samples obtained through crawling X. Experiments were conducted by comparing the performance of DBN with 2, 3, and 4 hidden layers across different test data sizes (10%-50%). The results indicate that the 3-hidden-layer configuration achieved the best performance, with a highest accuracy of 76% at a 20% test data size. Additionally, testing different learning rates (10⁻⁴ to 10⁻⁷) produced consistent results, but the fastest computation time was achieved with a learning rate of 10⁻⁴. Based on these findings, DBN with a 3-hidden-layer configuration and a learning rate of 10⁻⁴ is considered a more efficient alternative for multimodal sentiment analysis based on text and images.
Downloads
References
K. Singh, “The Role of Mathematics in Artificial Intelligence and Machine Learning,” International Journal for Research Publication and Seminar, vol. 14, no. 5, pp. 186–197, Dec. 2023, doi: 10.36676/jrps.v14.i5.1434.
S. Zhao, F. Blaabjerg, and H. Wang, “An Overview of Artificial Intelligence Applications for Power Electronics,” IEEE Transactions on Power Electronics, vol. 36, no. 4, pp. 4633–4658, Apr. 2021, doi: 10.1109/tpel.2020.3024914.
N. Parveen, P. Chakrabarti, B. T. Hung, and A. Shaik, “Twitter sentiment analysis using hybrid gated attention recurrent network,” Journal of Big Data, vol. 10, no. 1, Apr. 2023, doi: 10.1186/s40537-023-00726-3.
A. Daza, N. D. González Rueda, M. S. Aguilar Sánchez, W. F. Robles Espíritu, and M. E. Chauca Quiñones, “Sentiment Analysis on E-Commerce Product Reviews Using Machine Learning and Deep Learning Algorithms: A Bibliometric Analysis, Systematic Literature Review, Challenges and Future Works,” International Journal of Information Management Data Insights, vol. 4, no. 2, p. 100267, Nov. 2024, doi: 10.1016/j.jjimei.2024.100267.
P. Subarkah, B. A. Kusuma, and P. Arsi, “SENTIMENT ANALYSIS ON RENEWABLE ENERGY ELECTRIC USING SUPPORT VECTOR MACHINE (SVM) BASED OPTIMIZATION,” JITK (Jurnal Ilmu Pengetahuan dan Teknologi Komputer), vol. 10, no. 2, pp. 252–260, Nov. 2024, doi: 10.33480/jitk.v10i2.5575.
Q. A. Xu, V. Chang, and C. Jayne, “A systematic review of social media-based sentiment analysis: Emerging trends and challenges,” Decision Analytics Journal, vol. 3, p. 100073, Jun. 2022, doi: 10.1016/j.dajour.2022.100073.
A. P. Rodrigues et al., “Real-Time Twitter Spam Detection and Sentiment Analysis using Machine Learning and Deep Learning Techniques,” Computational Intelligence and Neuroscience, vol. 2022, pp. 1–14, Apr. 2022, doi: 10.1155/2022/5211949.
R. A. Rudiyanto and E. B. Setiawan, “Sentiment Analysis Using Convolutional Neural Network (CNN) and Particle Swarm Optimization on Twitter,” JITK (Jurnal Ilmu Pengetahuan dan Teknologi Komputer), vol. 9, no. 2, pp. 188–195, 2024, doi: 10.33480/jitk.v9i2.5201.
Y. Wang, J. Guo, C. Yuan, and B. Li, “Sentiment Analysis of Twitter Data,” Applied Sciences, vol. 12, no. 22, p. 11775, Nov. 2022, doi: 10.3390/app122211775.
I. K. S. Al-Tameemi, M.-R. Feizi-Derakhshi, S. Pashazadeh, and M. Asadpour, “A comprehensive review of visual–textual sentiment analysis from social media networks,” Journal of Computational Social Science, vol. 7, no. 3, pp. 2767–2838, Sep. 2024, doi: 10.1007/s42001-024-00326-y.
A. Gandhi, K. Adhvaryu, S. Poria, E. Cambria, and A. Hussain, “Multimodal sentiment analysis: A systematic review of history, datasets, multimodal fusion methods, applications, challenges and future directions,” Information Fusion, vol. 91, pp. 424–444, Mar. 2023, doi: 10.1016/j.inffus.2022.09.025.
C. G. Maurya and S. K. Jha, “Sentiment Analysis: A Hybrid Approach on Twitter Data,” Procedia Computer Science, vol. 235, pp. 990–999, 2024, doi: 10.1016/j.procs.2024.04.094.
Z. Luo, Z. Tao, Y. Huang, K. He, and C. Liu, “MC-DBN: A Deep Belief Network-Based Model for Modality Completion,” 2024, [Online]. Available: http://arxiv.org/abs/2402.09782
R. Song, Z. Wang, L. Guo, F. Zhao, and Z. Xu, “Deep Belief Networks (DBN) for Financial Time Series Analysis and Market Trends Prediction,” World Journal of Innovation and Modern Technology, vol. 7, no. 4, pp. 1–10, Jul. 2024, doi: 10.53469/wjimt.2024.07(04).01.
M. Scarpiniti, F. Colasante, S. Di Tanna, M. Ciancia, Y.-C. Lee, and A. Uncini, “Deep Belief Network based audio classification for construction sites monitoring,” Expert Systems with Applications, vol. 177, p. 114839, Sep. 2021, doi: 10.1016/j.eswa.2021.114839.
I. Sohn, “Deep belief network based intrusion detection techniques: A survey,” Expert Systems with Applications, vol. 167, p. 114170, Apr. 2021, doi: 10.1016/j.eswa.2020.114170.
N. Zeng, H. Li, and Y. Peng, “A new deep belief network-based multi-task learning for diagnosis of Alzheimer’s disease,” Neural Computing and Applications, vol. 35, no. 16, pp. 11599–11610, Jun. 2021, doi: 10.1007/s00521-021-06149-6.
R. B. Hadiprakoso, H. Setiawan, R. N. Yasa, and Girinoto, “Text Preprocessing for Optimal Accuracy in Indonesian Sentiment Analysis Using a Deep Learning Model with Word Embedding,” AIP Conference Proceedings, vol. 2680, no. 1, 2023, doi: 10.1063/5.0126116.
T. Ghosh and S. K. B. Math, : A Concise yet In-Depth Guide on Fundamentals of Computer Vision, NLP, Complex Deep Neural Networks and Machine Learning (English Edition). India: BPB publications. 2022.
S. Sarica and J. Luo, “Stopwords in technical language processing,” PLOS ONE, vol. 16, no. 8, p. e0254937, Aug. 2021, doi: 10.1371/journal.pone.0254937.
R. Pramana, Debora, J. J. Subroto, A. A. S. Gunawan, and Anderies, “Systematic Literature Review of Stemming and Lemmatization Performance for Sentence Similarity,” 2022 IEEE 7th International Conference on Information Technology and Digital Applications (ICITDA), pp. 1–6, Nov. 2022, doi: 10.1109/icitda55840.2022.9971451.
M. Chiny, M. Chihab, O. Bencharef, and Y. Chihab, “LSTM, VADER and TF-IDF based Hybrid Sentiment Analysis Model,” International Journal of Advanced Computer Science and Applications, vol. 12, no. 7, 2021, doi: 10.14569/ijacsa.2021.0120730.
K. Maharana, S. Mondal, and B. Nemade, “A review: Data pre-processing and data augmentation techniques,” Global Transitions Proceedings, vol. 3, no. 1, pp. 91–99, Jun. 2022, doi: 10.1016/j.gltp.2022.04.020.
S. Surono, M. Y. F. Afitian, A. Setyawan, D. K. Eni Arofah, and A. Thobirin, “Comparison of CNN Classification Model using Machine Learning with Bayesian Optimizer,” HighTech and Innovation Journal, vol. 4, no. 3, pp. 531–542, Sep. 2023, doi: 10.28991/hij-2023-04-03-05.
S. Surono, M. Rivaldi, D. A. Dewi, and N. Irsalinda, “New Approach to Image Segmentation: U-Net Convolutional Network for Multiresolution CT Image Lung Segmentation,” Emerging Science Journal, vol. 7, no. 2, pp. 498–506, Feb. 2023, doi: 10.28991/esj-2023-07-02-014.
X. Zhang, N. Gu, J. Chang, and H. Ye, “Predicting stock price movement using a DBN-RNN,” Applied Artificial Intelligence, vol. 35, no. 12, pp. 876–892, Aug. 2021, doi: 10.1080/08839514.2021.1942520.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Hilmi Hibatullah, Aris Thobirin, Sugiyarto Surono
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
-a.jpg)
-b.jpg)
