DIGITALLY FILE EXTRACTION OPTIMISED WITH GPT-4O BASED MOBILE APPLICATION FOR RELEVANT EXERCISE PROBLEM GENERATION
DOI:
https://doi.org/10.33480/jitk.v10i3.6101Keywords:
digital file extraction, GPT-4o model, practice question, rouge metrics, question generationAbstract
This research studies the creation of an AI-driven question extraction system using the GPT-4o model to improve the accessibility and variety of practice questions for students. The study tackles the difficulties in sourcing relevant practice materials and aims to transform educational technology by integrating mobile learning. A mobile application was built with Dart and Flutter, designed to extract questions from PDF files. The system is capable of generating both multiple-choice and essay questions across different difficulty levels. The quality and relevance of the generated questions were assessed using ROUGE metrics. The results indicated strong performance for multiple-choice questions, especially in single-answer and true/false formats. However, the system encountered difficulties in producing complex essay questions, highlighting the need for further improvements in understanding intricate contextual relationships. Key findings reveal effective generation of multiple-choice questions with high precision and recall; inconsistent performance in essay question generation, with simpler questions yielding better results; and ROUGE-1 metrics surpassing ROUGE-2 and ROUGE-L, indicating a stronger ability to generate straightforward questions. The research concludes that while the developed system shows potential in enhancing educational resources, additional research is necessary to refine complex question generation. Recommendations include broadening the training dataset and creating specialized models for question generation tasks to enhance the effectiveness of AI-assisted learning tools.
Downloads
References
F. M. Sinaga, S. Irviantina, and S. J. Pipin, “Pelatihan Pembuatan Konten Pembelajaran Berbasis Video pada SMA Methodist 6,” Journal of Social Responsibility Projects by Higher Education Forum, vol. 4, no. 3, pp. 139–144, Mar. 2024, doi: 10.47065/jrespro.v4i3.4588.
M. Montenegro-Rueda, J. Fernández-Cerero, J. M. Fernández-Batanero, and E. López-Meneses, “Impact of the Implementation of ChatGPT in Education: A Systematic Review,” Computers, vol. 12, no. 8, p. 153, Jul. 2023, doi: 10.3390/computers12080153.
P. Reilly, Z. Al Khuridah, R. Cooper, L. L. Hsu, and C. Hentea, “Commercial Question Banks Present a Limited Scope of Sickle Cell Disease,” Blood, vol. 140, no. Supplement 1, pp. 10779–10780, Nov. 2022, doi: 10.1182/blood-2022-162751.
M. L. Rivers, “Test Experience, Direct Instruction, and Their Combination Promote Accurate Beliefs about the Testing Effect,” Journal of Intelligence, vol. 11, no. 7, p. 147, Jul. 2023, doi: 10.3390/jintelligence11070147.
H. Geng and H. Wei, “Exploring ChatGPT’s Capabilities in Creative Writing: Can GPT-4o Conduct Rhetorical Move Analysis in Narrative Short Stories?,” ASEAN Journal of Applied Linguistics, vol. 3, no. 1, pp. 44–59, 2024.
S. Donthi et al., “Improving LLM Abilities in Idiomatic Translation,” Jul. 2024, doi: 10.48550/arXiv.2407.03518.
S.-V. Fulgencio, “Developing Effective Educational Chatbots with GPT: Insights from a Pilot Study in a University Subject,” Trends in Higher Education, vol. 3, no. 1, pp. 155–168, Mar. 2024, doi: 10.3390/higheredu3010009.
J. Savelka, A. Agarwal, C. Bogart, Y. Song, and M. Sakr, “Can Generative Pre-trained Transformers (GPT) Pass Assessments in Higher Education Programming Courses?,” Proceedings of the 2023 Conference on Innovation and Technology in Computer Science Education V. 1, pp. 117–123, Jun. 2023, doi: 10.1145/3587102.3588792.
W. Suharmawan, “Pemanfaatan Chat GPT Dalam Dunia Pendidikan,” Education Journal : Journal Educational Research and Development, vol. 7, no. 2, pp. 158–166, 2023, doi: 10.31537/ej.v7i2.1248.
M. Mustafa, “Aktivitas Siswa dalam Memecahkan Masalah Matematika dengan Berpikir Komputasi Berbantuan Chat-GPT,” MATHEMA: JURNAL PENDIDIKAN MATEMATIKA, vol. 5, no. 2, pp. 283–298, Oct. 2023, doi: 10.33365/jm.v5i2.3469.
A. Setiawan and U. K. Luthiyani, “Penggunaan ChatGPT Untuk Pendidikan di Era Education 4.0: Usulan Inovasi Meningkatkan Keterampilan Menulis ,” Jurnal PETISI (Pendidikan Teknologi Informasi), vol. 04, no. 01, pp. 49–58, Feb. 2023.
S. Bedi et al., “QUEST-AI: A System for Question Generation, Verification, and Refinement using AI for USMLE-Style Exams,” Biocomputing 2025, pp. 54–69, Nov. 2024, doi: 10.1142/9789819807024_0005.
I. Hsiao and C.-Y. Chung, “AI-Infused Semantic Model to Enrich and Expand Programming Question Generation,” Journal of Artificial Intelligence and Technology, vol. 2, no. 2, Mar. 2022, doi: 10.37965/jait.2022.0090.
M. Rathod, T. Tu, and K. Stasaski, “Educational Multi-Question Generation for Reading Comprehension,” Proceedings of the 17th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2022), 2022, doi: 10.18653/v1/2022.bea-1.26.
B. Ananda and S. Suranto, “Transformasi Pembelajaran di Sekolah Menengah Kejuruan: Analisis Mendalam Fleksibilitas M-learning,” Ideguru: Jurnal Karya Ilmiah Guru, vol. 9, no. 2, pp. 695–701, Jan. 2024, doi: 10.51169/ideguru.v9i2.936.
B. Jerome, R. Van Campenhout, and B. G. Johnson, “Automatic Question Generation and the SmartStart Application,” Proceedings of the Eighth ACM Conference on Learning @ Scale, pp. 365–366, Jun. 2021, doi: 10.1145/3430895.3460878.
E. Kristanti, G. I. Kharisma, and N. P. Sari, “Pelatihan Penyusunan Soal Berbasis Mobile Learning Sebagai Upaya Menghadapi Era Pendidikan 4.0,” WIDYA LAKSANA, vol. 10, no. 1, pp. 59–65, Mar. 2021, doi: 10.23887/jwl.v10i1.28915.
A. Widyatama and F. W. Pratama, “Pengembangan Mobile Learning PINTHIR Berbasis Android sebagai Sumber Belajar dan Sarana Mengerjakan Soal Trigonometri SMA,” Mosharafa: Jurnal Pendidikan Matematika, vol. 11, no. 1, pp. 25–36, Jan. 2022, doi: 10.31980/mosharafa.v11i1.684.
Halimah, S. Agustian, and S. Ramadhani, “Peringkasan Teks Otomatis (Automated Text Summarization) Pada Artikel Berbahasa indonesia Menggunakan Algoritma Lexrank,” Jurnal Computer Science and Information Technology (CoSciTech), vol. 3, no. 3, pp. 371–381, Dec. 2022.
A. Yogi Setiawan, I. G. Mahendra Darmawiguna, and G. Aditra Pradnyana, “Sentiment Summarization Evaluasi Pembelajaran Menggunakan Algoritma LSTM (Long Short Term Memory),” Kumpulan Artikel Mahasiswa Pendidikan Teknik Informatika (KARMAPATI) , vol. 11, no. 2, pp. 183–191, Aug. 2022.
F. Noprianto, S. Agustian, and M. Irsyad, “Clustering Peringkasan Teks Otomatis Dokumen Berita Menggunakan Metode K-Means,” in Sendiko - Prosiding Seminar Nasional Hasil Penelitian dan Pengabdian Masyarakat Bidang Ilmu Komputer, R. Pamungkas, Saifulloh, and Andria, Eds., Madiun: Universitas PGRI Madiun, Jun. 2023, pp. 139–147.
M. Barbella and G. Tortora, “Rouge Metric Evaluation for Text Summarization Techniques,” SSRN Electronic Journal, May 2022, doi: 10.2139/ssrn.4120317.
R. Harang, “Beyond ROUGE: Applying an ELO algorithm to rank model performances in summarization,” In Proceedings of the 30th Annual Meeting of the Association for Natural Language Processing, pp. 2799-2804, 2024.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Syanti Irviantina, Hernawati Gohzaly, Dustin Lionel, Peter Fomas Hia
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
-a.jpg)
-b.jpg)
