USER EXPERIENCE ANSWER SYSTEM AUTOMATICALLY WITH USER CENTERED DESIGN AND USER EXPERIENCE QUESTIONNAIRE-SHORT
Abstract
The interface of the automatic answer assessment system is plagued by several issues, including an unfamiliar layout, unresponsive design, inconsistency in elements, and a lack of clarity in presenting information. These problems significantly hinder the overall user experience. As a result, this study aimed to address these challenges by designing a user-centered experience for the automatic answer assessment system, using a high-fidelity prototype tailored to meet user needs. The user-centered design method involved four crucial stages: Specify The Context Of Use, Specify Requirements, Create Designs, and Evaluate Designs. Through rigorous usability testing with teachers, the design achieved an impressive effectiveness rating of 90%, firmly establishing it as a "very effective" solution. Additionally, it demonstrated high efficiency with a value of 0.01307 goals/sec, and teachers expressed positive feedback, confirming the satisfaction and usability of the new interface. Similarly, students' usability testing yielded noteworthy results, with a 90% effectiveness rating, also classified as "very effective." The interface showcased a high level of efficiency, with a value of 0.0849 goals/sec. While the satisfaction value fell below the PSSUQ norm, students still found the interface to be user-friendly and satisfactory. Furthermore, the user experience testing, utilizing the UEQ-S, provided valuable insights. For teachers, the pragmatic aspect scored 1.85, the hedonic aspect scored 2.33, and the overall aspect received a commendable score of 2.09, all of which fell within the excellent category on benchmarks. Similarly, students' ratings were highly positive, with scores of 2.14 for both pragmatic and hedonic aspects, and an overall score of 2.14, signifying an excellent user experience.The retrospective think-aloud validation test reaffirmed the positive response from prospective users. Overall, this research, employing a user-centered design approach, successfully delivered a highly satisfactory and effective user experience for both teachers and students using the automatic answer assessment system.
Downloads
References
M. Rohmah, F. A. Muqtadiroh, and A. S. Nisafani, “Analisis Faktor Penyebab Kegagalan Proses Requirement Engineering pada Pengembangan Perangkat Lunak Menggunakan Metode Fault Tree Analysis: Studi Kasus Proyek Pemerintah Skala Kecil,” repository.its.ac.id, p. 153, 2017.
S. R. Henim and R. P. Sari, “User Experience Evaluation of Student Academic Information System of Higher Education Using User Experience Questionnaire,” J. Komput. Terap., vol. 6, no. Vol. 6 No. 1 (2020), pp. 69–78, 2020, doi: 10.35143/jkt.v6i1.3582.
D. R. Rohmasari and R. Miharti, “Kebutuhan Pengguna dalam Perancangan Disain Antarmuka SIMRS RSU ‘Aisyiyah Ponorogo,” J. Kesehat. Vokasional, vol. 3, no. 2, p. 81, 2018, doi: 10.22146/-.38563.
R. T. Haniifah, R. Aisy, K. Candra Brata, and H. Muslimah Az-Zahra, “Perancangan User Experience Mobile Learning menggunakan Metode Design Thinking (Studi Kasus : SD Negeri Wates Kabupaten Kediri),” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 5, no. 8, pp. 3247–3255, 2021, [Online]. Available: http://j-ptiik.ub.ac.id
I. N. H. Wahyu, Samuel, “Perancangan Interaksi Panduan Pembelajaran Berbasis Personalisasi Menggunakan,” vol. 7, no. 2, pp. 227–239, 2021.
P. M. Hasanah, L. Fanani, and R. I. Rokhmawati, “Perancangan User Experience Learning Management System (LMS) Menggunakan Metode Goal-Directed Design (Studi Kasus: Rakryan Digital Academy),” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 5, no. 7, pp. 2877–2885, 2021, [Online]. Available: http://j-ptiik.ub.ac.id
K. Mangaroska, R. Martinez-Maldonado, B. Vesin, and D. Gašević, “Challenges and opportunities of multimodal data in human learning: The computer science students’ perspective,” J. Comput. Assist. Learn., vol. 37, no. 4, pp. 1030–1047, 2021, doi: 10.1111/jcal.12542.
M. Nyashanu, F. Pfende, and M. Ekpenyong, “Exploring the challenges faced by frontline workers in health and social care amid the COVID-19 pandemic: experiences of frontline workers in the English Midlands region, UK,” J. Interprof. Care, vol. 00, no. 00, pp. 655–661, 2020, doi: 10.1080/13561820.2020.1792425.
G. Melfi, K. Müller, T. Schwarz, G. Jaworek, and R. Stiefelhagen, “Understanding what you feel: A Mobile Audio-Tactile System for Graphics Used at Schools with Students with Visual Impairment,” Conf. Hum. Factors Comput. Syst. - Proc., pp. 1–12, 2020, doi: 10.1145/3313831.3376508.
I. Maslov, S. Nikou, and P. Hansen, “Exploring user experience of learning management system,” Int. J. Inf. Learn. Technol., vol. 38, no. 4, pp. 344–363, 2021, doi: 10.1108/IJILT-03-2021-0046.
M. Skjuve, I. M. Haugstveit, A. Følstad, and P. B. Brandtzaeg, “Help! Is my chatbot falling into the uncanny valley? An empirical study of user experience in human-chatbot interaction,” Hum. Technol., vol. 15, no. 1, pp. 30–54, 2019, doi: 10.17011/ht/urn.201902201607.
M. Pande and S. V. Bharathi, “Theoretical foundations of design thinking – A constructivism learning approach to design thinking,” Think. Ski. Creat., vol. 36, no. February, p. 100637, 2020, doi: 10.1016/j.tsc.2020.100637.
P. Cankurtaran and M. B. Beverland, “Using design thinking to respond to crises: B2B lessons from the 2020 COVID-19 pandemic,” Ind. Mark. Manag., vol. 88, no. May, pp. 255–260, 2020, doi: 10.1016/j.indmarman.2020.05.030.
D. Yoo et al., “Computational Alternatives Vignettes for Place- and Activity-Centered Digital Services in Public Libraries,” Conf. Hum. Factors Comput. Syst. - Proc., pp. 1–12, 2020, doi: 10.1145/3313831.3376597.
Q. Zhang, Y. Cui, and G. Qian, “Goal-directed design of metal–organic frameworks for liquid-phase adsorption and separation,” Coord. Chem. Rev., vol. 378, pp. 310–332, 2019, doi: 10.1016/j.ccr.2017.10.028.
R. R. Marbun, F. Al Mufied, and R. Fauzi, “Perancangan User Interface/User Experience (Ui/Ux) Website Helpmeong Untuk Shelter Menggunakan Metode Goal-Directed Design,” JIPI (Jurnal Ilm. Penelit. dan Pembelajaran Inform., vol. 7, no. 4, pp. 1096–1109, 2022, doi: 10.29100/jipi.v7i4.3190.
Y. L. Feng, C. C. Chen, and S. M. Wu, “Evaluation of Charm Factors of Short Video User Experience using FAHP-A Case Study of Tik Tok APP,” IOP Conf. Ser. Mater. Sci. Eng., vol. 688, no. 5, 2019, doi: 10.1088/1757-899X/688/5/055068.
K. Crespo and M. Kaushanskaya, “Is 10 Better than 1? The Effect of Speaker Variability on Children’s Cross-Situational Word Learning,” Lang. Learn. Dev., vol. 17, no. 4, pp. 397–410, 2021, doi: 10.1080/15475441.2021.1906680.
A. Hinderks, M. Schrepp, F. J. Domínguez Mayo, M. J. Escalona, and J. Thomaschewski, “Developing a UX KPI based on the user experience questionnaire,” Comput. Stand. Interfaces, vol. 65, no. April 2018, pp. 38–44, 2019, doi: 10.1016/j.csi.2019.01.007.
M. S. Ferreira, J. Antão, R. Pereira, I. S. Bianchi, N. Tovma, and N. Shurenov, “Improving real estate CRM user experience and satisfaction: A user-centered design approach,” J. Open Innov. Technol. Mark. Complex., vol. 9, no. 2, p. 100076, 2023, doi: 10.1016/j.joitmc.2023.100076.
B. Ferreira, W. Silva, E. Oliveira, and T. Conte, “Designing personas with empathy map,” Proc. Int. Conf. Softw. Eng. Knowl. Eng. SEKE, vol. 2015-Janua, no. May, pp. 501–505, 2015, doi: 10.18293/SEKE2015-152.
M. Ehn, M. Derneborg, Å. Revenäs, and A. Cicchetti, “User-centered requirements engineering to manage the fuzzy front-end of open innovation in e-health: A study on support systems for seniors’ physical activity,” Int. J. Med. Inform., vol. 154, no. June, 2021, doi: 10.1016/j.ijmedinf.2021.104547.
M. Schrepp and J. Thomaschewski, “Design and Validation of a Framework for the Creation of User Experience Questionnaires,” Int. J. Interact. Multimed. Artif. Intell., vol. 5, no. 7, p. 88, 2019, doi: 10.9781/ijimai.2019.06.006.
O. L. Aiyegbusi, “Key methodological considerations for usability testing of electronic patient-reported outcome (ePRO) systems,” Qual. Life Res., vol. 29, no. 2, pp. 325–333, 2020, doi: 10.1007/s11136-019-02329-z.
L. Zhou, J. Bao, I. M. A. Setiawan, A. Saptono, and B. Parmanto, “The mhealth app usability questionnaire (MAUQ): Development and validation study,” JMIR mHealth uHealth, vol. 7, no. 4, pp. 1–15, 2019, doi: 10.2196/11500.
Copyright (c) 2023 Meida Cahyo Untoro, Sophia Nouriska, Aidil Afriansyah, Mugi Praseptiawan, Winda Yulita, Ilham Firman Ashari
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
-a.jpg)
-b.jpg)
