COMPARATIVE ANALYSIS OF AUTOMATION FUNCTIONAL TESTING TOOLS PERFORMANCE FOR PLAYSTORE APPS WITH DIA METHOD
Abstract
The complexity of smartphone applications presents challenges for developers, who must ensure flawless functionality despite limitations such as budget and time constraints. Manual testing is time-consuming, prompting a shift towards automated testing methods to ensure efficiency and reliability. In this context, researchers are evaluating the efficacy of three leading test automation frameworks—Robot Framework, Katalon Studio, and UI Path—against key performance parameters. Using the Distance to the Ideal Alternative (DIA) method on playstore apps. The main performance parameters used as a reference are automated testing progress and tools usability. Katalon Studio emerges as the top performer, securing the top rank with a remarkably close to the alternative ideal positive distance (Ri) value of 0.00001. UI Path occupies the second position with a Ri value of 0.00135, while Robot Framework trails behind with a Ri value of 0.00295. This research contributes to the understanding of the performance of different automation frameworks in the context of functional testing, providing valuable insights for developers and organizations seeking to optimize their testing processes. The findings underscore the significance of Katalon Studio's exceptional performance and highlight opportunities for improvement in UI Path and Robot Framework. Additionally, implementing a robust monitoring and evaluation framework is crucial for tracking the ongoing performance and optimizing the efficiency of these automation frameworks.
References
Abdulwareth, A. J., & Al-Shargabi, A. A. (2021). Toward a Multi-Criteria Framework for Selecting Software Testing Tools. IEEE Access, 9, 158872–158891. https://doi.org/10.1109/ACCESS.2021.3128071
Al-Gharabally, M., Almutairi, A. F., & Salman, A. A. (2021). Particle swarm optimization application for multiple attribute decision making in vertical handover in heterogenous wireless networks. Journal of Engineering Research (Kuwait), 9(1), 176–187. https://doi.org/10.36909/JER.V9I1.10331
Arya, S., Chitranshi, M., & Singh, Y. (2021). Analysing Distance Measures in Topsis: A Python-Based Tool. 275–292. https://doi.org/10.1007/978-981-16-1528-3_24
Aslam, Z., Ayub, N., Ali, M., Zubair, S., & Naz, A. (2022). Performance-Based Analysis Of Test Automation Tools For Android Applications. Researchgate.Net. https://doi.org/10.17605/OSF.IO/D3BHQ
Baktha, K. (2020). Evaluating the Performance and Capabilities of Popular Android Mobile Application Testing Automation Frameworks in Agile/DevOps Environment. https://www.diva-portal.org/smash/record.jsf?pid=diva2:1471376
Berihun, N. G., Dongmo, C., & Van der Poll, J. A. (2023). The Applicability of Automated Testing Frameworks for Mobile Application Testing: A Systematic Literature Review. Computers, 12(5). https://doi.org/10.3390/computers12050097
Chakraborty, S. (2022). TOPSIS and Modified TOPSIS: A comparative analysis. Decision Analytics Journal, 2, 100021. https://doi.org/10.1016/j.dajour.2021.100021
Gota, L., Gota, D., & Miclea, L. (2020, May). Continuous Integration in Automation Testing. In 2020 IEEE International Conference on Automation, Quality and Testing, Robotics (AQTR) (pp. 1-6). IEEE. https://doi.org/10.1109/AQTR49680.2020.9129990
Karlsson, S., Čaušević, A., Sundmark, D., & Larsson, M. (2021, April). Model-based automated testing of mobile applications: an industrial case study. In 2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW) (pp. 130-137). IEEE. https://doi.org/10.1109/ICSTW52544.2021.00033
Kozak, I., & Berko, A. (2022, November). Three-module framework for automated software testing. In 2022 IEEE 17th International Conference on Computer Sciences and Information Technologies (CSIT) (pp. 454-457). IEEE. https://doi.org/10.1109/CSIT56902.2022.10000806
Lin, J. W., Salehnamadi, N., & Malek, S. (2020, December). Test automation in open-source android apps: A large-scale empirical study. In Proceedings of the 35th IEEE/ACM International Conference on Automated Software Engineering (pp. 1078-1089). https://doi.org/10.1145/3324884.3416623
Menegassi, A. A., & Endo, A. T. (2020). Automated tests for cross-platform mobile apps in multiple configurations. IET Software, 14(1), 27–38. https://doi.org/10.1049/iet-sen.2018.5445
Prasad, L., Yadav, R., & Vore, N. (2021). A Systematic Literature Review of Automated Software Testing Tool. Lecture Notes in Networks and Systems, 167, 101–123. https://doi.org/10.1007/978-981-15-9712-1_10
Salam, M. A., Taha, S., & Hamed, M. G. (2022, October). Advanced Framework for Automated Testing of Mobile Applications. In 2022 4th Novel Intelligent and Leading Emerging Sciences Conference (NILES) (pp. 233-238). IEEE. https://doi.org/10.1109/NILES56402.2022.9942374
Tran, H. M., Ninh, T. D., Tran, T. D., Van Ngo, V., & Nguyen, L. D. (2023, October). Automation Testing with Appium Framework in IP Multimedia Subsystem. In 2023 14th International Conference on Information and Communication Technology Convergence (ICTC) (pp. 579-582). IEEE. https://doi.org/10.1109/ICTC58733.2023.10392322
Zayat, W., Kilic, H. S., Yalcin, A. S., Zaim, S., & Delen, D. (2023). Application of MADM methods in Industry 4.0: A literature review. Computers & Industrial Engineering, 177, 109075. https://doi.org/10.1016/J.CIE.2023.109075
Abstract viewed = 52 times
PDF downloaded = 60 times
Copyright (c) 2024 Faizal Riza, Berliyanto Berliyanto, Aji Nurrohman, Rachmat Setiabudi
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
The copyright of any article in the TECHNO Nusa Mandiri Journal is fully held by the author under the Creative Commons CC BY-NC license.
- The copyright in each article belongs to the author.
- Authors retain all their rights to published works, not limited to the rights set out on this page.
- The author acknowledges that Techno Nusa Mandiri: Journal of Computing and Information Technology (TECHNO Nusa Mandiri) is the first to publish with a Creative Commons Attribution 4.0 International license (CC BY-NC).
- Authors can enter articles separately, manage non-exclusive distribution, from manuscripts that have been published in this journal into another version (for example: sent to author affiliation respository, publication into books, etc.), by acknowledging that the manuscript was published for the first time in Techno Nusa Mandiri: Journal of Computing and Information Technology (TECHNO Nusa Mandiri);
- The author guarantees that the original article, written by the stated author, has never been published before, does not contain any statements that violate the law, does not violate the rights of others, is subject to the copyright which is exclusively held by the author.
- If an article was prepared jointly by more than one author, each author submitting the manuscript warrants that he has been authorized by all co-authors to agree to copyright and license notices (agreements) on their behalf, and agrees to notify the co-authors of the terms of this policy. Techno Nusa Mandiri: Journal of Computing and Information Technology (TECHNO Nusa Mandiri) will not be held responsible for anything that may have occurred due to the author's internal disputes.
.png){kind=logo}
