AI-Assisted Software Testing: Opportunities and Challenges
Main Article Content
Keywords
software testing, artificial intelligence, machine learning, test automation, defect prediction
Abstract
Software testing has played an important role in checking software reliability, security, and quality. However, testing has remained one of the most costly and time-consuming parts of the software development life cycle, especially as modern systems have b ecome larger and more frequently updated. Traditional manual testing depends on human experience, while automated testing still requires testers to write and maintain many test scripts. This paper reviewed the opportunities and challenges of AI-assisted software testing based on selected academic studies, technical reports, and industry examples. The review found that AI techniques, such as machine learning, search-based optimization, fuzzing, and large language models, can support testing tasks such as test generation, defect prediction, code analysis, test prioritization, and debugging. These applications may reduce manual effort, improve test coverage, and help teams identify risky components earlier. At the same time, the review identified several challenges, including limited data quality, weak generalization across projects, low interpretability, false positives, integration difficulties, and risks related to LLM outputs. The paper argues that AI is unlikely to fully replace human testers in the near future. Instead, AI should be used as an assistant that supports human judgment in software quality assurance.
References
- [1] J. M. Zhang, M. Harman, L. Ma, and Y. Liu, “Machine Learning Testing: Survey, Landscapes and Horizons,” IEEE Transactions on Software Engineering, vol. 48, no. 1, pp. 1–36, 2022, doi: 10.1109/TSE.2019.2962027.
- [2] J. Wang, Y. Huang, C. Chen, Z. Liu, S. Wang, and Q. Wang, “Software Testing with Large Language Models: Survey, Landscape, and Vision,” IEEE Transactions on Software Engineering, 2024, doi: 10.1109/TSE.2024.3368208.
- [3] K. Mao, M. Harman, and Y. Jia, “Sapienz: Multi-objective Automated Testing for Android Applications,” in Proceedings of the 25th International Symposium on Software Testing and Analysis (ISSTA), 2016, pp. 94–105, doi: 10.1145/2931037.2931054.
- [4] M. Harman, S. A. Mansouri, and Y. Zhang, “Search-Based Software Engineering: Trends, Techniques and Applications,” ACM Computing Surveys, vol. 45, no. 1, Article 11, pp. 1–61, 2012, doi: 10.1145/2379776.2379787.
- [5] Google, “ClusterFuzz: Scalable Fuzzing Infrastructure,” Google Open Source Documentation. Available: https://google.github.io/clusterfuzz/
- [6] P. Godefroid, M. Y. Levin, and D. Molnar, “Automated Whitebox Fuzz Testing,” in Proceedings of the Network and Distributed System Security Symposium (NDSS), 2008.
- [7] X. Hou, Y. Zhao, Y. Liu, Z. Yang, K. Wang, L. Li, X. Luo, D. Lo, J. Grundy, and H. Wang, “Large Language Models for Software Engineering: A Systematic Literature Review,” ACM Transactions on Software Engineering and Methodology, vol. 33, no. 8, Article 220, 2024, doi: 10.1145/3695988.
- [8] M. Monperrus, “Automatic Software Repair: A Bibliography,” ACM Computing Surveys, vol. 51, no. 1, Article 17, pp. 1–24, 2018, doi: 10.1145/3105906.
- [9] M. Tufano, C. Watson, G. Bavota, M. Di Penta, M. White, and D. Poshyvanyk, “An Empirical Study on Learning Bug-Fixing Patches in the Wild via Neural Machine Translation,” ACM Transactions on Software Engineering and Methodology, vol. 28, no. 4, Article 19, pp. 1–29, 2019, doi: 10.1145/3340544.
- [10] T. Hall, S. Beecham, D. Bowes, D. Gray, and S. Counsell, “A Systematic Literature Review on Fault Prediction Performance in Software Engineering,” IEEE Transactions on Software Engineering, vol. 38, no. 6, pp. 1276–1304, 2012, doi: 10.1109/TSE.2011.103.
Article Sidebar
This work is licensed under a Creative Commons Attribution 4.0 International License.