Challenges of Large-ScaleSoftware Testing and the Role of Quality Characteristics : Empirical Study

Sammanfattning: Currently, information technology is influencing every walks of life. Our livesincreasingly depend on the software and its functionality. Therefore, thedevelopment of high-quality software products is indispensable. Also, inrecent years, there has been an increasing interest in the demand for high-qualitysoftware products. The delivery of high-quality software products and services isnot possible at no cost. Furthermore, software systems have become complex andchallenging to develop, test, and maintain because of scalability. Therefore, withincreasing complexity in large scale software development, testing has been acrucial issue affecting the quality of software products. In this paper, large-scalesoftware testing challenges concerning quality and their respective mitigations arereviewed using a systematic literature review, and interviews. Existing literatureregarding large-scale software development deals with issues such as requirementand security challenges, so research regarding large-scale software testing and itsmitigations is not dealt with profoundly.In this study, a total of 2710 articles were collected from 1995-2020; 1137(42%)IEEE, 733(27%) Scopus, and 840(31%) Web of Science. Sixty-four relevant articleswere selected using a systematic literature review. Also, to include missed butrelevant articles, snowballing techniques were applied, and 32 additional articleswere included. A total of 81 challenges of large-scale software testing wereidentified from 96 total articles out of which 32(40%) performance, 10(12 %)security, 10(12%) maintainability, 7(9 %) reliability, 6(8%) compatibility, 10(12%)general, 3(4%) functional suitability, 2(2%) usability, and 1(1%) portability weretesting challenges were identified. The author identified more challenges mainlyabout performance, security, reliability, maintainability, and compatibility qualityattributes but few challenges about functional suitability, portability, and usability.The result of the study can be used as a guideline in large-scale software testingprojects to pinpoint potential challenges and act accordingly.