An Ambidextrous Approach to User Acceptance Testing Management and Automation : How software customers can optimize their processes to explore new technology as well as current performance to improve the effectiveness of user acceptance testing

Sammanfattning: User acceptance tests are tests that are executed by companies that are updating or integrating a new IT system. The system is usually procured by another company which has developed the system; acceptance testing is important to validate functionality and ensure that the system is working within the target IT landscape. A lack of research around management of acceptance testing from the customer’s perspective was identified. This thesis aims to explore how management practices affect acceptance testing and related resources, as well as investigate if automated acceptance testing can be applied by finance companies interested in upgrading or replacing critical systems. Data was collected in three steps aimed to show the issue from different perspectives such that biases in answers from interviews with testers could be adjusted for. As a first step a tool assisted literature review was conducted using GPT-3 (Generative Pre-trained transformer 3) to understand the academic consensus. Then, employees that had worked with user acceptance testing were interviewed to gain insights relating to how people think about the problems that arise during user acceptance testing. Finally, log files from test and production systems were data mined to understand how consensus and thoughts translate to actions in the testing environments. The results show strong support for software customers to integrate knowledge management and change management processes into their strategy for user acceptance testing. Recommendations are given for communication and planning of user acceptance testing. For example, it is recommended to clearly communicate the test’s purpose to employees as this can affect motivation and performance. When it comes to automation it seems evident that the return on investment generally is not favorable for user acceptance testing automation. This is because acceptance testing does not happen frequently enough before test cases need maintenance, it is also difficult to automate test cases for a completely new system. This thesis gives clear recommendations to the industry while also expanding the research about user acceptance testing in combination with knowledge management, automation, and change management.