Understanding Test Case Design: An Exploratory Survey of the Testers’ Routine and Behavior

Sammanfattning: Testing is an important component of every software since it enables the delivery of reliable solutions that meet the needs of end-users. Valuable testing is represented by the results of the test cases, which may provide knowledge into the presence of software system flaws. Testers create test cases because it lets one check and ensures that all user requirements and situations that users can go through are fully covered. In addition, test cases enable to find out the design problems early and thus allow testers to find solutions as soon as possible. The main goal of this thesis is to understand the routine and behavior of human testers when performing testing and to gain a better understanding of the software testing field. This thesis also aims to discover the challenges testers face when testing. The report shows the results of a combined qualitative and quantitative survey, responded by 38 experienced software testers and developers. The survey explores testers' cognitive processes when performing testing by investigating the knowledge they bring, the activities they perform, and the challenges they face in their routine. By analyzing the survey's results, we identified several main themes (related to knowledge, activities, and challenges) and brought more knowledge on the course of the problem-solving process cycle from understanding the test goal, planning the test strategy, executing tests to checking of the test results. We report a more refined test design model. The results of this thesis suggest that testers use several sources of knowledge in their routine when creating and executing test cases such as documentation, code, and their experience. In addition, we found that the main activities of testers are related to specific tasks such as the comprehension of software requirements, learning as much as possible about the software, and discussing of the results with the developing team or other testers to get feedback about the outcomes. Finally, testers face many challenges in their routine when understanding, planning, executing, and checking tests: e.g., incomplete, or ambiguous requirements, complex or highly configurable scenarios that are hard to test, the lack of time and hard deadlines and unstable environments.