A study of Pseudo-tested Methods in an Android Continuous Integration Environment

Sammanfattning: As mobile devices become more capable and mobile adaption grows across the world, the complexity of mobile applications increases. For example, mobile applications let consumers view their banking details and legitimize themselves. With mobile applications serving these crucial functions, the need for quality and robustness increases. Unit tests are one way to measure the quality of an application. However, there is a need to evaluate the unit tests themselves in order to increase their effectiveness. Mutation testing is one approach to test the application source code by deliberately mutating it in an effort to see if the unit tests of the application detect the change. This study investigates mutation testing on an Android application. More specifically, it evaluates a form of extreme mutation testing using pseudotested methods. With pseudo-tested methods, entire method bodies are replaced with a single return statement. This study introduces a framework for detecting pseudo-tested methods within the continuous integration environment as well as the capability of measuring the usefulness of generated reports that highlight these methods. Moreover, the study conducts interviews with multiple developers discussing the pseudo-tested methods in detail. We implement this framework within the continuous integration environment using multiple components. Our results show that developers are positive towards having a tool for detecting pseudo-tested within the continuous integration environment. Moreover, the results indicate that methods written by developers are more useful to test than auto-generated methods. We draw the conclusion that detection of pseudo-tested methods can be beneficial in increasing the quality of mobile applications. However, we note that further work is needed to eliminate the prevalence of false-positives found in our study