Implementation and evaluation of a tool for automated testing of performance and resource consumption in Java

Sammanfattning: Detecting performance issues in software systems is desirable and sometimes critical. In this thesis, a new tool is presented for the purpose of testing performance at a smaller scale. This tool, JFRNG, is implemented as an extension to the Java testing framework TestNG with the purpose of allowing developers to test the performance of Java code in unit and integration tests. The thesis also contains a study of the current methods and approaches to automating the process of detecting performance issues.  An evaluation of JFRNG is performed both in terms of performance and usability, where the tool is compared to JfrUnit, a similar tool that targets the JUnit framework. A usability survey took place during the usability evaluation where professional Java developers gave their opinions on the clarity of the code that is produced by JFRNG and JfrUnit. The survey participants found the code produced by JFRNG to be easier to read and understand and also rated JFRNG slightly higher on a number of specific aspects related to the clarity of the code. The added performance overhead that was observed for JFRNG during the performance evaluation is on par with JfrUnit and does add some run time for tests, suggesting that developers should be mindful when selecting tests that should utilize JFRNG. The functionality is relatively extensive, allowing developers to collect and test for a large number of metrics related to the tested code. Finally, the thesis is concluded with some suggestions for future work, such as integrating more complex methods for detecting performance regressions based on collected metrics.