Possibilities of automatic detection of "Async Wait" Flaky tests in Python applications

Sammanfattning: Flaky tests are defined as tests that show non-deterministic outcomes, meaning they can show both passing and failing results without changes to the code. These tests cause a major problem in the software development process since it can be difficult to know if the cause of a failure originates from the production- or test code. Developers may choose to ignore failing tests known to be flaky when they might actually hide real bugs in the production code. This thesis investigates a specific category of flaky tests known as "Async Wait", which are tests that makes asynchronous calls to servers and other remote resources and fails to properly wait for the results to be returned. There are tools available for detecting flaky tests, but most of these need the test to be executed and operate on run time information. In order to detect potential flakiness in an even earlier state, this thesis looks in to if it is possible to predict flaky outcomes by analyzing only at the test code itself without running it. The scope is limited to the Async Wait only to determine in which cases and under what circumstances developing an algorithm to automatically detect these flaky tests would be possible in this category. Commits from open source projects on GitHub were scanned for Async Wait flaky tests with the intention of finding the characteristics of the asynchronous calls and how the waiting for them is handled as well as how the flakiness is resolved by developers in practice in order to see if the information in only the test code is enough to predict flaky behavior.