Bounded exhaustive generation of tests in model-based testing

Sammanfattning: There are some systems (or parts of systems) that are very critical and need especially good test suites to test them. For these critical systems, exhaustive testing may be a good way to test them. Thus, we have implemented two versions of bounded exhaustive search (BES) algorithms in a model-based testing tool called, Modbat. One of the BES versions (BESnL) visits each self-loop in the model only once. The other version (BESL) has no constraint or limitation on the number of time it visits each self-loop. We have then compared the two BES algorithms with each other and with an already implemented algorithm in Modbat called random search (RS). We have run the three mentioned algorithms (BESL, BESnL and RS) on five different models and compared their performance on these models in terms of time, coverage and finding faults. We have found that BESnL is faster than BESL and it can miss some faults that BESL can find. However, BESnL can find errors faster than BESL. BESL has sometimes better performance than BESnL in terms of branch coverage. In terms of other coverage criteria (like state coverage, transition coverage and instruction coverage), both BESL and BESnL has very similar performance. We have also found that running the RS algorithm is, in general, faster than both BES algorithms at generating tests (given the same total number of tests generated) if the model has a clear end state. RS may also be faster at finding faults than the BES algorithms. However, The BES algorithms and the RS algorithm have similar behaviours regarding coverage. Nevertheless, RS can sometimes reach higher coverage faster than the BES algorithms and with a smaller number of tests.