Remote User Testing of Dictation Tools : A study investigating which factors impact the user experience of a dictation tool

Sammanfattning: Lately, Automatic Speech Recognition (ASR) has improved in both performance and popularity and today the technology can be found in many different products, often in the form of a Voice User Interface (VUI). There are numerous factors to consider when developing a system using this technology. It is essential to comprehend both the user’s experience of using the product and the performance of the ASR. In terms of the user experience, only the user can tell you what is important, which is why user testing is a necessary part of further development. In this study, a test was designed to perform a comprehensive remote user test of the dictation feature Notebook Voice Control in the product BIOVIA Notebook. The aim of the user test was to understand what factors affect the performance of the feature, and when a user is satisfied. To carry out the tests a web-application was designed, created, and deployed. The web-application was integrated with the API of the product being tested and guided the participants through the tests as well as captured the test data. From the test results, it was clear that accurate recognition was highly important in terms of user satisfaction. In order to produce an accurate dictation, dictating in your native language seemed to be a success factor as well as having dictation experience. If you lack dictation experience, training appears to help. The errors that were the most remarkable in the tests and would need immediate attention were errors in the categories of numbers, abbreviations, units, and field-specific words. The issue of the time spent correcting errors was raised, and for some, this was what made them argue that the tool was currently not useful. The majority were positive towards the tool but wanted it to be a bit better, or perhaps they just have to practice using it a bit more.