Normalizing Flow based Hidden Markov Models for Phone Recognition

Sammanfattning: The task of Phone recognition is a fundamental task in Speech recognition and often serves a critical role in bench-marking purposes. Researchers have used a variety of models used in the past to address this task, using both generative and discriminative learning approaches. Among them, generative approaches such as the use of Gaussian mixture model-based hidden Markov models are always favored because of their mathematical tractability. However, the use of generative models such as hidden Markov models and its hybrid varieties is no longer in fashion owing to a large inclination to discriminative learning approaches, which have been found to perform better. The only downside is that these approaches do not always ensure mathematical tractability or convergence guarantees as opposed to their generative counterparts. So, the research problem was to investigate whether there could be a process of augmenting the modeling capability of generative Models using a kind of neural network based architectures that could simultaneously prove mathematically tractable and expressive. Normalizing flows are a class of generative models that have been garnered a lot of attention recently in the field of density estimation and offer a method for exact likelihood computation and inference. In this project, a few varieties of Normalizing flow-based hidden Markov models were used for the task of Phone recognition on the TIMIT dataset. It was been found that these models and their mixture model varieties outperformed classical generative model varieties like Gaussian mixture models. A decision fusion approach using classical Gaussian and Normalizing flow-based mixtures showed competitive results compared to discriminative learning approaches. Further analysis based on classes of speech phones was carried out to compare the generative models used. Additionally, a study of the robustness of these algorithms to noisy speech conditions was also carried out.