Investigating the Effect of Complementary Information Stored in Multiple Languages on Question Answering Performance : A Study of the Multilingual-T5 for Extractive Question Answering

Sammanfattning: Extractive question answering is a popular domain in the field of natural language processing, where machine learning models are tasked with answering questions given a context. Historically the field has been centered on monolingual models, but recently more and more multilingual models have been developed, such as Google’s MT5 [1]. Because of this, machine translations of English have been used when training and evaluating these models, but machine translations can be degraded and do not always reflect their target language fairly. This report investigates if complementary information stored in other languages can improve monolingual QA performance for languages where only machine translations are available. It also investigates if exposure to more languages can improve zero-shot cross-lingual QA performance (i.e. when the question and answer do not have matching languages) by providing complementary information. We fine-tune 3 different MT5 models on QA datasets consisting of machine translations, as well as one model on the datasets together in combination with 3 other datasets that are not translations. We then evaluate the different models on the MLQA and XQuAD datasets. The results show that for 2 out of the 3 languages evaluated, complementary information stored in other languages had a positive effect on the QA performance of the MT5. For zero-shot cross-lingual QA, the complementary information offered by the fused model lead to improved performance compared to 2/3 of the MT5 models trained only on translated data, indicating that complementary information from other languages do not offer any improvement in this regard.