Evaluating the robustness of DistilBERT to data shift in toxicity detection

Sammanfattning: With the rise of social media, cyberbullying and online spread of hate have become serious problems with devastating consequences. Mentimeter is an interactive presentation tool enabling the presentation audience to participate by typing their own answers to questions asked by the presenter. As the Mentimeter product is commonly used in schools, there is a need to have a strong toxicity detection program that filters out offensive and profane language. This thesis focuses on the topics of text pre-processing and robustness to datashift within the problem domain of toxicity detection for English text. Initially, it is investigated whether lemmatization, spelling correction, and removal of stop words are suitable strategies for pre-processing within toxicity detection. The pre-trained DistilBERT model was fine-tuned using an English twitter dataset that had been pre-processed using a number of different techniques. The results indicate that none of the above-mentioned strategies have a positive impact on the model performance. Lastly, modern methods are applied to train a toxicity detection model adjusted to anonymous Mentimeter user text data. For this purpose, a balanced Mentimeter dataset with 3654 datapoints was created and annotated by the thesis author. The best-performing model of the pre-processing experiment was iteratively fine-tuned and evaluated with an increasing amount of Mentimeter data. Based on the results, it is concluded that state-of-the-art performance can be achieved even when using relatively few datapoints for fine-tuning. Namely, when using around 500 − 2500 training datapoints, F1-scores between 0.90 and 0.94 were obtained on a Mentimeter test set. These results show that it is possible to create a customized toxicity detection program, with high performance, using just a small dataset.