Sökning: "Knowledge distillation"

Visar resultat 1 - 5 av 25 uppsatser innehållade orden Knowledge distillation.

1. 1. Understanding the Robustnessof Self Supervised Representations

   Uppsats för yrkesexamina på avancerad nivå, Luleå tekniska universitet/Institutionen för system- och rymdteknik

   Författare :Johan Rodahl Holmgren; [2023]
   Nyckelord :Self-Supervision; AI; Robustness; Computer-Vision;

   Sammanfattning : This work investigates the robustness of learned representations of self-supervised learn-ing approaches, focusing on distribution shifts in computer vision. Joint embedding architecture and method-based self-supervised learning approaches have shown advancesin learning representations in a label-free manner and efficient knowledge transfer towardreducing human annotation needs. LÄS MER

3. 2. Knowledge distillation for anomaly detection

   Master-uppsats, Uppsala universitet/Institutionen för informationsteknologi

   Författare :Nils Gustav Erik Pettersson; [2023]
   Nyckelord :;

   Sammanfattning : The implementation of systems and methodologies for time series anomaly detection holds the potential of providing timely detection of faults and issues in a wide variety of technical systems. Ideally, these systems are able to identify deviations from the normal behavior of systems even before any problems manifest, thus enabling proactive maintenance. LÄS MER

4. 3. Efficient Sentiment Analysis and Topic Modeling in NLP using Knowledge Distillation and Transfer Learning

   Master-uppsats, KTH/Skolan för elektroteknik och datavetenskap (EECS)

   Författare :George Malki; [2023]
   Nyckelord :Large Language Model; RoBERTa; Knowledge distillation; Transfer learning; Sentiment analysis; Topic modeling; Stor språkmodell; RoBERTa; Kunskapsdestillation; överföringsinlärning; Sentimentanalys; Ämnesmodellering;

   Sammanfattning : This abstract presents a study in which knowledge distillation techniques were applied to a Large Language Model (LLM) to create smaller, more efficient models without sacrificing performance. Three configurations of the RoBERTa model were selected as ”student” models to gain knowledge from a pre-trained ”teacher” model. LÄS MER

5. 4. Enhancing Neural Network Accuracy on Long-Tailed Datasets through Curriculum Learning and Data Sorting

   Master-uppsats, KTH/Matematik (Avd.)

   Författare :Daniel Barreira; [2023]
   Nyckelord :Machine Learning; Neural Network; CORAL-framework; Long-Tailed Data; Imbalance Metrics; Teacher-Student models; Curriculum Learning; Training Scheme; Maskininlärning; Neuralt Nätverk; CORAL-ramverk; Long-Tailed Data; Imbalance Metrics; Teacher-Student modeler; Curriculum Learning; Tränings- scheman;

   Sammanfattning : In this paper, a study is conducted to investigate the use of Curriculum Learning as an approach to address accuracy issues in a neural network caused by training on a Long-Tailed dataset. The thesis problem is presented by a Swedish e-commerce company. LÄS MER

7. 5. Distilling Multilingual Transformer Models for Efficient Document Retrieval : Distilling multi-Transformer models with distillation losses involving multi-Transformer interactions

   Master-uppsats, KTH/Skolan för elektroteknik och datavetenskap (EECS)

   Författare :Xuecong Liu; [2022]
   Nyckelord :Dense Passage Retrieval; Knowledge Distillation; Multilingual Transformer; Document Retrieval; Open Domain Question Answering; Tät textavsnittssökning; kunskapsdestillering; flerspråkiga transformatorer; dokumentsökning; domänlöst frågebesvarande;

   Sammanfattning : Open Domain Question Answering (OpenQA) is a task concerning automatically finding answers to a query from a given set of documents. Language-agnostic OpenQA is an increasingly important research area in the globalised world, where the answers can be in a different language from the question. LÄS MER