Latent Task Embeddings forFew-Shot Function Approximation

Sammanfattning: Approximating a function from a few data points is of great importance in fields where data is scarce, like, for example, in robotics applications. Recently, scalable and expressive parametric models like deep neural networks have demonstrated superior performance on a wide variety of function approximation tasks when plenty of data is available –however, these methods tend to perform considerably worse in low-data regimes which calls for alternative approaches. One way to address such limitations is by leveraging prior information about the function class to be estimated when such data is available. Sometimes this prior may be known in closed mathematical form but in general it is not. This the-sis is concerned with the more general case where the prior can only be sampled from, such as a black-box forward simulator. To this end, we propose a simple and scalable approach to learning a prior over functions by training a neural network on data from a distribution of related functions. This steps amounts to building a so called latent task embedding where all related functions (tasks) reside and which later can be efficiently searched at task-inference time - a process called fine-tuning. The pro-posed method can be seen as a special type of auto-encoder and employs the same idea of encoding individual data points during training as the recently proposed Conditional Neural Processes. We extend this work by also incorporating an auxiliary task and by providing additional latent space search methods for increased performance after the initial training step. The task-embedding framework makes finding the right function from a family of related function quick and generally requires only a few informative data points from that function. We evaluate the method by regressing onto the harmonic family of curves and also by applying it to two robotic systems with the aim of quickly identifying and controlling those systems.