The effects of Deep Belief Network pre-training of a Multilayered perceptron under varied labeled data conditions

Sammanfattning: Sometimes finding labeled data for machine learning tasks is difficult. This is a problem for purely supervised models like the Multilayered perceptron(MLP). A Discriminative Deep Belief Network(DDBN) is a semi-supervised model that is able to use both labeled and unlabeled data. This research aimed to move towards a rule of thumb of when it is beneficial to use a DDBN instead of an MLP, given the proportions of labeled and unlabeled data. Several trials with different amount of labels, from the MNIST and Rectangles-Images datasets, were conducted to compare the two models. It was found that for these datasets, the DDBNs had better accuracy when few labels were available. With 50% or more labels available, the DDBNs and MLPs had comparable accuracies. It is concluded that a rule of thumb of using a DDBN when less than 50% of labels are available for training, would be in line with the results. However, more research is needed to make any general conclusions.