Deep Learning of Constrained Autoencoders for Enhanced Understanding of Data
Abstract of Deep Learning of Constrained Autoencoders
Deep Learning of Constrained Autoencoders
Deep Learning of Constrained Autoencoders for Enhanced Understanding of Data.Unsupervised feature extractors are known to perform an efficient and discriminative representation of data. Insight into the mappings they perform and human ability to understand them, however, remain very limited.
This is especially prominent when multilayer deep learning architectures are used.
This paper demonstrates how to remove these bottlenecks within the architecture of non-negativity constrained autoencoder.It is shown that using both L1 and L2 regularizations that induce non-negativity of weights, most of the weights in the network become constrained to be non-negative, thereby resulting into a more understandable structure with minute deterioration in classification accuracy.Also, this proposed approach extracts features that are more sparse and produces additional output layer sparsification. The method is analyzed for accuracy and feature interpretation on the MNIST data, the NORB normalized uniform object data,
and the Reuters text categorization data set
Conclusion
This Deep Learning of Constrained Autoencoders for Enhanced Understanding of Data paper addresses the concept and properties of special regularization of DL AE that takes advantage of non-negative encodings and at the same time of special regularization. It has been shown that using both L1 and L2 to penalize the negative weights, most of them are forced to be nonnegative and sparse, and hence, the network interpretability is enhanced. In fact, it is also observed that most of the weights in the Softmax layer become non-negative and sparse.
