In (Hao et al. 2012), we propose a novel model in the deep learning framework concentrated specifically on textures in images. The model is a convolutional variant of the Gaussian gated Boltzmann machine inspired by a so called co-occurrence matrix in traditional texture analysis. This model is also applicable to both classification and reconstruction tasks.

We have used small images as example data while developing new models and methods in deep learning in general. For instance, in (Raiko et al. 2012), we showed competitive classification performance of small images with very low computational complexity. Deep learning models are often applicable to both classification and reconstruction tasks at once. An example of image reconstruction can be found in (Cho et al. 2011), where left half of a face is reconstructed from the right half (see figure) using a model learned from full images of other people..

There is one master's thesis written on image-specific models:

• Hao, Tele. Gated Boltzmann Machine in Texture Modeling. Master's Thesis, Aalto Univerisy, 2012.

References

T. Hao, T. Raiko, A. Ilin, and J. Karhunen.
Gated Boltzmann Machine in Texture Modeling.
In Artificial Neural Networks and Machine Learning - ICANN 2012, Lecture Notes in Computer Science, volume 7553, pages 124-131, September 2012.

T. Raiko, H. Valpola, and Y. LeCun.
Deep Learning Made Easier by Linear Transformations in Perceptrons.
In Proc. of the 15th Int. Conf. on Artificial Intelligence and Statistics (AISTATS 2012), JMLR W&CP, volume 22, pp. 924-932, La Palma, Canary Islands, April 21-23, 2012.

K. Cho, T. Raiko, and A. Ilin
Gaussian-Bernoulli Deep Boltzmann Machine.
In the NIPS 2011 Workshop on Deep Learning and Unsupervised Feature Learning, Granada, Spain, December 16, 2011.