We describe an unsupervised learning algorithm for ex- tracting sparse and locally shift-invariant features. We also devise a principled procedure for learning hierarchies of in- variant features. Each feature detector is composed of a set of trainable convolutional filters followed by a max-pooling layer over non-overlapping windows, and a point-wise sig- moid non-linearity. A second stage of more invariant fea- tures is fed with patches provided by the first stage feature extractor, and is trained in the same way. The method is used to pre-train the first four layers of a deep convolutional network which achieves state-of-the-art performance on the MNIST dataset of handwritten digits. The final testing error rate is equal to 0.42%. Preliminary experiments on com- pression of bitonal document images show very promising results in terms of compression ratio and reconstruction er- ror.