We present a fast, fully parameterizable GPU implementation of a Convolutional Neural Network. Our feature extractors are neither carefully designed nor pre-wired, but rather learned in a supervised way. Using deep, hierarchical architectures we improve the state of the art on a digit (MNIST) and two object classification benchmarks (NORB, CIFAR10). Deep nets trained by simple back-propagation perform better than more shallow ones, and learning is surprisingly rapid. NORB is completely trained in five epochs and test error rates on MNIST drop to 2.42%, 0.97% and 0.48% after 1, 3 and 17 epochs, respectively. On MNIST, NORB and CIFAR10 we obtain recognition rates of 0.35%, 2.53% and 19.51%, which are better than any previously published results.