In this work, we tackle the problem of rule extraction from multi-layer perceptrons. Our approach consists in characterizing discriminant hyper-plane frontiers built by a special neural network model denoted to as Discretized Interpretable Multi Layer Perceptron (DIMLP). Rules are extracted in polynomial time with respect to the size of the problem. Further, the degree of matching between extracted rules and neural network responses is 100%. We apply DIMLP to five data sets of the public domain in which for some of them, it gives better average predictive accuracy than standard multi-layer perceptrons and C4.5 decision trees.