Hybrid models combining the analytical (rulebased) and connectionist (artificial neural network (ANN)) paradigms are called Knowledge Based Neural Networks (KBNN). The Knowledge Based Artificial Neural Network (KBANN) is one such model that makes use of the domain theory represented as propositional rules and training examples. In this article, we analyze the performance of KBANN and suggest some ideas of improving its capabilities. A study on the effect of inductive bias on KBANN, use of adaptive learning algorithms instead of the standard backpropagation to improve the training times and the use of regularization methods for improving generalization performance is presented. It is shown that for better performance, the initial weight assignment to links obtained by domain theory varies with the domain and the performance of KBANN is improved by using regularization methods like the weight decay along with Rprop training algorithm instead of the standard backpropagation.