We consider three classes of single neuron models, namely the network models typically used in neural networks, the diffusion models, and the jump-diffusion models directly related to neurophysiology. A limit passage in the jump-diffusion models leads to the diffusion models, while a simplification of the diffusion models leads to the network models. The diffusion models are very well known yet they learning behavior was never analyzed. By analogy with network models, we assume that the weights of such neurons can change to implement learning. We will show that such a natural assumption has disastrous consequence on a wide class of diffusion models: they cannot be obtained from the jump-diffusion models, hence the fundamental relation between the diffusion models and neurophysiology is broken. We will also show that if the input chemical (neurotransmitter) quanta are random, the diffusion models can be constructed, hence such models can learn.