Several means for improving the performance and training of neural networks for classification are proposed. Crossvalidation is used as a tool for optimizing network parameters and architecture. It is shown that the remaining residual generalization error can be reduced by invoking ensembles of similar networks.
[1] 993E. Levin, N. Tishby, and S. Solla, "A statistical approach to learning and generalization in layered neural networks,"Proc. IEEE (Special Issue on Neural Networks), C. Lau, Guest Ed., 1990, to be published.
[2] D. E. Rumelhart, G. E. Hinton, and R. J. Williams, "Learning internal representation by error propagation,"Parallel Distributed Processing: Explorations in the Microstructure of Cognition, vols. 1 and 2. Cambridge, MA: MIT Press, 1986.
[3] D. H. Ackley, G. E. Hinton, and T. J. Sejnowski, "A learning algorithm for Boltzmann machines,"Cognitive Sci., vol. 9, pp. 147- 169, 1985.
[4] B. Widrow and R. Winter, "Neural nets for adaptive filtering and adaptive pattern recognition,"Computer, vol. 25, Mar. 1988.
[5] F. J. Pineda, "Generalization of backpropagation to recurrent neural networks,"Phys. Rev. Lett., vol. 59, pp. 2229-2232, 1987.
[6] T. J. Sejnowski and C. R. Rosenberg, "Parallel networks that learn to pronounce English text,"Complex Syst., vol. 1, pp. 145-168, 1987.
[7] D. E. Eckhardt, Jr., and L. D. Lee, "A theoretical basis for the analysis of multiversion software subject to coincident errors,"IEEE Trans. Software Eng., vol. SE-11, pp. 1511-1517, 1985.
[8] G. T. Toussaint, "Bibliography on estimation of misclassification,"IEEE Trans. Inform. Theory, vol. 20, pp. 472-479, 1974.
[9] N. Qian and T. J. Sejnowsci, "Predicting the secondary structure of globular proteins using neural network models,"J. Molecular Biol., vol. 202, pp. 865-884, 1989.
[10] H. Bohr, J. Bohr, S. Brunak, R. M. J. Cotterill, B. Lautrup, L. Nørskov, O. H. Olsen, and S. B. Petersen, "Protein secondary structure and homology by neural networks,"Fed. European Biochem. Soc. Lett., vol. 241, pp. 223-228, 1988.
[11] R. P. Lippman, "An introduction to computing with neural nets,"IEEE ASSP Msg., vol. 4, pp. 4-22, 1987.
[12] R. G. Palmer, "Broken ergodicity,"Advances Phys., vol. 31, pp. 669-735, 1982.
[13] S. Kirkpatrick, C. D. Gelatt, Jr., and M. P. Vecchi, "Optimization by simulated annealing,"Science, vol. 220, pp. 671-680, 1983.
[14] T. Schneider and E. Stoll, "Molecular-dynamics study of a three dimensional one-component model for distortive phase transitions,"Phys. Rev. B, vol. 17, p. 1302, 1978.
[15] J. Riordan,An Introduction to Combinatorial Analysis. Princeton, NJ: Princeton University Press, 1980.
[16] F. Rosenblatt,Principles of Neurodynamics. New York: Spartan, 1959.
Index Terms:
pattern recognition; crossvalidation; performance; training; neural networks; classification; residual generalization error; learning systems; neural nets; pattern recognition
Citation:
L.K. Hansen, P. Salamon, "Neural Network Ensembles," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 12, no. 10, pp. 993-1001, Oct. 1990, doi:10.1109/34.58871
PURCHASE ARTICLE: $19
IEEE Xplore Subscribers
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb