Most of the speech recognition systems are all based on the technology of HMM because that HMM is a valid probability tool for modeling and recognizing time-series signal and can provide a better statistical architecture. But the weakness such as the poor performance in classification and the high dependence on the statistical knowledge of the pre-experimentation is unconquerable. So we introduce the Support Vector Machine which is a powerful machine-learning scheme and has been used in the classifiers of the multidimensional non-linear successfully. In this paper, we present a speech recognition system based on the hybrid HMM/SVM architecture. Additional, several issues that arise as a result of the hybrid framework have been addressed, including estimation of posterior probability and the use of segment-level data.. Having been proved in the experiment, the hybrid system has combined the predominance of both HMM and SVM and has a better performance than traditional one.