This paper presents a novel face detection method in video by using Discriminating Feature Analysis (DFA) and Support Vector Machine (SVM). Our method first incorporates temporal and skin color information to locate the field of interests. Then the face class is modelled using a small training set and the nonface class is defined by choosing nonface images that lie close to the face class. Finally, the SVM classifier together with Bayesian statistical analysis procedure applies the efficient features defined by DFA for face and nonface classification. Experiments using both still images and video streams show the feasibility of our new face detection method. In particular, when using 92 images (containing 282 faces) from the MIT-CMU test sets, our method achieves 98.2% correct face detection accuracy with 2 false detections. When using video streams, our method detects faces reliably with computational efficiency of more than 20 frames per second.