Support vector machines (SVMs) have been successfully used in many classification fields. However, conventional SVMs do not consider rejecting inputs and thus suffer from false alarms. The first reason for this is that every input is assumed to belong to one of the object classes and is accepted in some class. In this paper, we will show that the second reason is that conventional SVMs do not describe each object class well. Thus, use of an output threshold does not solve this problem. We present a new support vector representation and discrimination machine (SVRDM), which has a discrimination capability comparable to that of the conventional SVM, and also offers good rejection ability. False alarm rates are greatly reduced. We analyze the properties of these two classifiers (SVM and SVRDM) in transformed feature space and compare their performances using both synthetic and real data.