The ability to recognize emotion is one of the hallmarks of emotional intelligence. This paper proposed to recognize emotion using physiological signals obtained from multiple subjects without much discomfort from the body surface. Four signals, electrocardiogram (ECG), skin temperature (SKT), skin conductance (SC) and respiration were selected to extract features for recognition. We collected a set of data from 60 undergraduates when experiencing the target emotion elicited by film clips. Canonical correlation analysis was used to find the relationship between emotion and extracted features. Using 17 features, 20 features and 22 features, recognition accuracy is 82%, 85.3%, 85.3% respectively.