The concept of the unidentified pattern comes from theoretic analysis of pattern space and experimental analysis of pattern distribution. The fuzzy mapping algorithm has been specially designed for the mapping of the unidentified pattern according to the clustering principle of normal and abnormal pattern in the normal and attack period of time. It provides the computation foundation, on which the concept of the unidentified pattern can be introduced into the anomaly detection of privileged programs providing host services. Experiment results indicate that the proposed modeling method of anomaly detection evidently increases the deviation of attack behaviors from normal profile, and ultimately increases detection capability against known and unknown attacks. The research achievements have laid the strong theoretical and experimental foundations to develop the security technologies of system services.