In this paper we propose a new robust way of extracting accurate human silhouettes indoors with an active stereo camera. We first infer the parts of object and background areas of high confidence by fusing color, stereo matching information and image segmentation methods. Then the inferred areas(seeds) are incorporated in a graph cut. The experimental results were presented with image sequences taken with pan-tilt stereo camera. Our proposed algorithms were evaluated with respect to the ground truth data. We proved that our algorithms can outperform other methods that are based on either color/contrast or stereo/contrast principles alone.