In this paper, we present a new segment-based stereo matching algorithm using graph cuts. In our approach, the reference image is divided into non-overlapping homogeneous segments and the scene structure is represented as a set of planes in the disparity space. The stereo matching problem is formulated as an energy minimization problem in the segment domain instead of the traditional pixel domain. Graph cuts technique is used to fast approximate the optimal solution, which assigns the corresponding disparity plane to each segment. Experiments demonstrate that the performance of our algorithm is comparable to the state-of-the-art stereo algorithms on various data sets. Furthermore, strong performance is achieved in the conventionally difficult areas such as: textureless regions, disparity discontinuous boundaries and occluded portions.