Research in visual surveillance systems is shifting from using few stationary, passive cameras to employing large heterogeneous sensor networks. One promising type of sensor in particular is the Pan-Tilt-Zoom (PTZ) camera, which can cover a potentially much larger area than passive cameras, and can obtain much higher resolution imagery through zoom capacity. In this paper, a system that can track objects with multiple calibrated PTZ cameras in a cooperative fashion is presented. Tracking and calibration results are combined with several image processing techniques in a statistical segmentation framework, through which the cameras can hand over targets to each other. A prototype system is presented that operates in real time.