This paper concerns the autonomous tracking of fish using a Remotely Operated Vehicle (ROV) equipped with a single camera. An efficient image processing algorithm is presented that enables pose estimation of a particular species of fish - a Large Mouth Bass. The algorithm uses a series of filters including the Gabor filter for texture, projection segmentation, and geometrical shape feature extraction to find the fishes distinctive dark lines that mark the body and tail. Feature based scaling then produces the position and orientation of the fish relative to the ROV. By implementing this algorithm on each frame of a series of video frames, successive relative state estimates can be obtained which are fused across time via a Kalman Filter. Video taken from a VideoRay MicroROV operating within Paradise Lake, Ontario, Canada was used to demonstrate off-line fish state estimation. In the future, this approach will be integrated within a closed-loop controller that allows the robot to autonomously follow the fish and monitor its behavior.