Detection and tracking of moving objects are the essential steps of many video understanding applications such as traffic monitoring, video surveillance and visual event recognition. Moving object detection process segments the scene into foreground (moving) and background regions. Moving cast shadows cause serious problems in this process because they can easily be misclassified as foreground. This misclassification may lead to drastic changes in the shapes of objects or merging of multiple objects. In this paper, we present a method to detect moving cast shadows to improve the performance of moving object detection. The foreground regions are processed in terms of intensity, chromaticity, and reflectance ratio. To further refine the results, compactness constraint is enforced on the foreground and shadow masks. The algorithm exploits spatial and spectral information; no a priori knowledge about camera, illumination or object/scene characteristics are required. Obtained results show better performance compared to other work in recent literature.