Detecting Motion Regions in the Presence of a Strong Parallax from a Moving Camera by Multiview Geometric Constraints
|
We present a method for detecting motion regions in video sequences observed by a moving camera, in the presence of strong parallax due to static 3D structures. The proposed method classifies each image pixel into planar background, parallax or motion regions by sequentially applying 2D planar homographies, the epipolar constraint and a novel geometric constraint, called "structure consistency constraint". The structure consistency constraint, as the main contribution of this paper, is derived from the relative camera poses among three frames and implemented within the "Plane+Parallax" framework. Unlike previous planar-parallax constraints, the proposed constraint does not require the reference plane to be constant across multiple views. It directly measures the inconsistency between the projective structures from the same point under camera motion and reference plane change. The structure consistency constraint is capable of detecting moving objects followed by a moving camera in the same direction, a so called degenerate configuration where the epipolar constraint fails. We demonstrate the effectiveness and robustness of our method with experimental results on real-world video sequences.
[1] 1627 J.L. Barron, D.J. Fleet, and S.S. Beauchemin, “Performance of Optical Flow Techniques,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 43-77, 1994.
[2] S. Birchfield, “An Implementation of the Kanade-Lucas-Tomasi Feature Tracker,” http://www.ces.clemson.edu/stbklt/, 2006.
[3] M. Brown and D.G. Lowe, “Recognizing Panoramas,” Proc. Int'l Conf. Computer Vision, pp. 1218-1225, 2003.
[4] M.A. Fischler and R.C. Bolles, “Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography,” Comm. ACM, vol. 24, pp. 381-395, 1981.
[5] R.C. Gonzalez and R.E. Woods, Digital Image Processing. Prentice Hall, 2002.
[6] R. Hartley, “In Defence of the 8-Point Algorithm,” Proc. Int'l Conf. Computer Vision, pp. 1064-1070, 1995.
[7] R. Hartley and R. Vidal, “The Multibody Trifocal Tensor: Motion Segmentation from 3 Perspective Views,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 769-775, 2004.
[8] R.I. Hartley and A. Zisserman, Multiple-View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004.
[9] P.J. Huber, Robust Statistics. John Wiley & Sons, 2003.
[10] M. Irani and P. Anandan, “A Unified Approach to Moving Object Detection in 2D and 3D Scenes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 6, pp. 577-589, June 1998.
[11] M. Irani, P. Anandan, and D. Weinshall, “From Reference Frames to Reference Planes: Multi-View Parallax Geometry and Applications,” Proc. European Conf. Computer Vision, vol. 2, pp. 829-845, 1998.
[12] J. Kang, I. Cohen, and G. Medioni, “Continuous Tracking within and across Camera Streams,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 267-272, 2003.
[13] J. Kang, I. Cohen, G. Médioni, and C. Yuan, “Detection and Tracking of Moving Objects from a Moving Platform in Presence of Strong Parallax,” Proc. Int'l Conf. Computer Vision, pp. 10-17, 2005.
[14] R. Kumar, P. Anandan, and K. Hanna, “Direct Recovery of Shape from Multiple Views: A Parallax-Based Approach,” Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 685-688, 1994.
[15] M.I. Lourakis, A.A. Argyros, and S.C. Orphanoudakis, “Independent 3D Motion Detection Using Residual Parallax Normal Flow Fields,” Proc. Int'l Conf. Computer Vision, pp. 1012-1017, 1998.
[16] Q.-T. Luong and O.D. Faugeras, “The Fundamental Matrix: Theory, Algorithms, and Stability Analysis,” Int'l J. Computer Vision, vol. 17, no. 1, pp. 43-75, 1996.
[17] V.Y. Mariano, J. Min, J.-H. Park, R. Kasturi, D. Mihalcik, H. Li, D. Doermann, and T. Drayer, “Performance Evaluation of Object Detection Algorithms,” Proc. Int'l Conf. Pattern Recognition, vol. 3, pp. 965-969, 2002.
[18] B. Matei, B. Georgescu, and P. Meer, “A Versatile Method for Trifocal Tensor Estimation,” Proc. Int'l Conf. Computer Vision, vol. II, pp. 578-585, 2001.
[19] J. Nocedal and S.J. Wright, Numerical Optimization. Springer, 1999.
[20] H.S. Sawhney, Y. Guo, J. Asmuth, and R. Kumar, “Independent Motion Detection in 3D Scenes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1191-1199, Nov. 2000.
[21] A. Shashua and N. Navab, “Relative Affine Structure: Canonical Model for 3D from 2D Geometry and Applications,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 9, pp. 873-883, Sept. 1996.
[22] A. Shashua and L. Wolf, “Homography Tensors: On Algebraic Entities that Represent Three Views of Static or Moving Planar Points,” Proc. European Conf. Computer Vision, vol. 1, pp. 507-521, 2000.
[23] C. Stauffer and W. Grimson, “Adaptive Background Mixture Models for Real-Time Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 246-252, 1999.
[24] P. Torr, “Outlier Detection and Motion Segmentation,” PhD dissertation, Eng. Dept., Univ. of Oxford, 1995.
[25] P. Torr, “Geometric Motion Segmentation and Model Selection,” Philosophical Trans.: Math., Physical and Eng. Sciences, vol. 356, no. 1740, pp. 1321-1340, 1998.
[26] R. Vidal, Y. Ma, S. Soatto, and S. Sastry, “Two-View Multibody Structure from Motion,” Int'l J. Computer Vision, vol. 68, no. 1, pp.7-25, 2006.
[27] Z. Zhang, “Determining the Epipolar Geometry and Its Uncertainty: A Review,” Int'l J. Computer Vision, vol. 27, no. 2, pp.161-195, 1998.
[28] B. Zitova and J. Flusser, “Image Registration Methods: A Survey,” Image and Vision Computing, vol. 21, pp. 977-1000, 2003.
Index Terms:
Motion detection, multiple view geometry, epipolar constraint, plane plus parallax
Citation:
Chang Yuan, Gerard Medioni, Jinman Kang, Isaac Cohen, "Detecting Motion Regions in the Presence of a Strong Parallax from a Moving Camera by Multiview Geometric Constraints," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 9, pp. 1627-1641, June 2007, doi:10.1109/TPAMI.2007.1084
PURCHASE ARTICLE: $19
IEEE Xplore Subscribers
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb