Low-Cost Telepresence for Collaborative Virtual Environments
|
Abstract—We present a novel low-cost method for visual communication and telepresence in a CAVE^{\rm{TM}}-like environment, relying on 2D stereo-based video avatars. The system combines a selection of proven efficient algorithms and approximations in a unique way, resulting in a convincing stereoscopic real-time representation of a remote user acquired in a spatially immersive display. The system was designed to extend existing projection systems with acquisition capabilities requiring minimal hardware modifications and cost. The system uses infrared-based image segmentation to enable concurrent acquisition and projection in an immersive environment without a static background. The system consists of two color cameras and two additional b/w cameras used for segmentation in the near-IR spectrum. There is no need for special optics as the mask and color image are merged using image-warping based on a depth estimation. The resulting stereo image stream is compressed, streamed across a network, and displayed as a frame-sequential stereo texture on a billboard in the remote virtual environment.
[1] 156 J. Allard, E. Boyer, J.-S. Franco, G. Raffin, and C. Menier, “Marker-Less Real Time 3D Modeling for Virtual Reality,” Immersive Projection Technology, 2004.
[2] L. Chong, H. Kaczmarski, and C. Goudeseune, “Video Stereoscopic Avatars for the Cave Virtual Environment,” Proc. IEEE Int'l Symp. Mixed and Augmented Reality, 2004.
[3] C. Cruz-Neira, D.J. Sandin, and T.A. DeFanti, “Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the Cave,” Proc. 20th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 135-142, 1993.
[4] J.W. Davis and A.F. Bobick, “Sideshow: A Silhouette-Based Interactive Dual-Screen Environment,” Technical Report 457, Mass. Inst. of Technology (MIT) Media Lab, 1998.
[5] P. Debevec, A. Wenger, C. Tchou, A. Gardner, J. Waese, and T. Hawkins, “A Lighting Reproduction Approach to Live-Action Compositing,” ACM Trans. Graphics, vol. 21, no. 3, pp. 547-556, 2002.
[6] A. Fusiello, E. Trucco, and A. Verri, “A Compact Algorithm for Rectification of Stereo Pairs,” Machine Vision and Applications, vol. 12, no. 1, pp. 16-22, 2000.
[7] S.J. Gibbs, C. Arapis, and C.J. Breitender, “Teleport—Towards Immersive Copresence,” Multimedia Systems, vol. 7, no. 3, pp. 214-221, 1999.
[8] M. Gross, S. Wuermlin, M. Naef, E. Lamboray, C. Spagno, A. Kunz, E. K-Meier, T. Svoboda, L. Gool, S. Lang, K. Strehlke, A.V. Moere, and O. Staadt, “Blue-C: A Spatially Immersive Display and 3D Video Portal for Telepresence,” ACM Trans. Graphics, vol. 22, no. 3, pp. 819-827, 2003.
[9] A. Laurentini, “The Visual Hull Concept for Silhouette-Based Image Understanding,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 2, pp. 150-162, Feb. 1994.
[10] S.-Y. Lee, I.-J. Kim, S.C. Ahn, H. Ko, M.-T. Lim, and H.-G. Kim, “Real Time 3D Avatar for Interactive Mixed Reality,” Proc. ACM SIGGRAPH Int'l Conf. Virtual Reality Continuum and its Applications in Industry (VRCAI '04), pp. 75-80, 2004.
[11] W. Matusik, “Image-Based Visual Hulls,” master's thesis, Mass. Inst. of Tech nology, 2001.
[12] W. Matusik, C. Buehler, and L. McMillan, “Polyhedral Visual Hulls for Real-Time Rendering,” Proc. 12th Eurographics Workshop Rendering, pp. 115-125, 2001.
[13] W. Matusik, C. Buehler, R. Raskar, S.J. Gortler, and L. McMillan, “Image-Based Visual Hulls,” Proc. SIGGRAPH '00, pp. 369-374, 2000.
[14] M. Naef, O. Staadt, and M. Gross, “Blue-C API: A Multimedia and 3D Video Enhanced Toolkit for Collaborative VR and Telepresence,” Proc. ACM SIGGRAPH Int'l Conf. Virtual Reality Continuum and Its Applications in Industry (VRCAI '04), pp. 11-18, 2004.
[15] P.J. Narayana, P. Rander, and T. Kanade, “Constructing Virtual Worlds Using Dense Stereo,” Proc. Int'l Conf. Computer Vision (ICCV '98), pp. 3-10, 1998.
[16] T. Ogi, T. Yamada, M. Hirose, M. Fujita, and K. Kuzuu, “High Presence Remote Presentation in the Shared Immersive Virtual World,” Proc. IEEE Virtual Reality Conf. (VR '03), 2003.
[17] T. Ogi, T. Yamada, Y. Kurita, Y. Hattori, and M. Hirose, “Usage of Video Avatar Technology for Immersive Communication,” Proc. First Int'l Workshop Language Understanding and Agents for Real World Interaction, 2003.
[18] T. Ogi, T. Yamada, K. Tamagawa, M. Kano, and M. Hirose, “Immersive Telecommunication Using Stereo Video Avatar,” Proc. Virtual Reality 2001 Conf. (VR '01), 2001.
[19] S. Pollard and S. Hayes, “View Synthesis by Edge Transfer with Application to the Generation of Immersive Video Objects,” Proc. ACM Symp. Virtual Reality Software and Technology, pp. 91-98, 1998.
[20] R. Raskar, G. Welch, M. Cutts, A. Lake, L. Stesin, and H. Fuchs, “The Office of the Future: A Unified Approach to Image-Based Modeling and Spatially Immersive Displays,” Proc. SIGGRAPH '98, pp. 179-188, 1998.
[21] A. Sadagic, H. Towles, J. Lanier, H. Fuchs, A. Van Dam, K. Daniilidis, J. Mulligan, L. Holden, and B. Zeleznik, “National Tele-Immersion Initiative: Towards Compelling Tele-Immersive Collaborative Environments,” Proc. Medicine Meets Virtual Reality Conf., 2001.
[22] M. Slater, A. Sadagic, M. Usoh, and R. Schroeder, “Small Group Behaviour in a Virtual and Real Environment: A Comparative Study,” Presence: Teleoperators and Virtual Environments, vol. 9, no. 1, pp. 37-51, 2000.
[23] Y. Suh, D. Hong, and W. Woo, “2.5D Video Avatar Augmentation for VR Photo,” Proc. International Conf. Artificial Reality and Telexistence (ICAT '02), 2002.
[24] S. Wuermlin, E. Lamboray, O. Staadt, and M. Gross, “3D Video Recorder: A System for Recording and Playing Free-Viewpoint Video,” Computer Graphics Forum, vol. 22, no. 2, pp. 181-193, 2003.
[25] G. Taubin, “Camera Model and Triangulation,” Note for EE-148: 3D Photography, CalTech, 2001.
[26] S. Subramanian, V. Rajan, D. Keenan, A. Johnson, D. Sandin, and T. DeFanti, “A Realistic Video Avatar System for Networked Virtual Environments,” Proc. Seventh Ann. Immersive Projection Technology Symp. (IPT '02), 2002.
[27] K. Yasuda, T. Naemura, and H. Harashima, “Thermo-Key: Human Region Segmentation from Video,” IEEE Computer Graphics and Application, vol. 24, no. 1, pp. 26-30, Jan./Feb. 2004.
Index Terms:
Remote systems, segmentation, stereo, virtual reality, teleconferencing.
Citation:
Seon-Min Rhee, Remo Ziegler, Jiyoung Park, Martin Naef, Markus Gross, Myoung-Hee Kim, "Low-Cost Telepresence for Collaborative Virtual Environments," IEEE Transactions on Visualization and Computer Graphics, vol. 13, no. 1, pp. 156-166, Jan./Feb. 2007, doi:10.1109/TVCG.2007.17
PURCHASE ARTICLE: $19
IEEE Xplore Subscribers
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb