An Infrastructure for Realizing Custom-Tailored Augmented Reality User Interfaces
|
Augmented Reality (AR) technologies are rapidly expanding into new application areas. However, the development of AR user interfaces and appropriate interaction techniques remains a complex and time-consuming task. Starting from scratch is more common than building upon existing solutions. Furthermore, adaptation is difficult, often resulting in poor quality and limited flexibility with regard to user requirements. In order to overcome these problems, we introduce an infrastructure for supporting the development of specific AR interaction techniques and their adaptation to individual user needs. Our approach is threefold: a flexible AR framework providing independence from particular input devices and rendering platforms, an interaction prototyping mechanism allowing for fast prototyping of new interaction techniques, and a high-level user interface description, extending user interface descriptions into the domain of AR. The general usability and applicability of the approach is demonstrated by means of three example AR projects.
[1] 722 D. Abawi, R. Dörner, M. Haller, and J. Zauner, “Efficient Mixed Reality Application Development,” Proc. First European Conf. Visual Media Production, Mar. 2004.
[2] M. Abrams, “Device-Independent Authoring with UIML,” Proc. W3C Workshop Web Device Independent Authoring, 1999.
[3] R. Azuma, “A Survey of Augmented Reality,” Presence: Teleoperators and Virtual Environments, vol. 6, no. 4, pp. 355-385, 1997.
[4] M. Bauer, B. Bruegge, G. Klinker, A. MacWilliams, T. Reichert, S. Riß, C. Sandor, and M. Wagner, “Design of a Component-Based Augmented Reality Framework,” Proc. Second IEEE and ACM Int'l Symp. Augmented Reality (ISAR 2001), pp. 45-54, Oct. 2001.
[5] A. Bierbaum and C. Just, “Software Tools for Virtual Reality Application Development,” ACM SIGGRAPH 98 Course #14: Applied Virtual Reality, pp. 3.1-3.45, 1998.
[6] W. Broll, S. Grünvogel, I. Herbst, I. Lindt, M. Maercker, J. Ohlenburg, and M. Wittkämper, “Interactive Props and Choreography Planning with the Mixed Reality Stage,” Proc. Int'l Conf. Entertainment Computing (ICEC 2004), pp. 185-192, Sept. 2004.
[7] W. Broll, I. Lindt, J. Ohlenburg, M. Wittkämper, C. Yuan, T. Novotny, C. Mottram, A. Fatah, and A. Strothmann, “ARTHUR: A Collaborative Augmented Environment for Architectural Design and Urban Planning,” Proc. Fourth Int'l Symp. Human and Computers (HC 2004), pp. 102-109, J. Virtual Reality and Broadcasting, vol. 1, no. 1, 2004.
[8] B. Bruegge, A. MacWilliams, and T. Reicher, “Study on Software Architectures for Augmented Reality Systems,” report to the ARVIKA consortium; technical report TUM-I0410, 2004.
[9] D.A. Bowman, J.L. Gabbard, and D. Hix, “A Survey of Usability Evaluation in Virtual Environments: Classification and Comparison of Methods,” Presence: Teleoperators and Virtual Environments, vol. 11, no. 4, pp. 404-424, 2002.
[10] D. Bowman, E. Kruijff, J. LaViola, and I. Poupyrev, 3D User Interfaces: Theory and Practice. Boston: Addison-Wesley, 2004.
[11] F. Doil, W. Schreiber, T. Alt, and C. Patron, “Augmented Reality for Manufacturing Planning,” Proc. Workshop Virtual Environments 2003, pp. 71-76, 2003.
[12] C. Endres, A. Butz, and A. MacWilliams, “A Survey of Software Infrastructures and Frameworks for Ubiquitous Computing,” Mobile Information Systems J., pp. 41-80, Jan.-Mar. 2005.
[13] P. Figueroa, M. Green, and H.J. Hoover, “InTml: A Description Language for VR Applications,” Proc. Seventh Int'l Conf. 3D Web Technology, pp. 53-58, 2002.
[14] W. Friedrich, “ARVIKA-Augmented Reality for Development, Production and Service,” Proc. First Int'l Symp. Mixed and Augmented Reality (ISMAR 2002), pp. 3-4, 2002.
[15] J.L. Gabbard, D. Hix, and J.E. Swan, “User-Centered Design and Evaluation of Virtual Environments,” IEEE Computer Graphics and Applications, vol. 19, no. 6, pp. 51-59, Nov./Dec. 1999.
[16] H. Ishii and B. Ullmer, “Tangible Bits. Towards Seamless Interfaces between People, Bits and Atoms,” Proc. ACM Computer Human Interaction Conf. (CHI '97), 1997.
[17] H. Kato, M. Billinghurst, B. Blanding, and R. May, “ARToolKit,” technical report, Hiroshima City Univ., Dec. 1999.
[18] G. Klinker, A.H. Dutoit, M. Bauer, J. Bayer, V. Novak, and D. Matzke, “FataMorgan— A Presentation System for Product Design,” Proc. First Int'l Symp. Mixed and Augmented Reality (ISMAR 2002), 2002.
[19] C. Kulas, C. Sandor, and G. Klinker, “Towards a Development Methodology for Augmented Reality User Interfaces,” Proc. Int'l Workshop Exploring the Design and Eng. of Mixed Reality Systems (MIXER 2004), Jan. 2004.
[20] F. Ledermann and D. Schmalstieg, “APRIL: A High-Level Framework for Creating Augmented Reality Presentations,” Proc. IEEE Virtual Reality 2005, 2005.
[21] G.A. Lee, C. Nelles, M. Billinghurst, and G.J. Kim, “Immersive Authoring of Tangible Augmented Reality Applications,” Proc. Third Int'l Symp. Mixed and Augmented Reality (ISMAR 2004), 2004.
[22] I. Lindt, I. Herbst, and M. Maercker, “Interacting within the Mixed Reality Stage,” Proc. Workshop AVIR 2003, Sept. 2003.
[23] B. MacIntyre, M. Gandy, S. Dow, and J.D. Bolter, “DART: A Toolkit for Rapid Design Exploration of Augmented Reality Experiences,” Proc. User Interface Software and Technology Conf. (UIST '04), Oct. 2004.
[24] P. Milgram and F. Kishino, “A Taxonomy of Mixed Reality Visual Displays,” IEICE Trans. Information Systems, vol. E77-D, no. 12, Dec. 1994.
[25] J. Ohlenburg, I. Herbst, I. Lindt, T. Fröhlich, and W. Broll, “The MORGAN Framework: Enabling Dynamic Multi-User AR and VR Projects,” Proc. ACM Symp. Virtual Reality Software and Technology (VRST 2004), pp. 166-169, 2004.
[26] A. Olwal and S. Feiner, “Unit: Modular Development of Distributed Interaction Techniques for Highly Interactive User Interfaces,” Proc. Int'l Workshop Exploring the Design and Eng. of Mixed Reality Systems (MIXER 2004), 2004.
[27] A. Penn, C. Mottram, A.F. Schieck, M. Wittkämper, M. Störring, O. Romell, A. Strothmann, and F. Aish, “Augmented Reality Meeting Table: A Novel Multi-User Interface for Architectural Design,” Recent Advances in Design and Decision Support Systems in Architecture and Urban Planning, J.P. van Leeuwen and H. Timmermans, eds., pp. 213-231, Kluwer Academic Publishers, 2004.
[28] W. Piekarski, B. Thomas, D. Hepworth, B. Gunther, and V. Demczuk, “An Architecture for Outdoor Wearable Computers to Support Augmented Reality and Multimedia Applications,” Proc. Third Int'l Conf. Knowledge-Based Intelligent Information Eng. Systems, pp. 70-73, 2000.
[29] W. Piekarski and B. Thomas, “ARQuake: The Outdoor Augmented Reality Gaming System,” Comm. ACM, vol. 45, no. 1, pp. 36-38, 2002.
[30] C. Plessl, R. Enzler, H. Walder, J. Beutel, M. Platzner, L. Thiele, and G. Tröster, “The Case for Reconfigurable Hardware in Wearable Computing,” Personal Ubiquitous Computing, vol. 7, no. 5, pp. 299-308, 2004.
[31] M. Ponder, G. Papagiannakis, T. Molet, N. Magnenat-Thalmann, and D. Thalmann, “VHD++ Development Framework: Towards Extendible, Component Based VR/AR Simulation Engine Featuring Advanced Virtual Character Technologies,” Proc. Computer Graphics Int'l 2003 (CGI 2003), 2003.
[32] G. Reitmayr and D. Schmalstieg, “An Open Software Architecture for Virtual Reality Interaction,” Proc. ACM Symp. Virtual Reality Software & Technology 2001 (VRST 2001), pp. 47-54, 2001.
[33] C. Sandor and T. Reicher, “CUIML: A Language for the Generation of Multimodal Human-Computer Interfaces,” Proc. European UIML Conf., 2001.
[34] D. Schmalstieg, A. Fuhrmann, and G. Hesina, “Bridging Multiple User Interface Dimensions with Augmented Realty,” Proc. IEEE and ACM Int'l Symp. Augmented Reality (ISAR 00), pp. 20-29, 2000.
[35] R. Simon, M. Jank, and F. Wegscheider, “A Generic UIML Vocabulary for Device- and Modality Independent User Interfaces,” Proc. 13th Int'l World Wide Web Conf., May 2004.
[36] S. Sotiriou, E. Chatzichristou, S. Savas, A. Pyrini, N. Ouzounoglou, M. Gargalakos, R. Makri, P. Tsenes, L.D. Dierking, H. Salmi, A. Hoffstein, and S. Rosenfeld, “CONNECT: Designing the Classroom of Tomorrow by Using Advanced Technologies to Connect Formal and Informal Learning Environments,” Proc. IADIS Int'l Conf. Cognition and Exploratory Learning in Digital Age (CELDA 2004), Dec. 2004.
[37] A. Tang, C. Owen, F. Biocca, and W. Mou, “Comparative Effectiveness of Augmented Reality in Object Assembly,” Proc. Conf. Human Factors in Computing Systems, pp. 73-80, 2003.
[38] T. Starner, S. Mann, B. Rhodes, J. Levine, J. Healey, D. Kirsch, R. Picard, and A. Pentland, “Augmented Reality through Wearable Computing,” Presence, vol. 6, no. 4, pp. 386-398, Aug. 1997.
[39] S. Trewin, G. Zimmermann, and G. Vanderheiden, “Abstract User Interface Representations: How Well Do They Support Univeral Access,” Proc. Computer-Human Interaction Conf., 2003.
[40] A. Webster, S. Feiner, B. MacIntyre, W. Massie, and T. Krueger, “Augmented Reality in Architectural Construction, Inspection and Renovation,” Proc. ASCE Third Congress Computing in Civil Eng., pp. 913-919, 1996.
[41] S. Wiedenmaier, O. Oehme, L. Schmidt, and H. Luczak, “Augmented Reality (AR) for Assembly Processes Design and Experimental Evaluation,” Int'l J. Human-Computer Interaction, vol. 16, no. 3, pp. 497-514, 2003.
[42] E. Woods, M. Billinghurst, J. Looser, G. Aldridge, D. Brown, B. Garrie, and C. Nelles, “Augmenting the Science Centre and Museum Experience,” Proc. Second Int'l Conf. Computer Graphics and Interactive Techniques in Australasia and South East Asia (GRAPHITE 2004), pp. 230-236, 2004.
[43] J. Zauner, M. Haller, A. Brandl, and W. Hartmann, “Authoring of a Mixed Reality Assembly Instructor for Hierarchical Structures,” Proc. Second Int'l Symp. Mixed and Augmented Reality (ISMAR 2003), Oct. 2003.
[44] J. Zauner and M. Haller, “Authoring of Mixed Reality Applications Including Multi-Marker Calibration for Mobile Devices,” Proc. 10th Eurographics Symp. Virtual Environments (EGVE 2004), pp. 87-90, June 2004.
Index Terms:
Index Terms- Distributed systems, distributed applications, multimedia information systems, artificial, augmented, and virtual realities, user interfaces, collaborative computing, graphics systems, distributed/network graphics, methodology and techniques, device independence, graphics data structures and data types, interaction techniques, three-dimensional graphics and realism, virtual reality.
Citation:
Wolfgang Broll, Irma Lindt, Jan Ohlenburg, Iris Herbst, Michael Wittk?mper, Thomas Novotny, "An Infrastructure for Realizing Custom-Tailored Augmented Reality User Interfaces," IEEE Transactions on Visualization and Computer Graphics, vol. 11, no. 6, pp. 722-733, Nov./Dec. 2005, doi:10.1109/TVCG.2005.90
PURCHASE ARTICLE: $19
IEEE Xplore Subscribers
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb