Markus Tatzgern
Extended Reality Researcher
I am Associate Professor of Creative Technologies at the University of Warwick. Prior to that I was Professor for Mixed Reality and Game Development and Head of Research at the Department of Creative Technologies, Salzburg University of Applied Sciences. My work is situated at the intersection of XR, human–computer interaction, and generative AI.
My research focuses on visualization and interaction techniques for Augmented and Virtual Reality, aiming to seamlessly bridge digital and physical worlds. I explore adaptive visualization, multimodal interaction, and AI-driven approaches to create human-centered AR/VR solutions, with applications in training, learning, healthcare, human–robot interaction, and everyday augmentation.
I received my M.S. and Ph.D. with highest distinction from Graz University of Technology, where I also worked as a researcher at the Christian Doppler Laboratory for Handheld Augmented Reality. My work has been published at top-tier venues including ACM CHI, IEEE ISMAR, IEEE VR, and IEEE TVCG, and has resulted in several patents on augmented reality visualization. My research has been recognized with multiple honorable mention awards.
I contribute actively to the scientific community as Program Committee member and reviewer, and served as Science & Technology Program Chair of IEEE ISMAR 2024 and Papers Chair of IEEE ISMAR 2025. I am Associate Editor of IEEE TVCG and currently serve as Treasurer of ACM SIGCHI Austria. Beyond academia, I am committed to outreach, engaging with industry, media, and the public to advance understanding and adoption of immersive technologies.
News
- Where’s the Team Spirit? An Exploratory Study on Team Development Through Co-located Tablet-Based VR at ACM CSCW 2026, Salt Lake City.
- Paper Magnification Lenses for Small-Scale Augmented Reality Instructions and poster Augmented Reality Action Summaries for Situated Instructions of Repetitive Tasks at IEEE ISMAR 2026, Bari.
- Started as Associate Professor of Creative Technologies at the University of Warwick.
- Co-organized the workshop Next Steps for Augmented Reality On-the-Move at ACM CHI 2026, Barcelona.
- Two papers at ACM VRST 2025, Montreal: From Lab to Sidewalk and SpatialMouse.
- TVCG paper See What I Mean? Mobile Eye-Perspective Rendering presented at IEEE ISMAR 2025, Daejeon.
Publications
2026
-
CSCW Where's the Team Spirit? An Exploratory Study on Team Development Through Co-located Tablet-Based VR
Proceedings of the ACM on Human-Computer Interaction (PACMHCI), CSCW Issue, 2026
We explore how narrative-driven asymmetric VR experiences can support the development of teamwork-related knowledge, skills, and attitudes (KSAs), such as communication, coordination, trust, and reflexivity. We present the design and evaluation of a tablet-based VR training experience structured around spatial separation, tool asymmetry, and interdependent tasks that require verbal coordination. The experience was designed based on interviews with HR professionals and mapped to a framework of established KSAs. We conducted a co-located user study (N=16) that involved two consecutive collaborative scenarios. Our findings show that users adapted dynamically using verbal exchange, role negotiation, and shared representations to coordinate under asymmetric conditions. We also observed active application of teamwork KSAs. Based on our insights, we present design recommendations for creating effective immersive team training interventions.
@article{paraschivoiu2026where, author = {Paraschivoiu, Irina and Layer-Wagner, Thomas and Neundlinger, Klaus and Rack, Simone and Tatzgern, Markus}, title = {Where's the Team Spirit? An Exploratory Study on Team Development Through Co-located {Tablet-Based} {VR}}, journal = {Proceedings of the ACM on Human-Computer Interaction}, volume = {10}, number = {6}, pages = {1--31}, year = {2026}, month = sep, publisher = {Association for Computing Machinery}, issn = {2573-0142}, doi = {10.1145/3817040}, url = {https://doi.org/10.1145/3817040} } -
ISMAR Magnification Lenses for Small-Scale Augmented Reality Instructions
IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2026
Small-scale objects that are difficult to see or distinguish from their surroundings are common in assembly and maintenance procedures. Augmented reality (AR) instructions involving such small parts may be difficult to perceive and lead to misinterpretations. This paper explores the use of AR magnification visualizations for small-scale instructions. Through an iterative design process and evaluation, we examine how magnified AR views impact user behavior, visual attention, and subjective perception in the context of AR instructions. Our findings demonstrate that live AR magnification assists task guidance, providing a strong foundation for future work on AR-assisted instruction and precision interaction design.
@inproceedings{skreinig2026magnification, author = {Skreinig, Lucchas Ribeiro and Kalkofen, Denis and Ritter, David and Mohr, Peter and Tatzgern, Markus and Schmalstieg, Dieter and Stanescu, Ana}, title = {Magnification Lenses for {Small-Scale} Augmented Reality Instructions}, booktitle = {Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR)}, year = {2026}, publisher = {IEEE}, location = {Bari, Italy}, url = {https://immersive-technology-lab.github.io/projects/smallparts/assets/ribeiroskreinig_ismar26_paper.pdf}, note = {To appear} } -
ISMAR Poster Augmented Reality Action Summaries for Situated Instructions of Repetitive Tasks
IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Poster, 2026
Augmented reality instructions help people understand how to operate, assemble, or maintain objects by overlaying visual guidance directly onto the physical environment. However, repeatedly presenting detailed instructions can negatively impact performance. We introduce an interactive AR instruction system that provides detailed guidance on the first occurrence of actions and a summary visualization for repeated actions. We use a multimodal large language model to automatically derive information about actions, groups of actions, and recurring patterns, thereby increasing the scalability of the AR guidance system. A pilot user study shows that summary visualizations are preferred over repetitive step-by-step instructions.
@inproceedings{skreinig2026augmented, author = {Skreinig, Lucchas Ribeiro and Stanescu, Ana and Mohr, Peter and Ebner, Christoph and Tatzgern, Markus and Schmalstieg, Dieter and Kalkofen, Denis}, title = {Augmented Reality Action Summaries for Situated Instructions of Repetitive Tasks}, booktitle = {Adjunct Proceedings of the IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct)}, year = {2026}, publisher = {IEEE}, location = {Bari, Italy}, url = {https://immersive-technology-lab.github.io/projects/actionsummaries/assets/ribeiroskreinig_ismara26_poster.pdf}, note = {To appear} } -
CHI Workshop Next Steps for Augmented Reality On-the-Move: Challenges and Opportunities
Extended Abstracts of the ACM CHI Conference on Human Factors in Computing Systems (CHI EA), 2026
Recent advancements in augmented reality (AR) technologies have brought us closer to the vision of everyday ubiquitous computing and pervasive AR use. State-of-the-art AR glasses now enable mobile, on-the-move experiences that extend beyond laboratory settings. This promise has already been explored across diverse mobile contexts, including transportation, entertainment, shopping, and tourism. This workshop focuses on AR on-the-move, examining how AR can support interaction in dynamic settings where environmental and social contexts shift rapidly and users are in locomotion. We will discuss both the unique challenges and the rich opportunities such scenarios present for meaningful augmentation and new forms of interaction. By uniting established scholars and emerging researchers from across HCI, this workshop seeks to map out critical directions for future inquiry. Through hands-on activities and reflection, participants will collectively identify key opportunities, challenges, and next steps for advancing research on AR on-the-move.
@inproceedings{stefanidi2026next, author = {Stefanidi, Helen and van Rheden, Vincent and He, Linjia and Adiwangsa, Michelle and Sünderkamp, Jan-Hendrik and Itzlinger, Alina and Matviienko, Andrii and Williamson, Julie R. and Oakley, Ian and Tatzgern, Markus and Meschtscherjakov, Alexander}, title = {Next Steps for Augmented Reality {On-the-Move}: Challenges and Opportunities}, booktitle = {Proceedings of the Extended Abstracts of the 2026 CHI Conference on Human Factors in Computing Systems}, series = {CHI EA '26}, pages = {1--9}, year = {2026}, month = apr, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, location = {Barcelona, Spain}, doi = {10.1145/3772363.3778775}, url = {https://doi.org/10.1145/3772363.3778775} }
2025
-
VRST From Lab to Sidewalk: Exploring AR Interaction Techniques Outdoors and in Motion using Fitts’ Law
ACM Symposium on Virtual Reality Software and Technology (VRST), 2025
With the emergence of consumer-grade mobile Augmented Reality (AR) head-mounted displays (HMDs), it is increasingly important to study interaction modalities in mobile, outdoor contexts to explore user interactions under real-world conditions and ensure ecological validity. We present findings from a user study examining how state-of-the-art interaction techniques, originally developed for stationary indoor use, perform in outdoor walking contexts. We compare three techniques using a Fitts’ law task: (1) eye gaze for target selection combined with a pinch gesture for activation, (2) hand-controlled raycasting for selection with pinch activation, and (3) a go-go-hand-inspired technique that enables users to reach distant targets via virtual touch with a virtual hand representation. We report both performance metrics and subjective user feedback for these techniques.
@inproceedings{sunderkamp2025from, author = {Sünderkamp, Jan-Hendrik and Stefanidi, Helen and Meschtscherjakov, Alexander and Tatzgern, Markus}, title = {From Lab to Sidewalk: Exploring {AR} Interaction Techniques Outdoors and in Motion using {Fitts}' Law}, booktitle = {Proceedings of the 2025 31st ACM Symposium on Virtual Reality Software and Technology}, series = {VRST '25}, pages = {1--11}, year = {2025}, month = nov, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, location = {Montreal QC Canada}, doi = {10.1145/3756884.3766006}, url = {https://doi.org/10.1145/3756884.3766006} } -
VRST SpatialMouse: A Hybrid Pointing Device for Seamless Interaction Across 2D and 3D Space
ACM Symposium on Virtual Reality Software and Technology (VRST), 2025
We introduce the SpatialMouse, a hybrid pointing device that combines the capabilities of a desktop mouse with the spatial input of a virtual reality (VR) controller, enabling seamless transitions between 2D and 3D interaction spaces in immersive mixed reality environments. Holistic usage scenarios in mixed reality involve tasks suited alternately to 2D or 3D information spaces. Yet, existing input devices excel in either 2D or 3D, but not both, making it necessary to switch between multiple input devices (e.g., mouse and VR controller). Our SpatialMouse addresses this issue, offering the affordances of a desktop mouse for indirect 2D pointing and the spatial capabilities of VR controllers with six degrees of freedom. In a user study with 12 participants, our prototype significantly reduced perceived task load and improved user experience compared to switching between separate devices. We extract design recommendations to further support such hybrid input approaches.
@inproceedings{hubenschmid2025spatialmouse, author = {Hubenschmid, Sebastian and Zagermann, Johannes and Erb, Robin and Feuchtner, Tiare and Grubert, Jens and Tatzgern, Markus and Schmalstieg, Dieter and Reiterer, Harald}, title = {{SpatialMouse}: A Hybrid Pointing Device for Seamless Interaction Across {2D} and {3D} Spaces}, booktitle = {Proceedings of the 2025 31st ACM Symposium on Virtual Reality Software and Technology}, series = {VRST '25}, pages = {1--13}, year = {2025}, month = nov, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, location = {Montreal QC Canada}, doi = {10.1145/3756884.3766047}, url = {https://doi.org/10.1145/3756884.3766047} } -
HFIA Revisiting Hybrid Input Devices for Immersive Analytics
IEEE Conference on Human Factors in Immersive Analytics (HFIA), IEEE VIS Workshop, 2025
Hybrid user interfaces offer a promising framework for analysts to leverage the strengths of heterogeneous environments, such as desktops and mixed reality, and transition fluidly between them. Especially for immersive analytics, which often blends 3D interaction spaces with traditional 2D workflows, such hybrid user interfaces can reduce cognitive workload and improve user experience. Prior work has combined desktop-based statistical analysis with an insitu exploration of spatiotemporal data in mixed reality. However, switching between devices and input modalities in such combinations can disrupt users’ flow. We revisit the concept of hybrid input devices that operate seamlessly across 2D and 3D contexts by leveraging the complementary strengths of heterogeneous input devices. We thereby propose a hypothetical hybrid input device that combines input from a mouse and a spatial controller, for which we present potential interaction techniques and discuss potential opportunities and challenges for immersive analytics.
@inproceedings{hubenschmid2025revisiting, author = {Hubenschmid, Sebastian and Zagermann, Johannes and Elmqvist, Niklas and Feuchtner, Tiare and Grubert, Jens and Tatzgern, Markus and Schmalstieg, Dieter and Reiterer, Harald}, title = {Revisiting Hybrid Input Devices for Immersive Analytics}, booktitle = {2025 IEEE Conference on Human Factors in Immersive Analytics (HFIA)}, pages = {29--33}, year = {2025}, month = nov, publisher = {IEEE}, location = {Vienna, Austria}, doi = {10.1109/hfia68651.2025.00011}, url = {https://doi.org/10.1109/hfia68651.2025.00011} } -
TVCG / ISMAR See What I Mean? Mobile Eye-Perspective Rendering for Optical See-Through Head-Mounted Displays
IEEE Transactions on Visualization and Computer Graphics (TVCG), ISMAR Special Issue, 2025
DOIPDF (open access)▶ VideoCode
Image-based scene understanding allows Augmented Reality (AR) systems to provide contextual visual guidance in unprepared, real-world environments. While effective on video see-through (VST) head-mounted displays (HMDs), such methods suffer on optical see-through (OST) HMDs due to misregistration between the world-facing camera and the user's eye perspective. To approximate the user's true eye view, we implement and evaluate three software-based eye-perspective rendering (EPR) techniques on a commercially available, untethered OST HMD (Microsoft HoloLens 2): (1) Plane-Proxy EPR, projecting onto a fixed-distance plane; (2) Mesh-Proxy EPR, using SLAM-based reconstruction for projection; and (3) Gaze-Proxy EPR, a novel eye-tracking-based method that aligns the projection with the user's gaze depth. A user study on real-world tasks underscores the importance of accurate EPR and demonstrates gaze-proxy as a lightweight alternative to geometry-based methods. We release our EPR framework as open source.
@article{emsenhuber2025see, author = {Emsenhuber, Gerlinde and Langlotz, Tobias and Kalkofen, Denis and Tatzgern, Markus}, title = {See what I Mean? Mobile {Eye-Perspective} Rendering for Optical {See-Through} {Head-Mounted} Displays}, journal = {IEEE Transactions on Visualization and Computer Graphics}, volume = {31}, number = {11}, pages = {9603--9613}, year = {2025}, month = nov, publisher = {IEEE}, issn = {1077-2626}, doi = {10.1109/tvcg.2025.3616739}, url = {https://doi.org/10.1109/tvcg.2025.3616739} } -
IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct), Demo, 2025
OST HMDs require correcting for the camera-to-eye offset whenever virtual cues are derived from camera imagery. We demonstrate two methods that align these cues with the user’s view by projecting the processed camera image onto a planar proxy: One keeps this plane at a constant depth, which is sufficient when parallax is small but causes misalignments otherwise, the other uses eye-tracking to adjust the plane dynamically to the user’s fixation, preserving alignment as the user shifts focus. We demonstrate the fixed-distance version on Microsoft HoloLens 2 and Snap AR Glasses, and the gaze-driven version on HoloLens 2.
@inproceedings{emsenhuber2025mobile, author = {Emsenhuber, Gerlinde and Smirnov, Michael and Langlotz, Tobias and Kalkofen, Denis and Zollmann, Stefanie and Tatzgern, Markus}, title = {Mobile Eye-perspective Rendering: Alignment of Image-based Vision Augmentations for Optical See-through Head-mounted Displays}, booktitle = {2025 IEEE International Symposium on Mixed and Augmented Reality Adjunct (ISMAR-Adjunct)}, pages = {917--918}, year = {2025}, month = oct, publisher = {IEEE}, location = {Daejeon, Korea, Republic of}, doi = {10.1109/ismar-adjunct68609.2025.00250}, url = {https://doi.org/10.1109/ismar-adjunct68609.2025.00250} } -
🏆 Honorable Mention for Best Paper
MobileHCI You’re making things AR-kward: Exploring Augmented Reality In-the-Wild
Proceedings of the ACM on Human-Computer Interaction (PACMHCI), MobileHCI Issue, 2025
Even though recent technological advancements have led to a growing interest in Augmented Reality (AR) head-mounted displays (HMDs), their actual deployment in everyday, outdoor and on-the-move applications remains uncertain. Most AR research is conducted in controlled laboratory settings, leaving a gap in our understanding of AR’s potentials and complexities in real-world environments. This paper explores the social aspects surrounding AR use in public spaces, by investigating the use of AR HMDs during two distinct use cases: shopping at a farmers market and outdoor presentations. Based on the analysis of observations and interviews with participants, passersby and vendors, we explore social impact of AR and share methodological and technological insights. We contribute seven lessons learned for researchers conducting AR studies in-the-wild. Our findings show that the current understanding of non-users should be revisited for in-the-wild AR studies. Furthermore, current AR HMDs lack social components, inducing awkwardness in social situations, which might fade with continuous exposure.
@article{stefanidi2025you, author = {Stefanidi, Helen and Sünderkamp, Jan-Hendrik and Tatzgern, Markus and Itzlinger, Alina and Meschtscherjakov, Alexander}, title = {You're making things {AR-kward}: Exploring Augmented Reality {In-the-Wild}}, journal = {Proceedings of the ACM on Human-Computer Interaction}, volume = {9}, number = {5}, pages = {1--24}, year = {2025}, month = sep, publisher = {Association for Computing Machinery}, issn = {2573-0142}, doi = {10.1145/3743740}, url = {https://doi.org/10.1145/3743740} } -
DIS Exploring AR In-the-Wild: An Autoethnographic Study at a Christmas Market
Companion Publication of the ACM Designing Interactive Systems Conference (DIS), 2025
Given the rapid advancements of Augmented Reality (AR) Head-Mounted Displays (HMDs), researchers are increasingly exploring their potential for everyday use, including outdoor and on-the-move settings. However, studies examining the use of AR HMDs in real-world environments remain scarce. Thus, we conducted an autoethnographic study capturing the lived experience of an AR expert using three different AR HMDs in the context of a crowded Christmas market. Our work contributes first-person insights into the experience, limitations, and social implications of interacting with AR HMDs in-the-wild; it shedding light on the social experience of using AR technology in public spaces and outlining design considerations regarding trust, context-aware deployment, and non-user perceptions. We highlight how device form factors, interface behaviors, and environmental components shape interaction quality, user experience, and non-users’ perception and engagement. Our findings emphasize the importance of adaptive interfaces, improved peripheral awareness, and socially transparent device designs suited for public, real-world use.
@inproceedings{stefanidi2025exploring, author = {Stefanidi, Helen and Sünderkamp, Jan-Hendrik and Itzlinger, Alina and Tatzgern, Markus and Meschtscherjakov, Alexander}, title = {Exploring {AR} {In-the-Wild}: An Autoethnographic Study at a {Christmas} Market}, booktitle = {Companion Publication of the 2025 ACM Designing Interactive Systems Conference}, series = {DIS '25}, pages = {560--566}, year = {2025}, month = jul, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, location = {Funchal Portugal}, doi = {10.1145/3715668.3736368}, url = {https://doi.org/10.1145/3715668.3736368} } -
Journal of Retailing and Consumer Services, 2025
Functional control in webshops allows consumers to virtually experience the different functions of a product. To date, however, the potential benefits of enabling functional control in online product presentations in addition to standard visual control have only been studied and proven for non-immersive virtual environments but not for immersive virtual reality shopping experiences in webshops. We close this research gap by comparing an immersive product presentation format with visual control (grabbing a product and viewing it from different angles) to that of an immersive product presentation format with additional functional control (experiencing and inspecting the different functionalities of the product) by means of a between-subjects experiment. Our special focus is on the influence of functional control on the hedonic aspects of online shopping since interactive immersive webshops are known to offer particular potential to enhance the hedonistic shopping experience of consumers. Our findings suggest that (1) the possibility to try out product functions in an immersive environment can significantly increase the shopping experience of consumers and that (2) hedonic aspects like the enjoyment of consumers actually seem to play a highly important role in immersive webshops. Accordingly, as a practical implication of our study, companies should consider investing in functional control elements when presenting their products in immersive virtual reality webshops as a way to gain a competitive advantage.
@article{kinzinger2025interactive, author = {Kinzinger, Arno and Steiner, Winfried J. and Tatzgern, Markus and Vallaster, Christine}, title = {Interactive product presentation in an immersive environment: The influence of functional control on hedonic aspects}, journal = {Journal of Retailing and Consumer Services}, volume = {83}, pages = {104156}, year = {2025}, month = mar, publisher = {Elsevier}, issn = {0969-6989}, doi = {10.1016/j.jretconser.2024.104156}, url = {https://doi.org/10.1016/j.jretconser.2024.104156} } -
CHI Workshop Bringing Adaptive On-the-Move AR to Outdoor Environments
ACM CHI Conference on Human Factors in Computing Systems (CHI), Workshop on Artificial Intelligence and Human Interfaces, 2025
@inproceedings{stefanidi2025bringing, author = {Stefanidi, Helen and Tatzgern, Markus and Itzlinger, Alina and Meschtscherjakov, Alexander}, title = {Bringing Adaptive {On-the-Move} {AR} to Outdoor Environments}, booktitle = {ACM CHI Conference on Human Factors in Computing Systems (CHI), Workshop on Artificial Intelligence and Human Interfaces}, year = {2025} }
2024
-
IEEE VRW Passive Haptic Feedback for Control Elements in Virtual Reality
IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2024
We designed a controller-based hand input method that allows users to perceive haptic feedback with typical handheld controllers when interacting with virtual control elements while at the same time benefitting from precise controller-based tracking. We performed an experiment comparing our controller-based input method against native HMD hand tracking. Interestingly, hand tracking was robust and precise during interactions with most control elements in our experimental setup. A surprising outcome was that haptic feedback generally slowed down users during interaction, potentially due to a fear of hitting the physical surface during interaction.
@inproceedings{tatzgern2024passive, author = {Tatzgern, Markus and Fasching, David and Gómez, Vivian and Calvache, Sebastián and Figueroa, Pablo}, title = {Passive Haptic Feedback for Control Elements in Virtual Reality}, booktitle = {2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)}, pages = {749--750}, year = {2024}, month = mar, publisher = {IEEE}, location = {Orlando, FL, USA}, doi = {10.1109/vrw62533.2024.00171}, url = {https://doi.org/10.1109/vrw62533.2024.00171} } -
MobileHCI Augmented Reality on the Move: A Systematic Literature Review for Vulnerable Road Users
Proceedings of the ACM on Human-Computer Interaction (PACMHCI), MobileHCI Issue, 2024
Due to the continuous improvement of Augmented Reality (AR) head-mounted displays (HMDs), these devices are bound to be increasingly integrated into our daily routines. So far, a major focus of AR research has been on indoor usage and deployment. However, since seamlessly supporting users in their activities while being on-the-move in various outdoor contexts becomes increasingly important, there is a need to investigate the current state-of-the-art of AR technologies while people are in motion outdoors. Therefore, we conducted a systematic literature review of pertinent HCI publications, specifically looking into applications concerning vulnerable road users. We identify the contexts in which such technologies have been researched, prevailing challenges in the field, and applied methodological approaches. Our findings show that most contributions address pedestrians, a shift towards HMDs, and a prevalence of lab studies due to technology limitations. Based on our findings, we discuss trends, existing gaps and opportunities for future research.
@article{stefanidi2024augmented, author = {Stefanidi, Helen and Tatzgern, Markus and Meschtscherjakov, Alexander}, title = {Augmented Reality on the Move: A Systematic Literature Review for Vulnerable Road Users}, journal = {Proceedings of the ACM on Human-Computer Interaction}, volume = {8}, number = {MHCI}, pages = {1--30}, year = {2024}, month = sep, publisher = {Association for Computing Machinery}, issn = {2573-0142}, doi = {10.1145/3676490}, url = {https://doi.org/10.1145/3676490} } -
ISMAR Immersive Authoring by Demonstration of Industrial Procedures
IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2024
This work presents an authoring tool for supporting the creation of immersive instructions for industrial processes. Our system simplifies the creation of instructional content by providing an immersive virtual reality environment that enables expert operators to interact directly with virtual replicas of industrial devices. Hand movements, tool usage, gaze, spoken comments, and machine part movement are recorded using a head-mounted display. Editing of instructions in virtual reality is aided by automatic segmentation of recorded data into individual steps and visualizations of regions with intensive activity. A qualitative evaluation of our system by industrial experts shows that it is a viable alternative to current practices in authoring instructions for assembly and maintenance.
@inproceedings{skreinig2024immersive, author = {Skreinig, Lucchas Ribeiro and Mohr, Peter and Berger, Blanca and Tatzgern, Markus and Schmalstieg, Dieter and Kalkofen, Denis}, title = {Immersive Authoring by Demonstration of Industrial Procedures}, booktitle = {2024 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)}, pages = {1293--1302}, year = {2024}, month = oct, publisher = {IEEE}, location = {Bellevue, WA, USA}, doi = {10.1109/ismar62088.2024.00146}, url = {https://doi.org/10.1109/ismar62088.2024.00146} } -
🏆 Honorable Mention for Best Student Paper
IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2024
Researching novel user experiences in medicine is challenging due to limited access to equipment and strict ethical protocols. Extended Reality (XR) simulation technologies offer a cost-and time-efficient solution for developing interactive systems. Recent work has shown Extended Reality Prototyping (XRP)’s potential, but its applicability to specific domains like controlling complex machinery needs further exploration. This paper explores the benefits and limitations of XRP in controlling a mobile medical imaging robot. We compare two XR visualization techniques to reduce perceived latency between user input and robot activation. Our XRP validation study demonstrates its potential for comparative studies, but identifies a gap in modeling human behavior in the analytic XRP validation framework.
@inproceedings{plumer2024xr, author = {Plümer, Jan Hendrik and Yu, Kevin and Eck, Ulrich and Kalkofen, Denis and Steininger, Philipp and Navab, Nassir and Tatzgern, Markus}, title = {{XR} Prototyping of Mixed Reality Visualizations: Compensating Interaction Latency for a Medical Imaging Robot}, booktitle = {2024 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)}, pages = {1--10}, year = {2024}, month = oct, publisher = {IEEE}, location = {Bellevue, WA, USA}, doi = {10.1109/ismar62088.2024.00014}, url = {https://doi.org/10.1109/ismar62088.2024.00014} }
2023
-
CHI Eye-Perspective View Management for Optical See-Through Head-Mounted Displays
ACM CHI Conference on Human Factors in Computing Systems (CHI), 2023
DOIPDF (open access)▶ Video▶ Talk
Optical see-through (OST) head-mounted displays (HMDs) enable users to experience Augmented Reality (AR) support in the form of helpful real-world annotations. Unfortunately, the blend of the environment with virtual augmentations due to semitransparent OST displays often deteriorates the contrast and legibility of annotations. View management algorithms adapt the annotations’ layout to improve legibility based on real-world information, typically captured by built-in HMD cameras. However, the camera views are different from the user’s view through the OST display which decreases the final layout quality. We present eye-perspective view management that synthesizes high-fidelity renderings of the user’s view to optimize annotation placement. Our method significantly improves over traditional camera-based view management in terms of annotation placement and legibility. Eye-perspective optimizations open up opportunities for further research on use cases relying on the user’s true view through OST HMDs.
@inproceedings{emsenhuber2023eye, author = {Emsenhuber, Gerlinde and Langlotz, Tobias and Kalkofen, Denis and Sutton, Jonathan and Tatzgern, Markus}, title = {{Eye-Perspective} View Management for Optical {See-Through} {Head-Mounted} Displays}, booktitle = {Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems}, series = {CHI '23}, pages = {1--16}, year = {2023}, month = apr, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, location = {Hamburg Germany}, doi = {10.1145/3544548.3581059}, url = {https://doi.org/10.1145/3544548.3581059} } -
IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2023
DOIPDF (open access)▶ VideoCode
Extended Reality (XR) technology has matured in recent years, leading to increased use of XR simulations for prototyping novel human-centered interfaces, approximating advanced display hardware, or exploring future user experiences, before realising them in real-world scenarios. However, the validity of utilizing XR prototyping (XRP) as a method for gathering performance data on novel user experiences is still underexplored, i.e, it is not clear if results gathered in simulations can be transferred to a real experience. To address this gap, we propose a validation framework that supports establishing equivalence of performance measures gathered with real products and simulated products and, thus, improve ecological validity of XRP. To demonstrate the utility of the framework, we conduct an exemplary validation study using a Varjo XR-3, a state-of-the-art XR head-mounted display (HMD). The study focuses on steering a small drone and comparing it to interactions with its real-world counterpart. We identify functional fidelity, i.e., functional similarity between real and simulated product, as well as simulation overhead from wearing an HMD as major confounding factors for XRP.
@inproceedings{plumer2023framework, author = {Plümer, Jan Hendrik and Tatzgern, Markus}, title = {Towards a Framework for Validating {XR} Prototyping for Performance Evaluations of Simulated User Experiences}, booktitle = {2023 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)}, pages = {810--819}, year = {2023}, month = oct, publisher = {IEEE}, location = {Sydney, Australia}, doi = {10.1109/ismar59233.2023.00096}, url = {https://doi.org/10.1109/ismar59233.2023.00096} }
2022
-
IEEE VRW Towards Eye-Perspective Rendering for Optical See-Through Head-Mounted Displays
IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW), 2022
The optical see-through (OST) head-mounted display (HMD) is a typical platform for Augmented Reality (AR) and allows users to experience virtual augmentations in a wearable form factor. Utilizing information of the real-world background, visualization algorithms adapt the layout and representation of content to improve legibility. Typically, this background information is captured via built-in HMD cameras. However, HMD camera views of the real-world scene are distinctively different to the user's view through the OST display. In this work, we propose eye-perspective rendering (EPR) as a solution to synthesize high fidelity renderings of the user's view for mobile OST HMD to enable adaptation algorithms to utilize visual information as seen from the perspective of the user to improve placement, rendering and, thus, legibility of content.
@inproceedings{emsenhuber2022eye, author = {Emsenhuber, Gerlinde and Domhardt, Michael and Langlotz, Tobias and Kalkofen, Denis and Tatzgern, Markus}, title = {Towards {Eye-Perspective} Rendering for Optical {See-Through} {Head-Mounted} Displays}, booktitle = {2022 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)}, pages = {640--641}, year = {2022}, month = mar, publisher = {IEEE}, location = {Christchurch, New Zealand}, doi = {10.1109/vrw55335.2022.00171}, url = {https://doi.org/10.1109/vrw55335.2022.00171} } -
Information Systems Journal, 2022
Online shops have become increasingly interactive, using different technologies to create virtual experiences that attempt to simulate a realistic product experience. We explore the impact of high sensory enabling (HSE) virtual product presentation modes using state‐of‐the‐art virtual reality (VR) technology that allows consumers to imitate natural movement and interactions via head‐mounted displays (HMD) and dual hand VR controllers. This will compare the HSE virtual product presentation mode with a typical low sensory enabling (LSE) virtual product presentation mode that utilises conventional computer screens, along with mouse and keyboard inputs, on a desktop computer. For the HSE virtual product presentation mode, the results show significantly higher values for the studied variables, including presence, perceived diagnosticity, attitude towards product, and purchase intention. Shopping frequency has a moderating effect on the significant differences of presence between presentation modes. Our research contributes to theory by building on attitude theory, cue summation theory, as well as repetitive learning and memory to explore and explain the effects of HSE virtual product presentation modes on the constructs considered. For managers and industry leaders, this study identifies the importance of using state‐of‐the‐art technology when creating HSE virtual experiences for their products.
@article{kinzinger2022comparing, author = {Kinzinger, Arno and Steiner, Winfried and Tatzgern, Markus and Vallaster, Christine}, title = {Comparing low sensory enabling ({LSE}) and high sensory enabling ({HSE}) virtual product presentation modes in e-commerce}, journal = {Information Systems Journal}, volume = {32}, number = {5}, pages = {1034--1063}, year = {2022}, month = mar, publisher = {Wiley}, issn = {1350-1917}, doi = {10.1111/isj.12382}, url = {https://doi.org/10.1111/isj.12382} } -
🏆 Honorable Mention for Best Paper
ACM CHI Conference on Human Factors in Computing Systems (CHI), 2022
DOIPDF (open access)▶ Video▶ TalkCode
Increased levels of interactivity and multi-sensory stimulation have been shown to enhance the immersion of Virtual Reality experiences. We present the AirRes mask that enables users to utilize their breathing for precise natural interactions with the virtual environment without suffering from limitations of the sensing equipment such as motion artifacts. Furthermore, the AirRes mask provides breathing resistance as novel output modality that can be adjusted in real-time by the application. In a user study, we demonstrate the mask’s precision measurements for interaction as well as its ability to use breathing resistance to communicate contextual information such as adverse environmental conditions that affect the user’s virtual avatar. Our results show that the AirRes mask enhances virtual experiences and has the potential to create more immersive scenarios for applications by enforcing the perception of danger or improving situational awareness in training simulations, or for psychotherapy by providing additional physical stimuli.
@inproceedings{tatzgern2022airres, author = {Tatzgern, Markus and Domhardt, Michael and Wolf, Martin and Cenger, Michael and Emsenhuber, Gerlinde and Dinic, Radomir and Gerner, Nathalie and Hartl, Arnulf}, title = {{AirRes} Mask: A Precise and Robust Virtual Reality Breathing Interface Utilizing Breathing Resistance as Output Modality}, booktitle = {CHI Conference on Human Factors in Computing Systems}, series = {CHI '22}, pages = {1--14}, year = {2022}, month = apr, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, location = {New Orleans LA USA}, doi = {10.1145/3491102.3502090}, url = {https://doi.org/10.1145/3491102.3502090} }
2021
-
EG Tutorial Visualization and Graphics in Mixed Reality
Eurographics 2021 – Tutorials, 2021
This tutorial will present the challenges and unique aspects of mixed reality visualization applications such as organization of data for visualization, real-world data sources for visualization, real time photo-realistic rendering techniques, diminished reality rendering techniques and cognitive and perceptual issues. We are also interested in ways that the existing body of research in the graphics and visualization community can be applied in this research area. We expect the tutorial to be a working event with both presentation of state-of- the-art followed by lively discussion about the key issues and challenges in this research area. Our tutorial is open both for academia and industry, and expected to be a community hub for both areas that are interested in an introduction to the unique challenges of mixed reality visualization. We welcome a diverse audience consisting of students, researchers and developers that have a basic understanding of computer graphics and computer vision.
@inproceedings{kalkofen2021visualization, author = {Kalkofen, Denis and Mori, Shohei and Tatzgern, Markus}, title = {Visualization and Graphics in Mixed Reality}, booktitle = {Eurographics 2021 - Tutorials}, year = {2021}, publisher = {The Eurographics Association}, isbn = {978-3-03868-135-9}, issn = {1017-4656}, doi = {10.2312/egt.20211036}, url = {https://doi.org/10.2312/egt.20211036} } -
IEEE VR Exploring Input Approximations for Control Panels in Virtual Reality
IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2021
Today's availability of sophisticated consumer-grade Virtual Reality (VR) hardware provides affordable access to training simulation technology. In contrast to more traditional high fidelity training simulations utilizing physical replicas of control panels that allow natural interaction, users of virtual training often rely on handheld VR controllers to manipulate simulated controls. Therefore, control manipulations of virtual buttons and sliders must be mapped to approximations of real hand manipulations. For instance, the turning of a physical knob with thumb and index finger can be mapped to a gross motor forearm rotation, or a fine motor joystick manipulation when manipulating a controller. In this paper, we present an exploration of hand input approximations of real hands to manipulate the buttons, toggles, knobs and sliders using typical handheld VR controllers. We use common Oculus Quest controllers that rely on capacitive sensing to create basic, approximate hand gestures. We demonstrate the potential of our designs by performing an experiment in which we compare approximate hand gestures against using the controller's joystick that allows fine motor control using thumb input, and a baseline ray-casting interaction that is commonly used in VR applications. We provide a detailed analysis of our interaction designs using the Framework for Interactive Fidelity Analysis (FIFA), which allows us to discuss differences between input approximations and the real-world hand manipulations.
@inproceedings{tatzgern2021exploring, author = {Tatzgern, Markus and Birgmann, Christoph}, title = {Exploring Input Approximations for Control Panels in Virtual Reality}, booktitle = {2021 IEEE Virtual Reality and 3D User Interfaces (VR)}, pages = {1--9}, year = {2021}, month = mar, publisher = {IEEE}, location = {Lisboa, Portugal}, doi = {10.1109/vr50410.2021.00092}, url = {https://doi.org/10.1109/vr50410.2021.00092} }
2020
-
UIST Video-Annotated Augmented Reality Assembly Tutorials
ACM Symposium on User Interface Software and Technology (UIST), 2020
We present a system for generating and visualizing interactive 3D Augmented Reality tutorials based on 2D video input, which allows viewpoint control at runtime. Inspired by assembly planning, we analyze the input video using a 3D CAD model of the object to determine an assembly graph that encodes blocking relationships between parts. Using an assembly graph enables us to detect assembly steps that are otherwise difficult to extract from the video, and generally improves object detection and tracking by providing prior knowledge about movable parts. To avoid information loss, we combine the 3D animation with relevant parts of the 2D video so that we can show detailed manipulations and tool usage that cannot be easily extracted from the video. To further support user orientation, we visually align the 3D animation with the real-world object by using texture information from the input video. We developed a presentation system that uses commonly available hardware to make our results accessible for home use and demonstrate the effectiveness of our approach by comparing it to traditional video tutorials.
@inproceedings{yamaguchi2020video, author = {Yamaguchi, Masahiro and Mori, Shohei and Mohr, Peter and Tatzgern, Markus and Stanescu, Ana and Saito, Hideo and Kalkofen, Denis}, title = {{Video-Annotated} Augmented Reality Assembly Tutorials}, booktitle = {Proceedings of the 33rd Annual ACM Symposium on User Interface Software and Technology}, series = {UIST '20}, pages = {1010--1022}, year = {2020}, month = oct, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, location = {Virtual Event USA}, doi = {10.1145/3379337.3415819}, url = {https://doi.org/10.1145/3379337.3415819} }
2019
-
CHI TrackCap: Enabling Smartphones for 3D Interaction on Mobile Head-Mounted Displays
ACM CHI Conference on Human Factors in Computing Systems (CHI), 2019
The latest generation of consumer market Head-mounted displays (HMD) now include self-contained inside-out tracking of head motions, which makes them suitable for mobile applications. However, 3D tracking of input devices is either not included at all or requires to keep the device in sight, so that it can be observed from a sensor mounted on the HMD. Both approaches make natural interactions cumbersome in mobile applications. TrackCap, a novel approach for 3D tracking of input devices, turns a conventional smartphone into a precise 6DOF input device for an HMD user. The device can be conveniently operated both inside and outside the HMD's field of view, while it provides additional 2D input and output capabilities.
@inproceedings{mohr2019trackcap, author = {Mohr, Peter and Tatzgern, Markus and Langlotz, Tobias and Lang, Andreas and Schmalstieg, Dieter and Kalkofen, Denis}, title = {{TrackCap}}, booktitle = {Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems}, series = {CHI '19}, pages = {1--11}, year = {2019}, month = may, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, location = {Glasgow Scotland Uk}, doi = {10.1145/3290605.3300815}, url = {https://doi.org/10.1145/3290605.3300815} } -
Curator: The Museum Journal, 2019
This paper introduces the AR Perpetual Garden App and highlights reasons why it is unique and important in its use of immersive, augmented reality for informal learning. The design and construction of the application created a virtual diorama based on facts, using realistic 3D models and accurate perspective. Design affordances permit annotations with interactive and multimodal facts and stories, delivering interactive knowledge artifacts. The app combines real and virtual into one holistic and comprehensive learning experience, transferring scientific knowledge to the novice in an instant. The use of immersive augmented reality produced from accurate information, data visualizations, and bioacoustics creates context‐sensitivity experiences inside the gallery and outside in the gardens. Distribution through the Apple and Google app stores to schools and homes can extend the broader impact of the educational mission of the Carnegie Museum of Natural History and its biological research center, the Powdermill Nature Reserve.
@article{harrington2019augmented, author = {Harrington, Maria C. R. and Tatzgern, Markus and Langer, Tom and Wenzel, John W.}, title = {Augmented Reality Brings the Real World into {Natural} {History} Dioramas with Data Visualizations and Bioacoustics at the {Carnegie} {Museum} of {Natural} {History}}, journal = {Curator: The Museum Journal}, volume = {62}, number = {2}, pages = {177--193}, year = {2019}, month = apr, publisher = {Wiley}, issn = {0011-3069}, doi = {10.1111/cura.12308}, url = {https://doi.org/10.1111/cura.12308} }
2017
-
CHI Retargeting Video Tutorials Showing Tools With Surface Contact to Augmented Reality
ACM CHI Conference on Human Factors in Computing Systems (CHI), 2017
A video tutorial effectively conveys complex motions, but may be hard to follow precisely because of its restriction to a predetermined viewpoint. Augmented reality (AR) tutorials have been demonstrated to be more effective. We bring the advantages of both together by interactively retargeting conventional, two-dimensional videos into three-dimensional AR tutorials. Unlike previous work, we do not simply overlay video, but synthesize 3D-registered motion from the video. Since the information in the resulting AR tutorial is registered to 3D objects, the user can freely change the viewpoint without degrading the experience. This approach applies to many styles of video tutorials. In this work, we concentrate on a class of tutorials which alter the surface of an object.
@inproceedings{mohr2017retargeting, author = {Mohr, Peter and Mandl, David and Tatzgern, Markus and Veas, Eduardo and Schmalstieg, Dieter and Kalkofen, Denis}, title = {Retargeting Video Tutorials Showing Tools With Surface Contact to Augmented Reality}, booktitle = {Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems}, series = {CHI '17}, pages = {6547--6558}, year = {2017}, month = may, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, location = {Denver Colorado USA}, doi = {10.1145/3025453.3025688}, url = {https://doi.org/10.1145/3025453.3025688} } -
🏆 Honorable Mention for Best Paper
3DUI Adaptive User Perspective Rendering for Handheld Augmented Reality
IEEE Symposium on 3D User Interfaces (3DUI), 2017
Handheld Augmented Reality commonly implements some variant of magic lens rendering, which turns only a fraction of the users real environment into AR while the rest of the environment remains unaffected. Since handheld AR devices are commonly equipped with video see-through capabilities, AR magic lens applications often suffer from spatial distortions, because the AR environment is presented from the perspective of the camera of the mobile device. Recent approaches counteract this distortion based on estimations of the users head position, rendering the scene from the user's perspective. To this end, approaches usually apply face-tracking algorithms on the front camera of the mobile device. However, this demands high computational resources and therefore commonly affects the performance of the application beyond the already high computational load of AR applications. In this paper, we present a method to reduce the computational demands for user perspective rendering by applying lightweight optical flow tracking and an estimation of the users motion before head tracking is started. We demonstrate the suitability of our approach for computationally limited mobile devices and we compare it to device perspective rendering, to head tracked user perspective rendering, as well as to fixed point of view user perspective rendering.
@inproceedings{mohr2017adaptive, author = {Mohr, Peter and Tatzgern, Markus and Grubert, Jens and Schmalstieg, Dieter and Kalkofen, Denis}, title = {Adaptive user perspective rendering for Handheld Augmented Reality}, booktitle = {2017 IEEE Symposium on 3D User Interfaces (3DUI)}, pages = {176--181}, year = {2017}, publisher = {IEEE}, location = {Los Angeles, CA, USA}, doi = {10.1109/3dui.2017.7893336}, url = {https://doi.org/10.1109/3dui.2017.7893336} }
2016
-
IEEE VR Adaptive Information Density for Augmented Reality Displays
IEEE Virtual Reality (VR), 2016
Augmented Reality (AR) browsers show geo-referenced data in the current view of a user. When the amount of data grows too large, the display quickly becomes cluttered. Clustering items by spatial and semantic attributes can temporarily alleviate the issue, but is not effective against an increasing amount of data. We present an adaptive information density display for AR that balances the amount of presented information against the potential clutter created by placing items on the screen. We use hierarchical clustering to create a level-of-detail structure, in which nodes closer to the root encompass groups of items, while the leaf nodes contain single items. Our method selects items and groups from different levels of this hierarchy based on user-defined preferences and on the amount of visual clutter caused by placing these items. The number of presented items is adapted during user interaction to avoid clutter. We compare our interface to a conventional AR browser interface in a qualitative user study. Users clearly preferred our interface, because it provided a better overview of the data and allowed for easier comparison. In a second study, we evaluated the effect of different degrees of clustering on search and recall tasks. Users generally made fewer errors, when using our interface for a search task, which indicates that the reduced clutter allowed them to stay focused on finding the relevant items.
@inproceedings{tatzgern2016adaptive, author = {Tatzgern, Markus and Orso, Valeria and Kalkofen, Denis and Jacucci, Giulio and Gamberini, Luciano and Schmalstieg, Dieter}, title = {Adaptive information density for augmented reality displays}, booktitle = {2016 IEEE Virtual Reality (VR)}, pages = {83--92}, year = {2016}, month = mar, publisher = {IEEE}, location = {Greenville, SC, USA}, doi = {10.1109/vr.2016.7504691}, url = {https://doi.org/10.1109/vr.2016.7504691} } -
TVCG Temporal Coherence Strategies for Augmented Reality Labeling
IEEE Transactions on Visualization and Computer Graphics (TVCG), 2016
Temporal coherence of annotations is an important factor in augmented reality user interfaces and for information visualization. In this paper, we empirically evaluate four different techniques for annotation. Based on these findings, we follow up with subjective evaluations in a second experiment. Results show that presenting annotations in object space or image space leads to a significant difference in task performance. Furthermore, there is a significant interaction between rendering space and update frequency of annotations. Participants improve significantly in locating annotations, when annotations are presented in object space, and view management update rate is limited. In a follow-up experiment, participants appear to be more satisfied with limited update rate in comparison to a continuous update rate of the view management system.
@article{madsen2016temporal, author = {Madsen, Jacob Boesen and Tatzgern, Markus and Madsen, Claus B. and Schmalstieg, Dieter and Kalkofen, Denis}, title = {Temporal Coherence Strategies for Augmented Reality Labeling}, journal = {IEEE Transactions on Visualization and Computer Graphics}, volume = {22}, number = {4}, pages = {1415--1423}, year = {2016}, month = apr, publisher = {IEEE}, issn = {1077-2626}, doi = {10.1109/tvcg.2016.2518318}, url = {https://doi.org/10.1109/tvcg.2016.2518318} }
2015
-
Pervasive and Mobile Computing, Special Issue on Mobile and Pervasive Applications in Tourism, 2015
Augmented reality (AR) enables users to retrieve additional information about real world objects and locations. Exploring such location-based information in AR requires physical movement to different viewpoints, which may be tiring and even infeasible when viewpoints are out of reach. In this paper, we present object-centric exploration techniques for handheld AR that allow users to access information freely using a virtual copy metaphor. We focus on the design of techniques that allow the exploration of large real world objects. We evaluated our interfaces in a series of studies in controlled conditions and compared them to a 3D map interface, which is a more common method for accessing location-based information. Based on our findings, we put forward design recommendations that should be considered by future generations of location-based AR browsers, 3D tourist guides or situated urban planning.
@article{tatzgern2015exploring, author = {Tatzgern, Markus and Grasset, Raphael and Veas, Eduardo and Kalkofen, Denis and Seichter, Hartmut and Schmalstieg, Dieter}, title = {Exploring real world points of interest: Design and evaluation of object-centric exploration techniques for augmented reality}, journal = {Pervasive and Mobile Computing}, volume = {18}, pages = {55--70}, year = {2015}, month = apr, publisher = {Elsevier}, issn = {1574-1192}, doi = {10.1016/j.pmcj.2014.08.010}, url = {https://doi.org/10.1016/j.pmcj.2014.08.010} }
2014
-
IEEE VR Transitional Augmented Reality Navigation for Live Captured Scenes
IEEE Virtual Reality (VR), 2014
Augmented Reality (AR) applications require knowledge about the real world environment in which they are used. This knowledge is often gathered while developing the AR application and stored for future uses of the application. Consequently, changes to the real world lead to a mismatch between the previously recorded data and the real world. New capturing techniques based on dense Simultaneous Localization and Mapping (SLAM) not only allow users to capture real world scenes at run-time, but also enables them to capture changes of the world. However, instead of using previously recorded and prepared scenes, users must interact with an unprepared environment. In this paper, we present a set of new interaction techniques that support users in handling captured real world environments. The techniques present virtual viewpoints of the scene based on a scene analysis and provide natural transitions between the AR view and virtual viewpoints. We demonstrate our approach with a SLAM based prototype that allows us to capture a real world scene and describe example applications of our system.
@inproceedings{tatzgern2014transitional, author = {Tatzgern, Markus and Grasset, Raphael and Kalkofen, Denis and Schmalstieg, Dieter}, title = {Transitional Augmented Reality navigation for live captured scenes}, booktitle = {2014 IEEE Virtual Reality (VR)}, pages = {21--26}, year = {2014}, month = mar, publisher = {IEEE}, location = {Minneapolis, MN, USA}, doi = {10.1109/vr.2014.6802045}, url = {https://doi.org/10.1109/vr.2014.6802045} } -
IEEE VR Hedgehog Labeling: View Management Techniques for External Labels in 3D Space
IEEE Virtual Reality (VR), 2014
Annotations of objects in 3D environments are commonly controlled using view management techniques. State-of-the-art view management strategies for external labels operate in 2D image space. This creates problems, because the 2D view of a 3D scene changes over time, and temporal behavior of elements in a 3D scene is not obvious in 2D image space. We propose managing the placement of external labels in 3D object space instead. We use 3D geometric constraints to achieve label placement that fulfills the desired objectives (e.g., avoiding overlapping labels), but also behaves consistently over time as the viewpoint changes. We propose two geometric constraints: a 3D pole constraint, where labels move along a 3D pole sticking out from the annotated object, and a plane constraint, where labels move in a dominant plane in the world. This formulation is compatible with standard optimization approaches for labeling, but overcomes the lack of temporal coherence.
@inproceedings{tatzgern2014hedgehog, author = {Tatzgern, Markus and Kalkofen, Denis and Grasset, Raphael and Schmalstieg, Dieter}, title = {Hedgehog labeling: View management techniques for external labels in {3D} space}, booktitle = {2014 IEEE Virtual Reality (VR)}, pages = {27--32}, year = {2014}, month = mar, publisher = {IEEE}, location = {Minneapolis, MN, USA}, doi = {10.1109/vr.2014.6802046}, url = {https://doi.org/10.1109/vr.2014.6802046} }
2013
-
JVRC Exploring Distant Objects with Augmented Reality
Joint Virtual Reality Conference of EGVE and EuroVR (JVRC), 2013
Augmented reality (AR) enables users to retrieve additional information about the real world objects and locations. Exploring such location-based information in AR requires physical movement to different viewpoints, which may be tiring and even infeasible when viewpoints are out of reach. In this paper, we present object-centric exploration techniques for handheld AR that allow users to access information freely using a virtual copy metaphor to explore large real world objects. We evaluated our interfaces in controlled conditions and collected first experiences in a real world pilot study. Based on our findings, we put forward design recommendations that should be considered by future generations of location-based AR browsers, 3D tourist guides, or in situated urban planning.
@inproceedings{tatzgern2013exploring, author = {Tatzgern, Markus and Grasset, Raphael and Veas, Eduardo and Kalkofen, Denis and Seichter, Hartmut and Schmalstieg, Dieter}, title = {Exploring Distant Objects with Augmented Reality}, booktitle = {Joint Virtual Reality Conference of EGVE - EuroVR}, pages = {49--56}, year = {2013}, publisher = {The Eurographics Association}, isbn = {978-3-905674-47-7}, issn = {1727-530X}, doi = {10.2312/EGVE.JVRC13.049-056}, url = {https://doi.org/10.2312/EGVE.JVRC13.049-056} } -
IEEE VR Dynamic Compact Visualizations for Augmented Reality
IEEE Virtual Reality (VR), 2013
In Augmented Reality (AR), careless augmentations can easily lead to information overflow. Especially on small screen devices, only a limited amount of information can be displayed comprehensively. Compact visualization filters data by reducing redundancies and creating a layout of the remaining information. Previously, this approach was applied to create static compact explosion diagrams. In this paper, we extend the approach to annotations, which are a major source of information in AR, and create compact layouts of annotations and annotated explosion diagrams. We present methods to transfer compact visualizations to dynamic AR settings and achieve interactive frame rates even on limited-resource hardware, such as mobile phones. Moreover, we create temporally coherent and scene-aware layouts.
@inproceedings{tatzgern2013dynamic, author = {Tatzgern, Markus and Kalkofen, Denis and Schmalstieg, Dieter}, title = {Dynamic compact visualizations for augmented reality}, booktitle = {2013 IEEE Virtual Reality (VR)}, pages = {3--6}, year = {2013}, month = mar, publisher = {IEEE}, location = {Lake Buena Vista, FL}, doi = {10.1109/vr.2013.6549347}, url = {https://doi.org/10.1109/vr.2013.6549347} }
2012
-
ISMAR Image-Driven View Management for Augmented Reality Browsers
IEEE International Symposium on Mixed and Augmented Reality (ISMAR), 2012
In this paper, we introduce a novel view management technique for placing labels in Augmented Reality systems. A common issue in many Augmented Reality applications is the absence of knowledge of the real environment, limiting the efficient representation and optimal layout of the digital information augmented onto the real world. To overcome this problem, we introduce an image-based approach, which combines a visual saliency algorithm with edge analysis to identify potentially important image regions and geometric constraints for placing labels. Our proposed solution also includes adaptive rendering techniques that allow a designer to control the appearance of depth cues. We describe the results obtained from a user study considering different scenarios, which we performed for validating our approach. Our technique will provide special benefits to Augmented Reality browsers that usually lack scene knowledge, but also to many other applications in the domain of Augmented Reality such as cultural heritage and maintenance applications.
@inproceedings{grasset2012image, author = {Grasset, Raphaël and Langlotz, Tobias and Kalkofen, Denis and Tatzgern, Markus and Schmalstieg, Dieter}, title = {Image-driven view management for augmented reality browsers}, booktitle = {2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR)}, pages = {177--186}, year = {2012}, month = nov, publisher = {IEEE}, location = {Atlanta, GA, USA}, doi = {10.1109/ismar.2012.6402555}, url = {https://doi.org/10.1109/ismar.2012.6402555} }
2011
-
C&G Multi-Perspective Compact Explosion Diagrams
Computers & Graphics, 2011
This article presents a system to automatically generate compact explosion diagrams. Inspired by handmade illustrations, our approach reduces the complexity of an explosion diagram by rendering an exploded view only for a suitable subset of the assemblies of an object. However, the exploded views are chosen so that they allow inferring the remaining unexploded assemblies of the entire 3D model. In particular, our approach demonstrates the assembly of a set of identical groups of parts by presenting an exploded view only for a single representative. In order to identify the representatives, our system automatically searches for recurring subassemblies. It selects representatives depending on a quality evaluation of their potential exploded view. Our system takes into account visibility information of both the exploded view of a potential representative as well as visibility information of the remaining unexploded assemblies. This allows rendering a balanced compact explosion diagram, consisting of a clear presentation of the exploded representatives as well as the unexploded remaining assemblies. Since representatives may interfere with one another, our system furthermore optimizes combinations of representatives. The optimization process also generates good views on the explosion diagram. Labels are added to the explosion diagram to increase the visibility of small or occluded parts. Throughout this article, we show a number of examples, which all have been rendered from unmodified 3D CAD data.
@article{tatzgern2011multi, author = {Tatzgern, Markus and Kalkofen, Denis and Schmalstieg, Dieter}, title = {Multi-perspective compact explosion diagrams}, journal = {Computers \& Graphics}, volume = {35}, number = {1}, pages = {135--147}, year = {2011}, month = feb, publisher = {Elsevier}, issn = {0097-8493}, doi = {10.1016/j.cag.2010.11.005}, url = {https://doi.org/10.1016/j.cag.2010.11.005} } -
ISMAR Workshop Embedded Virtual Views for Augmented Reality Navigation
IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Workshop on Visualization in Mixed Reality Environments, 2011
In this paper, we present virtual embedded views used for turn-based pedestrian navigation in Augmented Reality (AR). Embedded views allow users to see around occluding structures and at the same time seamlessly integrate the augmented navigation aid into the otherwise occluded view. Users get a preview on upcoming route changes, without the need to consult an additional map view. We compare embedded views to other methods revealing the occluded navigation aids. We demonstrate that the technique is more screen-space efficient when compared to a typical x-ray vision technique, and may better facilitate the mental linking of information, when compared to a mirror.
@inproceedings{tatzgern2011embedded, author = {Tatzgern, Markus and Kalkofen, Denis and Grasset, Raphael and Schmalstieg, Dieter}, title = {Embedded Virtual Views for Augmented Reality Navigation}, booktitle = {IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Workshop on Visualization in Mixed Reality Environments}, year = {2011}, location = {Basel, Switzerland}, url = {https://doi.org/10.13140/2.1.4398.1769} }
2010
-
NPAR Compact Explosion Diagrams
International Symposium on Non-Photorealistic Animation and Rendering (NPAR), 2010
This paper presents a system to automatically generate compact explosion diagrams. Inspired by handmade illustrations, our approach reduces the complexity of an explosion diagram by rendering an exploded view only for a subset of the assemblies of an object. However, the exploded views are chosen so that they allow inference of the remaining unexploded assemblies of the entire 3D model. In particular, our approach demonstrates the assembly of a set of identical groups of parts, by presenting an exploded view only for a single representative. In order to identify the representatives, our system automatically searches for recurring subassemblies. It selects representatives depending on a quality evaluation of their potential exploded view. Our system takes into account visibility information of both the exploded view of a potential representative as well as visibility information of the remaining unexploded assemblies. This allows rendering a balanced compact explosion diagram, consisting of a clear presentation of the exploded representatives as well as the unexploded remaining assemblies. Since representatives may interfere with one another, our system furthermore optimizes combinations of representatives. Throughout this paper we show a number of examples, which have all been rendered from unmodified 3D CAD models.
@inproceedings{tatzgern2010compact, author = {Tatzgern, Markus and Kalkofen, Denis and Schmalstieg, Dieter}, title = {Compact explosion diagrams}, booktitle = {Proceedings of the 8th International Symposium on Non-Photorealistic Animation and Rendering - NPAR 10}, series = {NPAR '10}, pages = {17--26}, year = {2010}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, location = {Annecy, France}, doi = {10.1145/1809939.1809942}, url = {https://doi.org/10.1145/1809939.1809942} }
2009
-
IEEE VR Explosion Diagrams in Augmented Reality
IEEE Virtual Reality (VR), 2009
This article introduces explosion diagrams to augmented reality (AR) applications. It presents algorithms to seamlessly integrate an object's explosion diagram into a real world environment, including the AR rendering of relocated objects textured with live video and the restoration of visual information which are hidden behind relocated objects. It demonstrates several types of visualizations for convincing AR explosion diagrams and it discusses visualizations of exploded parts as well as visual links conveying their relocation direction. Furthermore, we show the integration of our rendering and visualization techniques in an AR framework, which is able to automatically compute a diagram's layout and an animation of its corresponding explosion.
@inproceedings{kalkofen2009explosion, author = {Kalkofen, D. and Tatzgern, M. and Schmalstieg, D.}, title = {Explosion Diagrams in Augmented Reality}, booktitle = {2009 IEEE Virtual Reality Conference}, pages = {71--78}, year = {2009}, month = mar, publisher = {IEEE}, location = {Lafayette, LA}, doi = {10.1109/vr.2009.4811001}, url = {https://doi.org/10.1109/vr.2009.4811001} }
Theses
- Situated Visualization in Augmented Reality. PhD thesis, Graz University of Technology, 2015. PDF
- A Framework for Automatically Creating 3D Explosion Diagrams. Master's thesis, Graz University of Technology, 2008.
Patents
- Markus Tatzgern, Denis Kalkofen, Dieter Schmalstieg, Raphael Grasset. Hierarchical Clustering for View Management Augmented Reality. WO Patent 2015142576, September 2015.
- Raphael Grasset, Markus Tatzgern, Tobias Langlotz, Denis Kalkofen, Dieter Schmalstieg. Image-Driven View Management for Annotations. WO Patent 2013176830, November 2013.
- Markus Tatzgern, Denis Kalkofen, Dieter Schmalstieg. Efficient Information Presentation for Augmented Reality. WO Patent 2012033768, March 2012.
- Markus Tatzgern, Denis Kalkofen, Dieter Schmalstieg. Efficient Information Presentation for Augmented Reality. US Patent 20120075433, March 2012.