Thesis subject

MSc thesis topic: Augmented reality display on windshields: Improving the spatial awareness of autonomous car drivers

The craving for self-driving vehicles keeps growing in society. Along with major technical breakthroughs, researchers have raised concerns about the adaptation to self-driving vehicles, and more importantly, the psychological and behavioral challenges a driver will face. As evidenced in studies of spatial learning involving automotive navigation systems, drivers develop very weak spatial knowledge and degraded competence in spatial decision-making. These negative impacts will only intensify when drivers no longer need to actively steer in self-driving vehicles.

This project aims to augment the spatial competence of future drivers in self-driving vehicles by investigating the innovative use of the augmented reality (AR) display on windshields and their roles in spatial awareness. Building on a realistic
simulation of operating a self-driving car in different environments provided by us, the student will create a virtual reality (VR) simulation of the AR display showing distant landmarks along with directional cues to evaluate the effectiveness of this innovative display in spatial learning in urban and highway environments, respectively. In addition to task performance, eye-tracking data will be integrated with other measures such as head movements and self-report questionnaires, to examine drivers’ cognitive processing of information displayed on the simulated AR windshield.

Immersive technologies such as augmented and virtual reality are entering a new era. Thanks to substantial investments by leading companies (e.g., Meta(™), Microsoft(™), and Google(™)) catapulting immersive media into the mainstream, devices have become affordable and seldom cause cybersickness anymore. At the same time, the creation and design of immersive experiences are getting easier and easier and more accessible to many more people. Tools such as Unity(™) or Unreal(™), that is, game engines, are sophisticated and well-supported development environments. Together with the fast generation of 3D models of geographic environments, we are now in a position to simulate future scenarios and investigate spatial-relevant issues in a context-oriented and experience-driven way.

With big tech and car manufacturing companies investing heavily in automated driving, there is now increasing interest to exploit the windshield’s potential to serve as a medium for displaying information. Therefore, evaluating different design choices and how they impact different aspects of human cognition has become an important challenge. In this regard, immersive technologies provide a high-fidelity and scalable testbed for promoting the future acceptance, trust, and use of self-driving cars. This project will be conducted in collaboration with a domain expert in navigation assistance, and throughout the lifecycle of their thesis research, the student will receive sufficient information and guidance on the existing literature, VR development, experimental design, and user evaluation.

Relevance to research/projects at GRS or other groups

The project is supported by a university-wide research effort (WANDER) seeking to establish immersive technologies firmly in the portfolio of research and education tools for environmental and societal challenges.

Objectives

Replicate a previous video-based online study (under review) but using VR to realize different self-driving scenarios.
Investigate the effects of AR display on spatial learning of autonomous car drivers in different built environments.
Analyze and visualize gaze-behavior patterns to infer the cognitive processing of visual-spatial information displayed on the windshield.

Literature

Riegler, A., Wintersberger, P., Riener, A., & Holzmann, C. (2019). Augmented Reality Windshield Displays and Their Potential to Enhance User Experience in Automated Driving. In i-com (Vol. 18, Issue 2, pp. 127–149).
Li, R. (2019). Spatial Learning in Smart Applications: Enhancing Spatial Awareness through Visualized Off-Screen Landmarks on Mobile Devices. In Annals of the American Association of Geographers (Vol. 110, Issue 2, pp. 421–433).
Zhao, J., Sensibaugh, T., Bodenheimer, B., McNamara, T. P., Nazareth, A., Newcombe, N., Minear, M., & Klippel, A. (2020). Desktop versus immersive virtual environments: Effects on spatial learning. Spatial Cognition & Computation, 3(3), 1–36.
Kiefer, P., Giannopoulos, I., Raubal, M., & Duchowski, A. (2017). Eye tracking for spatial research: Cognition, computation, challenges. Spatial Cognition & Computation, 17(1-2), 1–19.

Requirements

Interest in working with immersive technologies
Interest in human behaviors
Interest in empirical studies and evaluations
Experience with Python and spatial data analysis

Theme(s): Modelling & visualisation; Human–space interaction