Mixed Reality (MR) can potentially provide more intuitive and immersive experiences, particularly for navigational applications. The benefit of MR in navigation is that the user does not have to take his/her eyes of a traffic scene in order to get navigating information on the route. However, at the same time there can be many highly salient objects in the field of view that distract the user from augmented information and traffic behaviour. These can be solely background information, such as visually striking objects on building facades or on vegetation.

There are cartographic design techniques to visually highlight important objects and visually downgrade background information. Though, first, semantic knowledge of a scene has to be collected in order to apply reasonable image object design. Then, background information can be visually ‘muffled’ and augmented navigational objects can be placed in appropriate positions.

The candidate must record a test scene (i.e. pedestrian video) and pre-process the video be using a state-of-the-art image segmentation algorithm. The Pyramid Scene Parsing Network (PSPNet) can be used for this task. This machine learning algorithm will have to be calibrated for a reasonable result. In the next step the user has to apply image object design techniques to the background information and determine appropriate positions in the images for the route visualization.
Even though a future application will have to work in real-time, this thesis can be done by post-processing analysis and implementation. Basic Python programming skills are required. A user test is not required.

Partner University:
TUM

Staff working on this field:
Christian Murphy