Name of the participant: Tristan Wirth
Description of the IT research project: Methods of augmented reality display virtual objects in the context of real-world environments. Therefore, they yield the potential for a multitude of relevant applications like the virtual try-on of glasses, clothes, and accessories or the planning of object placement in interior design, factory building, or installation space in cramped machinery.
Recent neural methods, i.e., Neural Radiance Fields, have demonstrated an impressive level of realism in the rendering of objects that were captured in unconstrained environments, e.g., mobile devices. However, these techniques are expensive concerning computation time and memory usage, especially compared to classical methods.
The IBER-Struct project aims at the research and development of processes that combine classical and NeRF-based techniques to make use of their respective benefits. As a result, end-users are going to benefit from resource-efficient, robust and realistic simulations of objects in different environments.
To validate the theoretical contributions, a prototype for the virtual try-on of glass frames is going to be implemented. For this purpose, a NeRF-based model of a human head is created based on a short video captured by mobile devices or webcams. The appearance of different glass frames placed on that head model is supposed to be simulated in a highly realistic manner. This includes the synthesis of novel view angles that have not been captured in the underlying video data. The proposed combination of classical and NeRF-based methods is supposed to mitigate common issues in the virtual try-on of glass frames, e.g., clipping on the nose and ear areas and unrealistic interactions of the glass frames with hairs.
Software Campus partners: TU Darmstadt, ZEISS
Implementation period: 01.03.2022– 01.08.2023
