Name of the participant: Julius Kühn
Description of the IT research project: With the aid of Vision Transformers, the traffic of a Smart City shall be optimized. Actual implemented solutions by the project partner Volkswagen are not yet capable of detecting road users within the Smart City Carmel via several cameras.
A concrete example that shows two main challenges is that a road user who gets out of the camera’s visual field and appears in the visual field of another camera does not get identified as the same road user. The system is missing out on the semantic understanding within many cross-functional camera perspectives.
In this project, the Segment Anything Model (SAM) is used, which is provided by Meta AI. SAM implements a Vision Transformer as its backbone and therefore allows an easy operation due to multi-modal input. Moreover, it serves for semantic segmentation as well as object tracking within a visual field. By doing so, with the camera’s location, it is possible to detect in which direction the object disappears from the visual sight of a camera and reappears at the visual sight of another camera. With this information, the traffic flow at Smart City Carmel will be optimized by improving the traffic streams and preventing accidents and traffic jams. So that the traffic can be more sustainable in the future.
Software Campus partners: Fraunhofer-Verbund IUK-Technologie, Volkswagen
Implementation period: