Name of the participant: Moritz Ruge
Description of the IT research project: The Internet of Things (IoT) and Industry 4.0 continue to drive digitisation and thus the amount of data generated continues to grow exponentially. This trend is also known as Big Data and requires new data processing paradigms to handle the volume of data.
Current state-of-the-art stream processing engines lose much of their performance because they abstract from the hardware when scaling and treat all nodes the same. New stream processing engines thus need to take into account the underlying hardware in order to maximise the performance of each node to handle the growing volumes of data.
In addition, Moore’s Law has been shown to be stagnant, so new hardware solutions are needed, with experts agreeing that the future lies in the use of heterogeneous hardware. By using heterogeneous hardware, the advantages of the different architectures can be used to hide their respective weaknesses.
In this project, we are implementing a set of operators for a prototype stream processing engine. To optimise performance, we work on heterogeneous hardware. To take advantage of the different hardware architectures, a universal software approach is developed for each operator and implemented across architectures. The implementation is followed by an analytical phase in which the operators are tested in different scenarios to find the parameterisation that maximises performance. By adapting the software approach to the hardware, we hope to achieve results that surpass the current state of the art.
Software Campus partners: TU Berlin, Huawei
Implementation period: 01.04.2021 – 31.03.202 04.2021 – 31.03.2022