The vision has already been played out in many sci-fi series and novels: Instead of simple video calls, people talk to each other as digital avatars as if they were sitting in the same room, although, in reality, there are thousands of kilometers between them. For this kind of "telepresence" to be possible, the users need to have photorealistic digital images, which can be generated in real time with little computing effort. Researchers at the Max Planck Institute for Informatics in Saarbrücken are working on the basic research behind such applications. As part of these efforts, they are building a cutting-edge computer science laboratory that will house some of the world's leading technical equipment.
 
The focus of the research laboratory named "Real Virtual Lab" is the high-precision recording of data for basic research in the fields of computer graphics, computer vision, and artificial intelligence. In its completed form, it will comprise a total of 350 square meters of laboratory space. That includes plans to build additional specialized facilities at the Max Planck Institute for Informatics on the Saarland Informatics Campus.

The researchers will present the already completed rooms and measurement setups of the Real Virtual Lab on the occasion of the launch of a so-called "Lightstage". The Lightstage is a device used to generate and record various lighting conditions and light scenes in a controlled manner. The Saarbrücken Lightstage has a diameter of 3.5 meters, includes around 13,000 LEDs, all of which can be controlled individually, and also boasts 40 high-resolution 6k cameras in cinema quality. The Max Planck researchers will use the technology to investigate fundamental problems in the reconstruction and simulation of photorealistic digital characters and environments.
 
"The lighting of a scene has a strong influence on how realistic it looks to us," explains Marc Habermann, who holds a PhD in computer science and runs the Real Virtual Lab. "The Lightstage gives us absolute control over how we can light and record people and scenes. This, in turn, helps us to understand the physical details of a scene's lighting as thoroughly as possible so that we can reproduce it realistically in digital space. By working with the Lightstage, we can understand how different surfaces and objects behave in different lighting situations and how people, objects, and scenes can be synthesized as realistically as possible in new previously unrecorded perspectives and lightings. We are also particularly interested in finding out the minimum information we need to be able to carry out calculations on perspectives and lighting," says Marc Habermann.

The aspect of computing efficiency is of particular interest to the Saarbrücken researchers. Professor Christian Theobalt, Director at the Max Planck Institute for Informatics and Head of the "Visual Computing and Artificial Intelligence" department, where the Real Virtual Lab is located, says: "Think of applications such as the Metaverse, where people will be able to meet in virtual rooms with digital avatars in the future. This could have great potential for environmental and climate protection if, for example, conferences could be held in digital space thus avoiding air travel." The Saarbrücken computer science professor adds: "However, for such technology to be accepted and used to this extent at all, it must be so advanced that there is little or no noticeable difference to a real meeting. It would also have to be usable with reasonable computing effort and ideally with a simple webcam. We are still a long way from achieving this.” With the Lightstage, the computer scientists from Saarbrücken can build and expand on existing research in this area.

Not only telepresence, but also other applications can benefit from the Lightstage: "The more realistically we can simulate environments, the better training data we can generate, for example, for autonomous cars or other image recognition algorithms in an industrial context," says lab manager, Marc Habermann.

The Saarbrücken Lightstage is one of the world's leading research facilities of its kind. The setup is unique in Europe, and there are only a few scientific facilities in the world with similar technical possibilities. Other research groups at the Saarland Informatics Campus will also benefit from the new equipment: researchers from the "Saarbrücken Research Center for Visual Computing, Interaction and Artificial Intelligence (VIA)", which emerged from a strategic research partnership with Google in 2022 and is also based at the Max Planck Institute for Informatics under the direction of Prof. Christian Theobalt, will also work with the new system.

 
Questions can be directed at:
 
Dr. Marc Habermann
Head “Real Virtual Lab” / Research Group Leader
Max Planck Institute for Informatics
Tel.: +49 681 9325 4507
E-Mail:
mhaberma@mpi-inf.mpg.de
 
Prof. Dr. Christian Theobalt
Scientific Director
Max Planck Institute for Informatics
 Tel.: +49 681 9325 4500
E-Mail:
d6-sek@mpi-inf.mpg.de
 

Editor:
Philipp Zapf-Schramm
Saarland Informatics Campus
Phone: +49 681 302-70741
E-Mail: pzapf@cs.uni-saarland.de