Computational Photography

Ivo Ihrke

Computational Photography

Computational photography

Our group investigates innovative algorithms in order to correct common weak points in ordinary cameras and develop new applications, such as 3D reconstruction.

High-speed capture of human actors

One example of limited camera capabilities is its limited frame rate, or temporal resolution, in video mode. It is, therefore, diffi cult to record very fast motions that are, e.g., performed by human actors in action scenes. The resulting imagery is motion-blurred a well-known effect that is deliberately utilized to illustrate fast motions, e.g., in comic books. In modern visual effects, such motion blur can be undesirable, for example, if slow motion shots are to be shown. Moreover, modern techniques allow for the capture of virtual humans, which enables a virtual change of viewpoint after the scene has been shot. Motion blur can be a major handicap to these techniques. In cooperation with the Graphics, Vision and Video group, we have developed a motion capture system that can counteract motion blur and, therefore, facilitate the markerless motion capture of highspeed human motions. The developed system employs standard camera hardware in a surround imaging setup. As an example, we have demonstrated capturing at 400 frames per second with a set of 20 cameras that can physically record only at 20 frames per second. Future systems exploiting our ideas could enable the inexpensive surround capture of high-speed motions.

A view inside the measurement device. The object to be scanned is a bust of Max Planck (partial view).

Theoretical analysis of multiplex camera systems

Cameras are often used in multiplex mode, i.e., various forms of information, such as color or directional information, are digitized by suitable optical fi lters in different pixels of a single sensor. This process allows for less expensive production in comparison to the use of separate sensor chips for each information channel such as the colors red, green, and blue, for example. These fi lters are known as color fi lter arrays (CFAs). Recent analysis shows that, in addition to color information, the directional distribution of light, the so-called light fi eld, can be recorded in different channels. The recording of light fi elds makes it possible, among other things, to change the focus of an image after the recording. Our group has established that many new types of cameras are special variations of one single overarching model. This knowledge can be used to create improved reconstruction algorithms, e.g., for color and directional image reproduction and for the analysis of camera performance. The application of the new model enables a comparison of different camera prototypes in terms of noise and resolution.

Device for the surround scanning of an object‘s geometry and reflectance. A large mirror system multiplies the number of available view points. A three colored laser is used to illuminate the object and simultaneously scan its geometry.

A surround imaging and projection system for refl ectance measurements

he three-dimensional digitization of objects is another important application area for new camera technologies. Digitization, also called 3D scanning, is used more and more commonly in measurement, quality control, in the digital preservation of cultural artifacts, but also in the entertainment industry. A fundamental problem with 3D scanning systems is the inability of conventional cameras to have an all-around view; an object can only be viewed from one perspective at a time. In previous work we have shown how kaleidoscopic mirror systems can be employed to circumvent this restriction. For capturing a life-like appearance of real objects, however, it is not only necessary to image the object from all possible view points, but also under all possible illumination conditions. Established technology requires a tremendous hardware effort: Typically, 200 cameras and 200 light sources are employed in such devices. Extending our idea that a single sensor can image hundreds of views by repeated mirroring of the viewing rays, we have developed a prototype system that can generate 200 viewing and lighting directions with a single projector and a single camera. This capability, in turn, enables the acquisition of an object’s reflectance in conjunction with its geometry, using a completely static setup. Our prototype does not yet match the accuracy of commercial scanners, but it provides an exciting outlook at the potential capabilities of future scanning devices.