Sven Bliedung von der Heide: The future of video capturing

Interview with Sven Bliedung von der Heide, CEO of Volucap GmbH

In June 2018, the first volumetric video studio on the European continent was officially opened in Potsdam-Babelsberg. The technology behind it comes from the Fraunhofer Heinrich-Hertz-Institut, HHI, the Volucap GmbH is running the studio. Sven Bliedung von der Heide, CEO of Volucap GmbH, gives some insights about the innovative technology, in which fields it is used and the history of the exciting project.

With the volumetric studio, an incredible coalition of technology providers, researchers, media companies and funders have come together to advance the dream of digital humans.

We created Germany’s first Volumetric Studio with the highest resolution worldwide, a unique light system and provide custom volumetric capture solutions. We enable a very simple and absolutely authentic three-dimensional recording of people in motion.

Our methods do not require markers, green screens or motion capture devices, so there are no performance limitations.

Film productions become more digital and movies leave the screen by means of Mixed Reality. They take place in 3D and spatially walkable within your own four walls.

If we look at this development, we can see that our old wall mounted linear 2D screens have long since been overtaken by the new interactive possibilities.

While many still refer to the cell phone and tablet as the second screen, these have long since become the first screen and are consumed many times over. So it’s not surprising that interactive content has caught up with the sales of big Hollywood blockbusters, and consumption and status symbols are no longer based on real objects.

And now, for the first time, we have the opportunity to really liberate the content.

For about two years, there were the first VR headsets suitable for the masses and this year, there will be the first mixed reality headsets for consumers.

This makes it possible to experience content in full three dimensions for the first time.

The creation and consumption of media are in a revolution and the most important factor is and remains the human being!

The fact that 3D worlds have not yet enabled users to meet real people, but only animated avatars, is a critical barrier to the success of future media production.

Volucap is thus an essential component in creating a future-proof ecosystem for immersive productions.

The basis for this technology was developed at HHI over more than 10 years. Without various financings of the base technology, it would not have been possible to find a regional solution that we could initially use for Volucap and with which HHI could collect real production requirements for its further development.

We got a GRW funding of two million Euro by the state and federal sources, one million each. And right now, we are working on a mobile solution which is funded via ProFIT. The financing of Volucap was significantly supported by the WFBB and ILB. Thus, we are the first company in the region that was funded as an innovation cluster.

The funds were provided by the federal and state governments, and the shareholders ARRI, Interlake, Studio Babelsberg, the Fraunhofer Institute and I also contributed a share. Thus, with an internationally relatively small initial investment of 3 million euros, a lighthouse could be established, which already now regularly generates international headlines.

The volumetric studio that we build can be seen as a new camera in which you step in and get recorded. Therefore, our technology can be used in almost every project where you could use a camera and which you like to transform into the AR or VR world – such as documentaries with a virtual host, training sessions for upcoming engineers, doctors or pilots with a holographic mentor or lessons with a virtual teacher.

Through our collaboration with leading international production studios, the MediaTech Hub, local associations and cooperation partners, Volucap has made a decisive contribution to the development of the regional XR Ecosystem.

This allows us to bundle all services from regional partners and thus to implement larger international projects together.

Thanks to our quality, research companies can better train artificial intelligences, which in the future will enable more accurate recognition of people on the road, medical diagnosis and forensic analysis to fight crime.

Hollywood now also benefits from our capabilities where new camera angles can be determined after the shooting or absolutely ground-breaking new visual effects can be shown with real 3D people. Even after the shoot, we can literally look behind people, which opens up completely new areas of application. The application ranges from the cinema screen to virtual productions, where our volucaps on LED video walls populate a restaurant or street as background actors directly and live on set without additional extras.

Similar to traditional film, volumetric can be filmed at different resolutions. The resolution in turn determines how far you can zoom in or approach the material or volucap (volumetric capture).

There are already possibilities to produce very low-resolution volumetric recordings at home, which can at least be used for creative applications due to the block-like coarse resolution.

However, if people are to be used volumetrically for cinema or in mixed reality, a much higher quality is required.

Thanks to our partners like ARRI and Studio Babelsberg, we are the first volumetric studio worldwide with a focus on cinema productions.

We film in our main stage with an RGB resolution of over 700 megapixels per frame and are the world leader by a large margin. We also build custom and mobile systems with which we have worked on major Hollywood productions with over 2000 megapixels. We are the first studio that has done projects for major feature film production companies for the big screen and not just VR/XR.

At the same time, we have built up our own production line, with which we can manufacture our own rig constructions and electronic components.

Our systems are constantly being developed and the latest cameras, light and AI-based processes are used to further expand our lead and push the boundaries of technical possibilities.

In and around Babelsberg Volucap was able to build on existing infrastructures for media productions as well as utilizing the knowledge of highly creative companies in the region to create an ecosystem that can support the complex workflows of our technologies. With this in mind and the great support from the local government Babelsberg is the right place to carry out this lighthouse project.

Until now, you could only encounter very stylized avatars in these worlds and even these avatars were associated with very high costs.

We bring real people into the 3D world and give creatives a very easy entry.

Stories can be told lighter, bigger, and more immersive than ever before, while still being able to work as if you were using a camera.

Old inflexible, expensive and very unrealistic methods like motion capture become obsolete.

Most Gamedev studios are still working with older pipelines, but those areas are changing as well as quality and demand increase.

Volucap not only looks photorealistic, but also moves absolutely naturally. Animated figures look more like characters from Disney movies.

Production with volucap is much easier, faster and cheaper.

For example, if I want to put a 5min speech of mine on your phone in Augmented Reality and do it like most video game companies.

Then you need a team of modelers, texture artists, animators, motion capture studio, riggers, sound designers etc. and several weeks of time. Even then I will only look like an animated cartoon character. With Volucap I can record directly in real time in the studio and see the result the next day.

People who have not yet studied the subject in depth often argue that game characters can be re-animated more easily. This is a widespread fallacy that only applies to a few very simple scenarios. All animations for AAA games are recorded in advance, just as with volumetry, and the respective pre-recorded clips are then started depending on the player’s decision.

As soon as a new clip is needed, it must be produced again. In the case of a AAA video game, this would mean motion capture, simulation and baking again.

That’s why such computer game characters are not really dynamic and you can easily test it. If you hit the fingers of a guitar player in Cyberpunk, he continues to play the same way, because they are pre-recorded sequences.

Saying this, I am sure that the Volucap is the state-of-the-art studio for all moving image media and we kindly invite the games studios from our region to visit us at the gates of Berlin to see what would be possible for them if they would implement the Volucap technology in their games.