Volumetric video in augmented reality

and how it will define the way we watch live events

Hello and welcome to the first Condense Reality newsletter.  This will be a monthly newsletter about Condense Reality’s progress, augmented reality and volumetric video.

We believe that augmented reality (AR) technology is going to completely change the way we interact with computers; and that the magnitude of this change could be even bigger than the smartphone revolution.

This prediction isn’t hard to justify. Established companies like Microsoft and Facebook are investing an enormous amount of money and effort into developing high quality AR consumption tech, and startups such as Magic Leap have raised gigantic sums of cash to create this technology (Magic Leap have raised  $2.1bn to date)

Similar to the way high quality content and applications facilitated the success of smartphones, high quality content and applications will act as a catalyst for the growth of AR technology and platforms.

A type of content known as volumetric video is so well suited to AR that it may as well been designed for it. At Condense Reality we are focused on building the technology which will allow content creators to broadcast live events using volumetric video. 

What is volumetric video?

Volumetric content can be viewed from any angle the viewer chooses. This makes it extremely well suited for augmented reality. Imagine being able to watch a 3D reconstruction of a sports event such as a boxing match, as if it has been miniaturised and placed on your living room coffee table.

Now imagine you are watching with friends, they would each have a different perspective depending on where they were sat. A conventional broadcast could still be viewed on a real or virtual TV and scores or 3D widgets can be placed around the volumetric content in 3D space.

Volumetric video is also accessible on many other devices outside of augmented reality. The ability to rotate the content and view it from any angle works almost as well on most smart devices (TV’s, smartphones, PC’s) and importantly in virtual reality.

First VR, then AR

In the future, volumetric video will be the standard way to watch live events. When combined with an AR headset, the experience is incredible and conventional viewing just can't compete. As of today AR headset technology is still a number of years off maturity. The currently available headsets give us only a glimpse of what we should expect from future versions as headsets will become lighter and the screens will become better.

Because the consumer market for AR is currently lacking we are initially building this technology for VR. There are currently a large number of VR users who are looking for unique and compelling ways to experience live events. These users are not (and will never be) satisfied with the quality of 360 video due to the constraints imposed by having a fixed viewpoint.

We see an enormous opportunity to completely overhaul the way live events are consumed in VR by creating a virtual space in which multiple types of content can be consumed, comfortably, over a long period of time, with a volumetric video acting as the centerpiece.

Recording volumetric video

In order to realise our vision, broadcasters and content creators need to be able to record and distribute volumetric content in real-time. Condense Reality are building the tools to allow them to do just that.

Volumetric video is recorded with a large number of cameras recording a scene from many different angles. Advanced computer vision and machine learning is used to fuse the multiple camera feeds into a single 3D model. Creating these 3D models is far more complex than recording conventional 2D (or even “3D”) video and currently requires an enormous amount of computing power and time.

As “live broadcasting” is so important for many live events, our focus from a technology perspective is to develop algorithms and techniques for creating and distributing volumetric video in as close to real-time as possible.

The future of volumetric video

2020 is set to be an exciting year for both our company and volumetric video in general. As real-time volumetric video moves from science fiction into reality, so does the technology that allows you to experience it to its fullest (AR headsets like the MagicLeap One and the Microsoft Hololens). I expect this is a topic that will jump in popularity as more impressive demos are shown off in the coming months.

Next time we will be unveiling a number of volumetric video recordings, captured using our technology. See you then!

Nick 

Loading more posts…