Beyond the green screen: the dawn of virtual production technologies

What happens when some of the most creative minds in the education, research and commercial sectors get together to explore new and innovative virtual production technologies?

Over the course of three years, Project Dreamspace brought together universities, industry and creative professionals to work together to research and develop tools that allow creative professionals to combine live performances, video and computer-generated imagery in real-time, and collaborate to create innovative entertainment and experiences.

Why? To make it possible for directors, designers and artists to blend live action and virtual content together on-set or in a performance space, and see the results immediately.

There was a time we used to define virtual production as ‘real-time live digital production’ - however the term has since evolved to encompass the end-to-end workflow of any film production that includes VFX.

That means virtual production as we know it today involves everything from pre-production to final delivery, including CG previsualization; virtual technology used live on set that enables the crew to visualise 3D extensions of the set and CG characters, and what was previously referred to as ‘post production’ (but which today begins before the shoot even starts).

The Dreamspace project has seen huge strides taken in the technology and techniques used to combine real and virtual assets, live on set. Some of the questions the team explored include:

• How do you capture the physical lighting environment, take that and put it into your virtual scene, so that you can seamlessly match the physical space with the virtual space you’re rendering?
• How is camera technology going to change in the future? What happens when you get to the point that you’re recording depth per pixel on the set, and you have that available to an artist post production?
• How do you get final quality rendering - which may take many hours per frame in post - working in real-time on set, so that you can really understand lighting?
• How do you get a live, configurable compositing pipeline available to you on set?

And how do you connect this to easy, intuitive to use tools that let you control this very complex setup and environment but with a very simple interface that non-technical users can pick up and put down?

The journey to answer these questions led to the development of imaginative new tools and techniques that are changing the way entertainment experiences are made. More than that - they’re changing the scope of what’s possible to create, opening up a new world of possibilities that’s as broad as the limits of human imagination. As Ben Grossmann, the Co-Founder and CEO of Magnopus, and Oscar-winning and Emmy Award-winning visual effects supervisor, explains: "Virtual Production systems will likely spread to the lower budget films after big budget productions have battle-tested the ideas and the tools mature. Once they become commonplace, this will streamline film production overall and also create a ‘sandbox’ for new directors, writers, and artists, with untested ideas, to ‘roam free’ before hundreds of millions of dollars have been invested. Plus, because the tools that are being developed essentially all run on consumer hardware and use freely-available game engines, we can give the same story-creating tools to the public that we give to big name directors. Then it’s just a matter of adding the best talent!"

"There’s also the tantalizing prospect that asset creation for virtual production leading to a theatrical release can cross-over immediately to VR experiences. As VR continues to gain momentum, this will become a driving factor in shifting asset creation to pre-production for use on-set in real-time visualisation as well as the final delivery of immersive experiences and games.”

Intrigued? Watch the video to find out more.

If you’re interested in finding out more about the future developments in VFX, check out our article: Are you ready for the new era in filmmaking?