Epic Games gets practical with remote multi-user virtual production.
During this coronavirus pandemic, we simply haven’t been able to shoot projects together like we had done before, owing to social distancing measures. But could we still shoot projects together, just virtually?
That’s an area being explored and implemented right now by Epic Games using its Unreal Engine and the company’s experience in virtual production and multi-user editing; this time at a remote level. Epic will soon be releasing a blueprint for productions to work this way. befores & afters got a special preview.
What exactly is remote multi-user virtual production?
Check out the video from Epic Games, above. It follows a multi-user production session, where a team of eight people collaborated from home on a piece of content featuring a CG environment and CG characters. Two motion capture performers in Xsens suits were ‘directed’ to produce real-time animation, while other artists could move props and other set elements around, scout locations, experiment with virtual camera angles and lighting, and continue to craft the character performances.
That’s Epic Games’ vision for remote multi-user virtual production, with Unreal Engine as the base. Excitingly, this is a set-up Epic says just about any group of people could establish and it’s in fact being followed right now by studios including Netflix and Bron to collaboratively previsualize content by their artists working from home.
We’ll go into a little of the detail about how it technically works below – and how you can start setting this up yourself – but one interesting thing to point out first is that Epic’s remote multi-user workflow actually comes directly from what was already happening say on virtual or LED stages (ie. those used for productions such as The Mandalorian).
“The idea there was that you could have multiple machines on the network and each one had the same scene loaded,” outlines Miles Perkins, business development manager at Epic Games. “What we were basically doing there was passing around the deltas – the difference in where certain objects or lights are. As a result, you can move objects on every single machine and it feels as if it’s happening simultaneously. The design was really made for being on a stage or where you might be in a different part of the building or over an extended network.”
“I think we pretty quickly realized, even before the pandemic, that as long as you were all on the same network, you could actually be across the country from one another and still do a multi-user session in Unreal Engine.”
What you can do in a multi-user session
Again, the demo video above is a great guide to what can be done in a multi-user session. In that project, the artists used a range of tools to work remotely in real-time in Unreal Engine, as Juan Gomez, lead technical account manager at Epic Games, explains.
“We used Take Recorder for recording the motion capture. We used Sequencer to play back and to keyframe the cameras. We used the Virtual Camera plugin with ARKit to stage cameras. And we used Live Link to connect the Xsens suits to Unreal Engine.”
Some of the collaboration happened with the aid of VR headsets, too. All of these tools are, of course, existing ones and part of Unreal Engine’s multi-user editing paradigm. You might normally use them in a studio environment but the tools can be used in this new remote way, which, suggests Gomez, has a slightly different creative aspect to it for artists.
“The good thing about remote multi-user editing from an artist’s perspective is that you can have the creatives – the executives or the director – right there with you virtually in the same scene. Artists are used to having their directors or their supervisors right behind their shoulders. This is a similar experience, but even better, because now they are also free to move around in the scene, too.”
A key area that this kind of remote working is being pushed for is previs, since that is the stage where imagery and assets can be crafted quickly and where things can be iterated on before any actual production begins. Plus, in a real-time set-up, previs assets are more and more becoming the prelude to final assets that might be used on a LED stage or in a live-action virtual production shoot, in any case.
“By getting into these things remotely, you’re actually starting production,” states Perkins, “meaning that as you’re getting into previs and you’re building your assets, these are the same assets that ultimately you can load later on a stage and keep working with.”
What needs to be set up for remote multi-user virtual production?
Epic Games will soon be releasing a blog post on this subject, plus you can already check out the company’s guide to multi-user editing, but here’s an overview of what you’d need to do to get started with a remote multi-user workflow.
One of the initial requirements will be establishing roles (ie mocap artist, art director, editor etc – this is similar to any production). From a technical perspective, working remotely requires connecting to a multi-user server on which you would run an Unreal Engine multi-user session. A VPN will do the trick. Epic recommends a particular system called Tailscale, a ‘mesh’ VPN alternative.
“The mesh network basically allows computer-to-computer connections,” outlines Gomez. “It doesn’t have the bottleneck of information going through a central location and then transferred to all the machines.”
Getting this network/connection part set up may require some expertise and bandwidth, acknowledges Gomez. But, he says, there will be guidance on the issue to come, and it should also be noted the virtual production community is incredibly active on different platforms in terms of advice and inspiration. Indeed, Gomez is excited about what the prospect of working remotely and continuing production this way can offer artists right now.
“The director or the DP can be in Australia and then the artist can be in Los Angeles and they can be all working at the same time on the same scene. They don’t have to say, ‘Okay, send me a file or send me a movie, send me a screenshot,’ and then wait – that person is free to explore and try so many new things. It’s basically a beautiful playground where everybody is free to go and play in the sandbox. ”
“If I want to convey one thing to artists about this,” adds Perkins on the prospect of working remotely this way, “it’s that even in this quarantine period, you can actually be working on story. You can actually be working on building assets, telling your story, doing camera moves; things that are not going to be useless. They’re actually going to be completely usable as you go into production.”
