tHE system offers a new way to control the process of tracking and solving an actor’s face in real-time.
If you’ve seen the Siren, Meet Mike, or Hellblade: Senua’s Sacrifice digital human demos, then you’ve already witnessed the kinds of impressive real-time facial animation technology that Cubic Motion is known for.
With those demos, and many other projects, the company has become a leader in offering computer vision technology that sits inside the process of translating – in real-time – the facial performance of an actor through to a high fidelity digital character for games, mobile, AR/VR and film and TV.
Cubic Motion has mostly delivered that via a software solution, but now, with its new Persona system, it is also offering a means by which customers can undertake the process themselves with hardware and software. befores & afters got a sneak peek at how Persona works.
What is Persona?
Think of Persona as a live capture and broadcast system for facial animation. Maybe you already have a CG character asset in mind that you want to bring to life interacting in real-time on stage with a live performer. Persona lets you deliver such a real-time animated performance from start to finish using both hardware (such as the helmet-mounted cameras for facial capture) and software (Cubic’s existing toolset and newly developed tools, as well).
“We’re transforming a one-off capability that Cubic could do at certain events into something that is a product that we can deploy to customers that they can use to do that kind of work,” says Tony Lacey, product manager for Persona at Cubic Motion.
And that’s the big difference. Previously, customers wanting to take advantage of Cubic Motion’s software had to have the company – as well as often other companies – involved at the live capture event itself. Now the customer can use Persona to run the process themselves.
Getting to this point involved significant engineering. “One of the toughest parts of making Persona was making it wireless, which we’ve done with on-body processing,” says Cubic Motion CTO Steve Caulkin. “We’ve made sure that the run-time tracking and solving is solved for each person, and we’ve separated that concern from the application that the performer is monitored and viewed in, so you don’t need a PC for every person.”
What do you get?
Sign up to use Persona and this is what you’ll receive, for every actor that is part of the facial performance:
Per-actor setup HMC
Helmet mounted camera system designed specifically to support live, with:
– 3 helmet sizes available with selection of padding to provide a bespoke fit
– Dual booms with twin Ximea 1K cameras in front-side configuration
– Built-in IR lighting and timecode sync electronics
– Mobile app to manage lights and timecode
Performance Pipeline
Capture, track, solve & stream animation in real-time direct from actor, with:
– On-body appliance pre-installed with all software, worn by the actor for an untethered performance
– High fidelity tracker and posed based solver pipeline able to deliver animations at 60 FPS
– Wired or wireless streaming of animations directly into Epic Unreal Engine V 4.19 and newer
Other Components
– 2x batteries with charger giving a typical uninterrupted performance time of 1 hour/battery
– Actor harness with configurable mounting system
– Individual Peli Case for each performer.
Persona’s software, Performance Manager, allows customers to manage four simultaneous pipelines in real-time from a single seat, running as a MS Windows GUI application. It lets creators remotely monitor pipeline HMC’s including all camera, tracking and animation feeds. Other tools also allow for controlling, recording and exporting of performance data, and for streaming previously recorded animations into Unreal Engine or Maya.
One extra consideration – Identities
As a Persona user, one thing to take into consideration is that you will most likely be bringing with you the particular digital character or characters that are to be driven by the actor. Since CG characters are of course made in a multitude of ways, Cubic Motion does get involved at an early stage to build clients an ‘Identity’.
“It’s how we embody the performance of that character in essentially a big file that we give to customers,” says Lacey. “It lets the actor, using Persona, drive the character. By doing it this way we can make sure they reach a very high level of fidelity.”
Persona is launching April 24th, and you’ll be able to find out more information on Cubic Motion’s website.
Become a befores & afters Patreon for bonus VFX content
