So…you need to track organic and warped surfaces, hey?
Tracking faces, body parts, cloth and bendy surfaces has always been very tricky. With their new sub-planar tracking engine PowerMesh in Mocha Pro 2021, Boris FX has set out to make those tracking tasks a lot easier.
VFX artists will already know that tracking algorithms tend to work incredibly well on rigid surfaces, while organic surfaces can be another matter, especially when a shot has occlusions or motion blur.
“Optical flow solutions can be slow and difficult to work with, so we took a different approach,” outlines Ross Shain, chief product officer at Boris FX, in relation to the development of PowerMesh.
“VFX artists already used Mocha Pro for face tracking and stabilization, for example, Pixel Magic famously did a lot of head replacements on the film Green Book. With PowerMesh the same process is not only much faster but now artists can work on warp stabilized image for difficult paint and clean up shots.”
So how does PowerMesh work, and what were the technical challenges in implementing a tool that could track warped surfaces? Shain says it goes back to how artists were already using Mocha’s planar tracker, including with the rigid planar constraint.
“This allows us to use all the pixel information in the entire area to calculate one consistent result, which can be extremely stable. With a non-planar surface, especially when the surface is deforming, each tiny area of the mesh needs to have its own motion calculation. When the tracking areas get very small, it’s easy for errors to be introduced and parts of the mesh to deviate away from where they should be. We had to find a way to overcome this while still allowing the mesh to deform as needed – which can vary from shot to shot.”
“Our solution,” continues Shain, “was adding a smoothness control which allows the user to dial in the right amount of rigidity, and an auto mode where Mocha will try to find the right balance.”
You can see in the videos above and below what the results with PowerMesh can be. To get to this point, Boris FX had a number of beta testers—mostly professional VFX artists—reviewing the tool. One added surprise of the beta testing process was that users found PowerMesh incredibly useful for roto.
“We were very focused on use cases like adding removing tattoos on the skin and digital makeup tasks, but a great surprise is how PowerMesh can be applied to rotoscoping,” notes Shain. “We are hearing that PowerMesh driven splines vs typical Mocha planar tracking is helping artists to roto with even less manual keyframes than before. We are definitely looking forward to seeing how the Mocha user base will use this tool in real productions.”
Mocha Pro 2021 is now available, and also includes AdjustTrack 2.0 for improved track editing, and a Python Script Editor. You can find out more at Boris FX’s website.
Become a befores & afters Patreon for bonus VFX content
