The Deep Defocus toolset at Wētā FX. An excerpt from issue #14 of befores & afters magazine.
Wētā FX has long been closely associated with deep compositing, initially with an early implementation of the approach inside of Shake and used for The Day The Earth Stood Still (2008) and Avatar (2009), then later inside of Nuke for The Adventures of Tintin (2011) and pretty much every project since then.
Indeed, deep compositing is simply something Wētā FX does as default. Now the deep compositing tools have essentially become multiple sets of nodes and workflows in Nuke.
Over the years, the visual effects studio has made several leaps forward in what is sometimes simply referred to as ‘deep’, the latest being the Deep Defocus toolset.
This deep tool allows artists to dial in desired depth of field and lens looks, such as bokeh and chromatic searing, to sharp rendered images based on physically plausible lens behaviors.
The tool is an extension of Wētā FX’s already extensive approach to deep compositing, which allows for compositing of objects having regard to their ‘pixel depth’ position from camera to be stored, removing the need for hold-out mattes and re-rendering.
Where Deep Defocus fits in Wētā’s toolset
“Deep Defocus came around from the fact that we don’t render the depth-of-field blur in our proprietary path trace renderer Manuka,” outlines Peter Hillman, who was also responsible for the original implementation of deep compositing at Wētā FX. “Generally, we try to avoid rendering depth-of-field blur because we can apply that as a post-process in compositing, that is, taking a sharp image and blurring it with a given lens settings.”
Lens mapping at the studio had been an ongoing exercise for many years, often to aid in incorporating rendered CG imagery properly into live-action plates. More recently, the idea of also mapping and replicating more ‘detuned’ and less sharp lenses has emerged, with, then, the need to craft VFX and CG shots that match that aesthetic.
Hillman says development on a deep defocus toolset on the compositing side also emanated from the fact that the VFX studio has always been looking to realize imagery in the most physically plausible way, in this case, by replicating what real-world lenses do. Before undertaking such an approach, they often faced different outcomes depending on which renderer or which compositing tool was being utilized. “For example, RenderMan’s settings only ever matched a kind of idealized perfect lens. So one of the interesting things we’ve been doing is working out how to mimic a vintage lens or detuned lens in compositing.”
Deep Defocus now sits as a set of nodes in Nuke at Wētā FX. Certainly, a defocus operation can occur without relying on deep data or compositing, but for the VFX studio the tool is so ingrained and part of its deep workflow that it does sit within that paradigm, as Hillman explains.
“To be able to render an image sharp and then defocus it, you have to deal with edges. Correctly handling edges between different objects when defocussing is a real challenge. Deep images store extra information at object edges, so we can now do a much better job with the defocus. Since we had written the deep workflow in Nuke, the tool had to read in deep, so it just became part of the toolset rather than as a separate thing.”
Where you’ve seen Deep Defocus in action
One of the more recent productions in which Wētā FX relied on its Deep Defocus toolset was on Ryan Coogler’s Black Panther: Wakanda Forever. On that film, there was a desire to match elements like the defocus, stigmatism, vignetting, bokeh and curvature of the live-action anamorphic lens choices made by DOP Autumn Durald Arkapaw–many of which were specifically detuned by the cinematographer–in the final digital visual effects work.
Bokeh replication, for example, can be seen in early scenes of the underwater mining mission. “We looked at a lot of underwater reference with submersibles, as well as what they shot on set, which had this beautiful bokeh of marine snow twisting, bending, and falling off,” details Christopher White, Wētā FX’s visual effects supervisor on Wakanda Forever. He collaborated closely with production visual effects supervisor Geoffrey Baumann and associate production visual effects supervisor Michael Ralla on lens tests.
“Those opening underwater scenes are still some of my favorite shots that I’ve worked on because, aesthetically, I just love imperfect stuff,” says White. “With the lens tuning and bokeh we were able to achieve, we went from a simple, dark scene with a light shining on the underwater environment, to something with many variations in color and texture. Straight out of the renderer it may have been an interesting underwater shot, but we were able to give it even more texture and variation in comp.”
“This process wasn’t treated as a post-effect thing added on top,” notes White. “It was actually part of the aesthetic, even as we were doing some of the 3D work. We could see what it would look like through this work and it became the creative part of that process. It wasn’t, ‘Oh, we’re going to throw it out of focus at the end or match what was in the plate.’ It was, instead, ‘Well, let’s push a little bit of this here, and a little bit of that there.’”
Wētā FX compositing supervisor Paul Raeburn adds that the particulate matter going through the water gave artists the ability to have many layers, different sizes of bokeh, and shapes floating through the scene. “You get a real dimensional sense from that. It definitely helped you to feel underwater, being in the currents with the marine snow floating with it.”
To get to that right look and feel for the film, Wētā FX embarked on a conversation early about the particular lenses that would be used during production. “One of the first conversations we had with the DP was about how important it was to match the look of her detuned and her special lenses,” says White. “It was so much part of the look. Paul jumped on this as a major project and worked with Peter to go through every little bit of what’s happening within those lenses.”
“Yes,” confirms Raeburn, “there was definitely a brief where Autumn said she’d previously been told that the lenses could be replicated, but she felt in the past whenever the image was sharp, it was just sharp, that it was just clear and crisp in a way that the lenses that she was tending towards–vintage and soft–were not.”
Raeburn comments that in the past few years many filmmakers have adopted breaking down lenses and detuning them, and that perhaps visual effects has been focused on a different direction.
“We’ve been striving towards perfect images, which the vastly improving optics and cameras of the last 20 years have given us. However, replicating the optics of older lens is much more of a challenge. This was about backtracking to exactly what lens aberrations do, how they do that, and where they do that in different parts of the image. There’s a lot that goes into it that is quite complex to model. We’ve made a lot of progress going deeper into those things, like how the bokeh changes over the frame because of the shape of the lens, and because of, say, flags in front of it and matte boxes. It can do some quite complex and interesting things, and we have to match how that diffusion realizes in a lens.”
This is just part of the article featured in issue #14 of befores & afters magazine. Read the full piece by picking up the mag here.
