Different vendors on ‘WandaVision’ explain their synthezoid methodologies (plus, how Baby Vision was made!).
Earlier this week, befores & afters published a story featuring WandaVision visual effects supervisor Tara DeMarco explaining how Paul Bettany was filmed on set as the character Vision.
DeMarco mentioned that three visual effects studios were principally responsible for translation of Bettany, who wore partial make-up on set, sometimes blue, sometimes red/purple, into synthezoid form, prior to him demonstrating his much more destructive powers. Those studios were Lola VFX, Monsters Aliens Robots Zombies (MARZ) and SSVFX (other vendors, including Rodeo FX, Digital Domain and ILM, also crafted Vision shots—we’ll be diving into DD’s extensive fight sequences with the ‘Visions’ in the next article).
To get a handle on just what went into tackling shots that involved going from live-action Bettany to the final Vision face and head, I talked to visual effects supervisors Trent Claus (Lola), Ryan Freer (MARZ) and Ed Bruce (SSVFX) about their individual approaches to this VFX challenge.
It’s worth noting that these studios also had to at times often craft digi-doubles of Vision, his CG cape, other suit pieces, and transitions from Paul Bettany to Vision, but this article is focusing on mainly the actual facial work.
Lola VFX’s history with Vision
Lola completed around 90 Vision shots, primarily in episode 105. The visual effects studio has a long history with Vision, with visual effects supervisor Trent Claus noting that he and his team at Lola designed the character for screen during Age of Ultron, based on a concept by Marvel head of visual development Ryan Meinerding. Back on Ultron, Bettany wore full prosthetics, but the vendors ultimately always removed them and replaced them with CG. The face, on the other hand, would typically be a 50/50 blend.
“Chris Townsend, the visual effects supervisor on that film, was a big advocate for Vision being this like undefinable quality, who in some ways he looks plastic,” outlines Claus. “In some ways he looks metal. In some ways he looks human and the way he always described it was that he wanted to approach the Uncanny Valley from the other way—taking a live action human and trying to make him look a little less real. So, the design elements on his face ended up to the point where they were semi-rigid.”
When WandaVision came around, Lola was involved in some initial lookdev testing (in the Fall of 2019). “When we did that, we were also starting to discuss what a black and white Vision would be,” recounts Claus. “Now, I’m a huge, I love Lucy fan. Somewhere in the back of my brain, I knew that Max Factor had done the makeup on I love Lucy and a number of other period shows and had experimented with blue and green lipstick so that it would appear more natural in black and white—the typical red makeup or red lipstick when viewed through black and white film stock looked black. So when it came time to where we were talking about Vision, my I love Lucy trivia came in handy, and I suggested using a lighter color, like blue or green.”
“Their initial plan was to just paint him gray, paint him the color that they wanted him to appear in black and white,” adds Claus. “But my suggestion was that if they ever decided to change his tone or his value, it would then have to be roto’d because grey is very difficult to key. So I suggested blue or green and they ended up with blue.”
For the shots Lola ended up being tasked with, Claus says they revolved around matchmoving and tracking, initially. Indeed, the tracking markers placed on Bettany’s face get used in Lola’s workflow as anchor points on which to register the CG elements. “They all have to get removed,” explains Claus. “They all have to get painted out and that has its own challenges, especially on the mouth and the lips and the eyes. But they are super helpful.”
Bettany’s real eyes, nose and mouth were generally retained, with the other parts of his head—including the panels and Infinity Stone, animated and rendered in CG. “We then go in there and really examine every pore and decide what to keep, what to lose,” states Claus. “For laugh lines or creases beneath his eyes or crow’s feet, if you get rid of all of them, you lose the expression and you lose the performance. So you have to keep some, but, you know, he can’t look like he’s been out in the sun or something. He’s a synthezoid. He’s perfect. So they have to look like they’re naturally folding in that spot.”
Lola’s main toolset for the Vision compositing work was Autodesk Flame. The studio does also utilize Foundry Nuke. “There’s certain 2D manipulation portions of the Flame, which are very, very useful and not only useful, but very fast in Flame,” says Claus. “You can almost do everything in both, but the things that we tend to rely on the most for the work that we do are a little bit faster in Flame.”
With all of Lola’s visual effects work for Vision, Claus praises the performance of Bettany as, of course, the complete basis on which the character is created.
“He has a particular way of playing the character. He’s very understated, he’s thoughtful and quiet. It might be just a little bit of an eyebrow raise that shows you what Vision’s thinking, or that he doesn’t feel like Wanda is telling him the whole truth, or something like that. Or, he won’t do like a giant smile. He’ll do just like a small raise at the corner of the mouth, very, very subtle movements that all have to be maintained and not hidden in any of the design that goes over his face.”
MARZ goes black and white and more
Taking on the lion’s share of black and white Vision shots (mostly seen in episodes 101 and 102), as well as many color ones, was MARZ, which crafted 400 shots for the show with about 85% of them involving Vision. The studio brought a pre-existing Vision asset into their pipeline for an early test that resulted in them being awarded the work. Visual effects supervisor Ryan Freer notes that having to then craft black and white shots upfront was a quick leap into detailed work for the studio.
“It’s not just like throwing a LUT over top. His panels needed to have certain contrast levels. His metal needs to be a certain heat, otherwise it’s just like this massive ball of spec that just blows out everything. Also, he was very slapstick-y and very comedic, and you’ve never seen him like that. It was something that they were all worried about, when they were filming, ‘How is he going to look with all the CG on him? Is it going to read?’”
MARZ’s process for the black and white Vision began with the plates, which were color plates with Paul Bettany in blue makeup and tracking markers. “We have to ingest the plates and then we track the shots,” details Freer. “Most of the scenes within the first episode, he’s within the kitchen or he’s in the living room—we would recreate the entire scene and all the lights that are actually in the scene based off HDRIs that were taken on set.”
Once tracking has occurred, the next step is animation in Autodesk Maya. “Animators matchmoved with a rig of him talking,” says Freer. “So every little expression, even down to a little bit of a cheek, even his eyes. Because when he blinks, things move. Every little bit of his face is matched, and it can be very tedious.”
Freer makes mention here also of Vision’s neck as a challenging tracking exercise, especially working out what the neck was attached to (the design of the character effectively has the neck tucked into the jacket). “You had to go in and eyeball and really fine tune that neck.”
The next step was taking a matchmoved CG head through lighting, followed by compositing in Nuke, where that CG head was ‘delicately’ overlaid over the top of Bettany’s face. “The tracking markers are obviously removed,” advises Freer. “We have developed an in-house machine learning AI script that removes the tracking markers from his face beforehand, which is something that we developed specifically for this reason. You can literally do 200 shots over a weekend, and it’s done.”
Other plate preparation work had to carried out, also, including edges for the blends between live-action and CG, the neck connection areas, and some slight background removal and re-creation since Bettany’s ears are removed.
“What was really cool throughout this whole show was, we were doing just these minor tweaks here and there to him, that you might not notice as a casual viewer,” observes Freer. “The first couple of episodes, Vision has eyelashes—he doesn’t have his synthezoid eyes. But then later on he has the electronic mechanical ‘contacts’ over his eyes. So that was pretty cool how that changed.”
Freer commends his team for the attention to detail on Vision, particularly with the matchmove and tracking, the animated micro-movements necessary for the character’s face panels, the smoothing of the live-action skin, and maintaining a convincing level of highlights on those panels and skin. “We would actually pull some highlights from Bettany’s actual face. So within comp, sometimes we would pull some highlights that just didn’t show up in the CG, that would give his face just a little bit more character. We’d pull some of that back as well when needed.”
MARZ’s biggest and longest Vision shot was from episode 106, where the character flies up into the air to take in the town of Westview. Production filmed Bettany for that on a massive wire rig.
“The entire town is CG and everything and the only thing that is plate in that whole shot was just two eyes and nose and a mouth,” says Freer. “All that work for two eyes and nose and mouth, but you know what, if you don’t have it in there, I mean, just the expression comes out so much more when Paul is there.”
SSVFX goes to the circus (and also delivers ‘Vision Baby’)
SSVFX was tasked with 497 VFX shots with their main episodes involving Vision visual effects on 107, when the circus comes to town, and 108, when a disassembled vision is shown at SWORD headquarters. The visual effects studio specializes in facial/head replacement de-ageing work, and they followed much of their own bespoke methodologies for the Vision shots.
While SSVFX approached their Vision in some similar ways to Lola and MARZ—i.e. with matchmoving, animation, skin smoothing and combining CG with the retained live-action areas, the studio adapted their proprietary de-ageing workflow to apply to both the clean-up of Paul Bettany’s face, and then also delivering a secondary finer matchmove that was per-pixel.
In particular, too, their unique system allowed SSVFX to get ‘beneath’ the on-set lighting enabling manipulation or addition without affecting core light shape. “That’s what we’ve been doing, in our facial augmentation tools, ensuring a non-destructive effect to the in-camera lighting,” says Bruce.
Skin clean-up and smoothing was a significant side to the work, too. “He was wearing quite strong makeup that literally almost defines every single pore, wrinkle and blemish that a face has,” discusses Bruce. “This was done similar to a cleanup step in the way that we would do with our de-ageing tools—it’s not a blur, but more a clean-up/balance per pixel and pore.”
“Then,” continues Bruce, “you’re also reducing the way the specular hits, diffusing out the specular’s shape, so it has a different type of sheen, Visions look, before you would start putting the CG panels on.”
Vision also has refined details in his face, such as the ‘carbon fiber’-like appearance around the sockets and panels. “His eyes were also replaced with a digital contact lens with an array of mechanical features just around the iris,” says Bruce. “Plus, the eye color is different.”
“With such nuance to each component of Vision’s head, how they connect, blend and interact around Paul’s emotive performance, we created a defined naming structure and a language for every single panel and connecting point,” adds Bruce. “Language for specific notes is crucial, especially with repetitive shots to ensure the continuity of every minutia detail that makes up the facial features.”
For the disassembled Vision, production filmed a basic mannequin of parts laid out on tables, with Paul Bettany also filmed in a lying down position. SSVFX was tasked with designing the look of Vision in his destroyed state ensuring the outside skin areas of Vision were visibly read as Vision, but they also needed to show his interior makeup of circuitry and other synthezoid-ness.
“The only reference of Vision’s interior was from Infinity War when his Mind Stone was ripped out of his head by Thanos,” notes Bruce. “You briefly got to see inside this cavity in the head, and you got to see the interesting mix of tech, with all the lights slowly dying. We used that as our initial bible for the ultimate design. Our team really enjoyed creating this look. Building Vision from the ground up. Skeletal, circulatory, respiratory, nervous and muscular systems designed and visualized in synthezoid fashion. It became quite an asset with lots and lots of feet of fibre-optic cables.”
And then there was Baby Vision. This was just a still photographic frame amongst other photographs of Vision growing up. The still provided to SSVFX was actually a baby photo from a VFX coordinator on the production.
“The second we saw the photos,” relates Bruce, “our crew just got really excited. We’d all been watching The Mandalorian which had the whole Baby Yoda thing. When we got sent this image, we were like, please tell us there’s going to be some actual shots! But of course it ended up only being a still.”
However, Bruce says the crew remained excited. The visual effects supervisor also remembers observing that ‘Baby Vision’ might be “the most talked about thing of all of our work. And of course, once aired, Twitter lit up with joy.”
To convert a human baby face to Baby Vision, SSVFX worked only on the photograph, but still modelled the Vision head “as if it was a one-frame shot,” says Bruce. “There was quite a lot of conversation to his baby look, a lot of laughter and happiness from crew and client. I knew it’d be a great talking point.”
