‘It’s like a constantly evolving three-dimensional puzzle’

The visual effects of ‘Furiosa: A Mad Max Saga’, including those crazy War Rig scenes, impressive wasteland environments, immense rotoscoping, and how Anya Taylor-Joy’s facial features were translated onto the young Furiosa.

Furiosa: A Mad Max Saga visual effects supervisor Andrew Jackson sums up his role in helping to make George Miller’s follow-up to Mad Max: Fury Road as “a constantly evolving three-dimensional puzzle.”

“George would storyboard and previs a scene and then would go out into the real world and film trucks and motorbikes driving around in a landscape or in set pieces,” says Jackson. “Then in VFX we have to design one world and try and shoehorn all these little bits of live-action into that one world so that everything makes sense when you cut from one shot of the car driving around, then go to a wide, but, where is that in the wide, it can’t be too far advanced–all those things. It really is a puzzle to design a world that can incorporate all of those pieces of live-action without any of them breaking or feeling like they’re in the wrong place.”

And solving the puzzle is exactly what Jackson, and his entire visual effects team, have done with Miller’s latest entry in the Mad Max series. It’s a film that tells the story of the young Furiosa (Alyla Browne) being taken by Dementus (Chris Hemsworth) into the wastelands of Australia, and eventually the older Furiosa (Anya Taylor-Joy) exacting revenge on him for the cruel death of her mother.

Like Fury Road, on which Jackson was also visual effects supervisor, Furiosa contains no shortage of effects-filled vehicle chases and raids. These moments began as sketches and storyboards that evolved into detailed previs crafted by PROXi Virtual Production, an outfit from Harrison and Guy Norris (who is also the film’s action designer and second unit director).

Then, the scenes were filmed via a combination of live-action stunts and special effects in western New South Wales, Australia and at locations closer to Sydney (visual effects supervisor Paul Butterworth was on set for this filming). A team of postvis artists then pieced shot elements and temporary CG ones together for several months while editorial began, and as a significant digital visual effects effort also ramped up, with DNEG as the main vendor and Framestore also handling a number of wasteland sequences.

Another side of the VFX work, too, was the translation of Anya Taylor-Joy’s facial features onto the younger Furiosa actor, Alyla Browne, a task accomplished with machine learning techniques by Rising Sun Pictures. Metaphysic also crafted a synthetic character–The Bullet Farmer–with its machine learning tools.

In this befores & afters coverage of Furiosa, we break down a number of key VFX moments and characters in the film–the stowaway sequence, the attack at the Bullet Farm, scenes at Gastown, the attempted rescue of young Furiosa, the pursuit of Dementus, and the creation of young Furiosa and The Bullet Farmer.

The stowaway scene

While stowing away on the newly built War Rig, Furiosa is embroiled in a raid on the rig by several Mortiflyers. They employ motorbikes, parachutists and even paragliders in the attack, but are eventually repelled by War Boys, Furiosa and the rig’s driver, Praetorian Jack (Tom Burke).

“For that sequence,” describes Jackson, “there was a huge range of incredibly complex stunts that were done with stunt performers dangling from a crane on a truck that’s driving along beside the War Rig. The shoot was challenged a lot with terrible weather. It’s supposed to be a desert with hot sun and blue sky, but it was often raining and overcast, so they would sometimes move into an impromptu stage and a tent. The material that we were working with was a combination of that outdoor live-action and the impromptu studio. The challenge for visual effects was to try and bring that all into one kind of dynamic space where everything felt exciting and threatening.”

That task lay with DNEG, which orchestrated the visual effects here made up of environments, digital vehicles, digital doubles and effects simulations. It began the process with replicating the War Rig and other vehicles designed by production designer Colin Gibson. “We knew early on that we were going to need full CG versions of pretty much every vehicle,” outlines DNEG visual effects supervisor Dan Bethell. “We built those from great scans and reference photography and then would rig them for all the kinds of action needed, such as the driveshaft underneath spinning–Andrew and I could probably talk for hours about the design of the driveshaft’s universal joint, all those kinds of details that hopefully nobody will ever notice, but that are really important to us as Mad Max fans.”

Many of the vehicles consist of retrofitted parts scrounged from the wasteland. “What our CG build team did was try to adopt that same mindset,” says Bethell. “When there was a requirement to model the underside of the pursuit vehicle, the one that flips up onto the War Rig, because we didn’t have a perfect reference for exactly what was there, we just reused some parts that we did have. So if we needed a brake manifold or drive shaft that we didn’t quite have for any one of the many vehicles we built, we’d try and find something appropriate from elsewhere in the wasteland.”

“Another example of this is Dementus’ horde which has hundreds of motorbikes,” continues Bethell. “A lot of them were fully fabricated in CG from different parts of other vehicles. I think that’s a lot more interesting than just your traditional build from a scan to match a real-world vehicle, because you actually have to get inside the mindset and start building these things to not only work, but also to live in this wasteland mentality where everything is reclaimed, reused.”

The stowaway sequence also highlights a major area in which visual effects contributed to the film, and that is in the realm of wasteland environments. “I don’t think there are any scenes in the film which have photographic backgrounds that were filmed on set,” states Jackson. “They were complete all the way to the horizon. Pretty much every scene was generated.”

The reason for this approach came from the need to have a consistent environment for the scenes–which were often filmed in pieces–to play out in, as well as replacing what tended to be too green or too damp western New South Wales backgrounds. The digital environments were built up initially from photography from Namibia where Fury Road had been lensed.

“There’s a ‘famous’ Lightroom catalog we had with 120,000 images in it that we could refer to,” observes Jackson. “Luckily, Dan was also on Fury Road and was familiar with all of the places that we filmed. He and I could just go, ‘Let’s use this one’. Interestingly, they all had George Miller names as well, like Blanky Flats, for example. We had this shorthand between us about where to go looking for the right sort of environment or the right sort of images to build the particular environments that we were trying to build.”

“The world building on Furiosa was epic,” suggests Bethell. “We’ve got canyons, we’ve got desert plains, and different qualities of sand as we travel through. The detail and the characters of the environments on this are just as much a part of the movie as any of the set pieces. In many ways the scope of this was much bigger than Fury Road, because we have a larger fleshed out world.”

The implication for visual effects was, of course, that just about every character or vehicle would need to be meticulously extracted from live-action frames via rotoscoping so that environments could be composited behind them. Says Bethell: “The team was vast and every single one of them did an amazing job. There is a real craft to doing that kind of work. Often, until you know what the background is going to be, it’s very difficult to do. We were constantly working on refining how these characters integrate with the environments. It’s not just a binary operation. There is a real craft and an aesthetic to doing that paint and roto work.”

During the raid on the War Rig, the War Boys utilize their thundersticks to try and fend off the Mortiflyers. Some filmed explosion elements were captured, but in general the fiery thunderstick moments were FX sims. “One of my favorite shots that DNEG VFX supervisor Stephen James took care of,” describes Bethell, “was where we start low following one of the Mortiflyer bikes just as the first parachute takes off. Then there’s a takeover and we come up with a parachute and loop around the back of the War Rig, and there’s a couple of very judicial applications of thunderstick explosions, but also big kicks of dust. They’re actually quite important to blend between multiple plates and help with CG takeovers, so they were very functional.”

The stunning climax of the stowaway sequence is the deployment of the spinning Bommy Knockers defense device at the rear of the War Rig. It takes out The Octoboss’ flying bike and the inflatable kite he is towing, which itself eventually wraps around the War Rig’s end structures.

“During filming,” explores Jackson, “the Bommy Knocker structure of chains and weights was a static thing, it did not spin. There was a prop for whenever the truck was driving around, but all of the spinning and all the subsequent destruction of the kite structure was all generated. The motorbike that the Octoboss is riding is real, although we did actually end up replacing that in a few shots just to give it a little bit more life because it wasn’t moving the way we needed to.”

The octopus kite was based on a kite from a Festival of the Winds event held at Bondi Beach, advises Jackson. DNEG then rigged it like a creature–indeed, combining it with the Bommy Knocker was a major challenge for the VFX studio, notes Bethell. “Working out how to entangle and snare this kite, pull in all the ropes and the rigging and then shred it was a massive undertaking. Lots of people spent a lot of time iterating and experimenting and researching all of that.”

A burly brawl at Bullet Town

As Furiosa learns about the art of War Rig driving from Jack, they are ambushed at Bullet Town by Dementus. This sees the War Rig being driven deep into the Bullet Town mine, crashing into vehicles and structures, and a sniper stand-off between Dementus and Furiosa.

A set was built for Bullet Town that consisted of the base of the main gate and the roadway in, plus some excavated favela dwellings and earthworks areas. DNEG effectively extended the entire Bullet Town. Many aspects of its design were based on reference of real open-cut mines in Australia. “We also researched a lot of open-cut mines and slowly started to build up a visual language of what the open-cut faces look like and what service roads look like,” says Bethell. “So this sequence had an incredibly detailed environment, vehicles, large effects simulations and choreography of big set pieces.”

At one point, the War Rig hurtles around the various levels of the mine before pushing a Kombi-Van into the depths below. “There were live-action plates for both vehicles up until the point where the Kombi-Van leaves the ground,” says Jackson. “Quite often in this film there’s a lot of cross-over from a live-action vehicle to a CG one.”

“I don’t think I’ve ever done quite as many,” adds Bethell. “We’re used to seeing live-action to CG takeovers for digi-doubles, but in this one, the characters are vehicles. So we did a lot of that with vehicles, but also with cameras, which is testament to our layout team. A lot of the time, we’re using real world tracked cameras and then we’ll take over to a synthetic camera, but it has to be in a way that feels natural. For example, when the Kombi falls down the pit, a human camera operator would be following and then maybe overshoot slightly and have to come back. So to get all those kinds of real world nuances of camera performance into our CG cameras as well, and then to blend seamlessly with live-action camera plates, was quite a big challenge.”

Another signature shot in the sequence is when the War Rig has been driven into a brick smoke stack structure that Dementus has been lurking atop of, causing the towers to collapse and him to nearly fall from the structure. A wealth of freshly-made bullets then fall onto and past him.

“For that scene,” outlines Bethell, “Chris was hanging off a gantry on set. The bullets then come down the gantry. They were a fairly complex effects simulation. We didn’t want to cover Chris’ face, so some of the bullets had to be slowly coerced out of the way. Bear in mind, there are hundreds of these bullets, so we had to do it in a way that didn’t look contrived or overly art directed.”

“Actually,” recalls Jackson, “for quite a long time, there was one bullet that was very individually animated to hit him in the eyes, but I don’t think it ended up in the film.”

At the gates of Gastown

Having seen Gastown only in the distance in Fury Road, we now go up close and inside the oil refinery for several scenes in Furiosa. Here, DNEG relied on concept art and real photography of existing refineries to build up the structures of the town, which initially appears in full production mode and then later in a more decayed disheveled condition after being taken over by Dementus.

“I always described Gastown as one place that had its own weather system,” jokes Jackson. “It was this toxic refinement that had its own nasty clouds and stuff going on that was kind of local. We tried to achieve this idea that that cloud system was right there, and when you’re looking out, you see it above you. But when you look to the horizon, it’s clear blue sky and then there’s this nasty little world hovering over it.”

Bethell shares that a particular plastic resin refinery in Sydney served as a key piece of reference for Gastown. “I went down there with some of the production. We spent a couple of days down there scanning it, photographing the pipes, the manifolds, the valves, and all these bits to build up a kind of a visual language, a bit of a Lego kit, if you will, of all these components combined with a lot of research into oil refineries. The furnaces over on the left as we come in through the Gastown gates are based on real world aspects, but then they’ve been weathered and aged and broken and fixed with reused parts from other bits of Gastown in that same spirit that we applied to the vehicles.”

Eagle-eyed viewers, particularly residents of Sydney, may have noticed that the towers across Gastown’s causeway of the oil moat were modeled on aspects of Sydney’s railway system. “The mechanism that holds the wires above the electric trains in Sydney have these counterweights hanging on the side,” explains Jackson. “If you’ve ever sat on a train and looked out, you’ll see all these little piles of weights stacked up. Well, they’re exactly based on those because, like everything in these films, it makes sense that things like that would be reused from the real world. And then in the visual effects world, we like to use things from the real world as well, capturing those inspirations.”

In pursuit of young Furiosa

Earlier in the film, we see Dementus’ raiders kidnap young Furiosa from the Green Place and ride out into the wasteland. They are pursued by Furiosa’s mother, Mary Jabassa (Charlee Fraser), who brings a number of them down before herself being captured and tortured. Framestore delivered visual effects for these desert moments, extending and replacing backgrounds and placing characters and bikes into the wasteland.

“These scenes were shot primarily outdoors,” outlines Framestore visual effects supervisor Josh Simmonds, “and we were originally tasked with some set extension for them just to get that additional desert into the background. Then, due to some pretty inclement weather at the time, lots of thunderstorms and rain, we ended up pretty much having to replace all of the ground right up to where their feet were because they were basically walking around through clumpy wet sand, which doesn’t really lend itself to a barren dry desert. That was combined, too, with very dark skies through that whole filmed section. So, we did have to balance that out and make sure that we could find ways to fit the additional lighting that had been done on set with a naturalistic sunny environment.”

“There was quite a bit of mixing of takes through that sequence as well,” adds Simmonds, “where some components of the plate were shot in a studio for safety reasons. They were putting a young child over the side of a motorbike–you can’t really be riding around like that. So, combining some of those interior takes with exterior elements was one of our big challenges for that sequence.”

Tire tracks proved to be one particularly important aspect of the work for Framestore. “There’s tire tracks as they make their journey, but there’s also tire tracks from their journey a few hours previously from where they came from,” notes Framestore executive visual effects producer Julian Dimsey. “So, there’s a whole lot of continuity that we had to really think about.”

Blowing sand, sand trails, dust and even a dust storm became another part of Framestore’s work on this part of the film. The studio, as Iloura, had previously crafted the toxic dust storm on Fury Road, although here the intention was that the dust storm that reaches Dementus’ camp would be only about 40% that level of intensity.

“George has obviously spent quite a bit of time in deserts,” notes Simmonds. “He knows what they look like and we have to do our best to try and keep him happy. I think we solved that fairly early on by actually doing a whole bunch of dune simulations that could give us key dunes, effectively tracking how sand moves and letting prevailing wind sculpt how some of those dunes worked. And then, we could up-res some of those with our asset team as well and get them super crisp and detailed, because at 4K, you need a lot of detail in those sand dunes to sell the scale.”

Like DNEG, Framestore was required to extract the actors from plates–without bluescreens–and replace just about all the backgrounds. “The sheer amount of flyaway hair and fine details made it particularly challenging,” says Simmonds. “There were almost 6,000 days of roto that we used on 240 shots, which is pretty astronomical.”

“Also,” continues Simmonds, “generally, when you start off with a sequence, you look at the plates and you match the lighting exactly to what’s happening in the plates. But here, sometimes we had sequences that may have been shot over a series of days where the light changes dramatically over the course of the days. There was a desire to have a consistent look for the entire background environment. So, in many of our shots, we actually ended up having two completely separate light rigs, one to light the foreground sand and get the shadows casting in the right direction, and then another one for the background dunes to make sure that they had the right shape, and they weren’t being too flattened out. I don’t think I’ve ever embarked on a project where there were two suns in just about every single shot, but I think it did help create a sense of continuity across those sequences.”

Furiosa on the chase

One of Framestore’s other principal sequences sees Furiosa pursue Dementus and the remains of his biker horde. She does this in the ‘Cranky Black’ hot rod. Framestore built a fully digital version of the practical car to augment a number of driving and crash scenes, created several Dementus horde digi-doubles, and again dealt with replacing backgrounds.

“The practical car looked fantastic and performed really well,” points out Simmonds. “But there were times where there were things just far too dangerous to be doing with a real car, like side-swiping motorbikes, and we’d have to do a digi-take over on that. We also built quite a few motorbikes along the way, all based on the art department and vehicle department’s designs.”

“The main challenge here was getting the lighting exactly right,” adds Simmonds. “We’ve got a fairly robust asset pipeline here where we know that a shader is going to respond the way it should, like chrome should come out as chrome, but there’s still a lot of artistry in getting the right specular response and getting all that kind of wear and tear that you need to match. We had a heap of reference photography of the practical vehicles under different lighting circumstances to know exactly how it was responding. There were some close-up shots where the Cranky Black was shot in a studio with Furiosa driving it. Here you had studio lighting on the car, rather than bright outdoor sunshine. So, in many cases, we were reskinning the car completely with our CG car.”

During Furiosa’s chase of Dementus, she causes one of his henchmen, Smeg, to be sucked into a sinkhole. For that moment, the actor was filmed sliding down a sandbank, with Framestore then modeling and simulating the sand sinkhole and sand grains completely around him.

“George definitely had a vision across those half-dozen sinkhole shots as to how he wanted it all to progress and how he wanted the actor to finally be sucked under the sand,” outlines Dimsey. “They had a good pull rig to simulate the gravity of a steep slope. The material they gave us for that was a really good starting point. Our team then did a fantastic job of realizing exactly what George wanted with CG simulations.”

Connecting young Furiosa to Anya Taylor-Joy

In the film, young Furiosa (as a child and teenager) is played by Alyla Browne. For her scenes, to give her a more close appearance to the older Furiosa played by Anya Taylor-Joy, Rising Sun Pictures was engaged to bring their REVIZE machine learning visual effects workflows to the younger character.

“What we did was,” explains Jackson, “we took Anya’s face and RSP used machine learning techniques and technology to generate the components of the younger face. Then they provided the face replacement components to the other VFX vendors who then incorporated that into the shots.”

“We didn’t replace her whole face,” continues Jackson, “but obviously her eyes are a very distinctive part of Anya’s face. So we replaced Alyla’s eyes and usually nose and some of her top lip to give her a look of a younger version of Anya. There was always a percentage of Alyla and a percentage of Anya, and it grows to more like 80% by the end of the young Furiosa’s time in the film and as we get closer to the switch over to Anya.”

Jackson is also at pains to point out that the approach was not a ‘push button’ one, and it included fine levels of compositing to pull off, especially where hair might be hanging over young Furiosa’s eyes or face, or if it was covered in dirt. However, all of the performance from Alyla was retained by RSP–“That absolutely drives the position of where the eyes and the face and all of the minute expressions came from,” notes Jackson. “I think the technology is so amazing at capturing all of that performance so perfectly. I’ve never seen full CG face replacements that are as close to the real person’s performance as we’ve seen now on these shots.”

Dan Bethell from DNEG praised the efforts of RSP in their machine learning generation of face pieces that were then turned over to his studio and to Framestore. “RSP are masters at that aspect, but they’re also very good at collaborating. We worked out a workflow where essentially we would receive plates with enhancement, so we could work our shot through to nearly completion using the principal photography of Alyla, and then we would effectively just get a drop-in replacement, which would sit over her face. Effectively we just received a 2D element that we put over the plate upstream of our VFX.”

Rise of The Bullet Farmer

Another character made possible with machine learning techniques, this time by Metaphysic, was The Bullet Farmer. Richard Carter had played that role in Fury Road, but passed away in 2019. So, on the set of Furiosa, actor Lee Perry performed the character before being replaced by a synthetic version of Carter.

Metaphysic built up the resurrected Bullet Farmer starting with reference from Fury Road, and other footage and imagery of Carter, according to Jackson. “The machine learning tool needs as much material as you can give it. It goes through this training process, which can take days or weeks and gets refined. It’s better and better the longer you leave it to do its training. It’s a CG face, but it has all those super subtle nuances that can be missing when you just model a CG face normally. Those are the things that I think just tip it over the edge to feeling believable, away from the Uncanny Valley.

“Also,” says Jackson, “this machine learning approach is actually doing something that we’ve been doing for years–face replacement for stunts and actors–but it just does it better. It’s really just a much better version of something. The approach is new, but we’ve been doing the same job for a long time, just not as convincingly.”

Finally, a note from Metaphysic’s VFX supervisor Jo Plaete that he published on LinkedIn:

“For this iconic role, our teams at Metaphysic.ai leveraged our neural performance toolset to bring the Bullet Farmer back for Mad Max: Furiosa, channeling Richard Carter’s likeness through the performance of stand-in actor Lee Perry. We meticulously trained our models on footage from Mad Max: Fury Road, ensuring the perfect fusion of Perry’s performance and Carter’s identity.”