The story behind those portals, Iron’s Man finger-snap, and making the ‘mega’ blip.
At FMX in Stuttgart recently, Avengers: Endgame production visual effects supervisor Dan DeLeeuw showed a breakdown of the cast being filmed on stages in Atlanta for the movie’s final showdown against Thanos and his army. It was incredible to see how those plates were transformed into the finished battle scenes, and a testament to the planning involved and the VFX that lay ahead for the artists to convert the plate photography into fight sequences.
Weta Digital took on a large proportion of these battle scenes. Of course, much of the Avengers vs. Thanos battle ended up as fully-CG shots, with digital characters and environments, but significant work also lay in integrating the live action with synthetic parts of each frame. Here, Weta Digital visual effects supervisor Matt Aitken breaks down some of the key components – including the portals, the fights themselves, Tony’s click of the fingers, and the final blip – for befores & afters.
Planning the shoot
Matt Aitken (Weta Digital visual effects supervisor): The way production approached it is that they really will shoot as much as they possibly can. And, you know, I’m on set with the on-set VFX supervisor and the rest of the Marvel visual effects team and to one side we’re kind of standing there going, ‘We won’t be able to use this.’ But it doesn’t matter, because it’s about exploring the story and it’s about giving the actors context for the work overall. It’s about approaching it as holistically as possible as an actual shoot, which is fantastic.
They extensively previs’d these fights – the previs is available on set, and it is treated like the Crown jewels. It’s very, very precious. It’s not widely distributed because the nature of this film in particular was that to reveal any of the story points, particularly for the third act battle that we were working on, to reveal any of the story points early, would have spoiled the film for the audience.
Typically what would happen is that Jeff Ford the editor would release updated cuts of the previs. He would have an assistant editor on set with a laptop and if you wanted to view the previs you had to look over his shoulder and say, ‘Oh, can you scroll through to such and such a section, I just want to see how that plays out.’ They’d be very happy to do that and you had complete access to look at it, but you couldn’t retain a copy of it yourself.
Most of the time we were working with The Third Floor, which did the previs. However, for the ‘Women of Marvel’ shots, they never got a chance to fully shape that. They shot some material, most of which we couldn’t use. So for that section in particular we got a call, which was quite late in the day, and they had re-conceived it. So we essentially got a text description of how they wanted it to play out. Our animation team previs’d that sequence and we cut it ourselves. We sent it to Jeff Ford who tweaked the cut, made it better, and then that was the blueprint that we finished that sequence to.
Live action at Pinewood, Atlanta
The whole third act battle was shot on greenscreen stages on quite minimal sets a lot of the time. They’d have dirt floors. The overall art direction look for the ruined Avengers compound was a mixture of destroyed building material from the ruined compound, but there were also tree stubs, burnt tree stubs and things on fire. Then, the minute it was all cut together, the way it played was that it was a little bit too like you were in a destroyed forest rather than a destroyed kind of architectural built environment.
So, we often referenced – and this game takes you to places you often don’t want to go – we referenced Mosul and similar places, looking for destroyed built-up environments, concrete rubble, dirt and masonry and rebar. Ultimately we would end up rotoscoping the characters off the plate and building the CG environment to help tell the story. We did keep that ruined forest look for some sections of the battle – there’s a section where Thanos is really laying into Cap and the shield gets destroyed and the destroyed forest look worked well there.
Choreographing those killer fight scenes
There were a lot of fight scenes to get done. For example, if Thanos is fighting a cast member, we’d have that cast member or a stuntie playing one of the lead characters; it’ll be on one of the sound stages on a hero set. So if Thanos is fighting Chris Evans’ Cap, it’ll be a stuntie playing Thanos. One of the stunties called Danny Hernandez played Thanos a lot for this fight sequence.
But we also did extensive motion capture. They converted one of the shooting stages at Pinewood Atlanta to a motion capture stage, and we used that for Thanos. So we did motion capture with Josh there, and we did motion capture with Danny doing stunts for Thanos. And we did motion capture with other stunties playing key cast, but also playing all the different flavors of army, both the goodies and the baddies who were going to participate in the fight.
We were instrumental in designing the sequences that were shot there. Myself and our animation supervisor Sidney Kombo-Kintombo, we blocked out all the different kinds of fighting moves that we would want. We were essentially capturing for Massive, our crowd simulation software. We were really careful to have each of the different flavors of the army, the Asgardians, the Ravages, the Wakandans, and even then there’s lots of different flavours of Wakandan soldier. Then we’ve got the Sakaarans and we’ve got the Chitauri, and we’ve got Thanos’ army.
Here we’d oversee fight vignettes. We’d say to the performers, ‘You three are Asgardians, and you three are Sakaarans.’ And we would give them foam weapons, and we talked through the different fighting styles, and we would capture vignettes of that fight so that we could populate the battle with that action.
They were shooting extensive coverage. They shot five weeks for a 30 minutes sequence. So that’s quite a big shooting ratio with multiple stages. There’d be three to four stages running through that five week period.
Portals, portals, everywhere
The portals were actually more of a plate-based sequence compared to our work overall, because it’s very character-driven. This is the pay-off for the audience. They have come to this film hoping that they’re going to see their heroes again, and that the snap at the end of Infinity War wasn’t actually as final as it looked. And this is that moment where they all turn up again. So we had already strong sense with the portal sequence that it could be fantastic, and we better not muck it up. It had to be great.
As the characters emerge from each portal, there’s a representation of the environment that the characters are coming from. So it’s Wakanda for T’Challa and Okoye and Shuri and all the mass Wakandan armies. And we’ve got New Asgard, which is the location in Norway where Thor’s been hanging out, playing Fortnite, letting himself go a little bit. We’ve got Contraxia. We’ve got Karmitage, where a bunch of sorcerers come from, Wong, and a bunch of others. We’ve got deep space where Ravages come from on their sky cycles. We’ve got Titan, because we left a bunch of our heroes, including Drax and Mantis and Quill and Spidey and Strange, on there.
Those environments were all digital because of the camera moves that were required. We would track the on-set camera and then render the environment with that same camera, and it just gets comp’d within the whole of the portal, and it all just marries up. They were created by our digital matte painting department who work in Clarisse.
Tony snaps his fingers
We got to do Tony’s snap, and the suit trying to protect him from the power of the gauntlet, and trying to rebuild itself but the suit getting damaged and Tony getting terribly damaged. Getting the bleeding edge tip was the trickiest part, where the suit is trying to reform.
That was actually the last thing we did on the show. It was about finding a balance between the effect on the suit and on his face, because we actually see it start to crawl up his face. It had to be strong enough that you could know that this was going to be fatal, that he has willingly taken on an act that he knows is going to be ultimately terminal to himself.
At the same time, we didn’t want to do anything that was going to distract from his performance, because we wanted the audience to be able to stay with him through that moment. It’s a really key Tony moment. So our initial pass was way over the top. We sent that through and we got the notes back to tone it down. And then we re-worked it through the simulations and through comp and lighting and multiple lighting passes. We sent it back again – it was getting quite late in the day now. And it was now just a little bit too ‘quiet’. Like, it didn’t look like there was enough energy being dealt to the suit and being dealt to Tony that was actually going to prove fatal to him, so we had to find some point in the middle.
And that all happened on the last day really. We had passes that we prepared that we could dial in selectively. And we had a really great comp’er on each of those two shots. We had support comp’ers working with him, doing utility stuff, so essentially three comp’ers per shot. And yeah, we really hit the deadline. But it was done, in time. Just.
A new kind of blip
Having done some of the key blips for Infinity War, we got to do the sequence we call the ‘mega blip’ in Endgame, when all Thanos’ army blip out. And we used a couple different solutions for that.
We had a blip ‘lite’ that we used for mid-ground characters that our FX department worked out. And then we also used Eddy, which is the volumetrics engine that runs as a plugin in NUKE that we’ve been trialling for a couple years now, that we used quite a lot on Infinity War and used extensively on Endgame to do background blips. Things like the Chitauri chariots and the sky vehicles that are falling and blipping out and the background character armies, they’re all actually blipped in Eddy. Eddy was built by Weta Digital people originally.
With Eddy, it’s like your card elements are suddenly volumetric. You can view them from any angle. You can tweak the simulation so you can change the scale, you can change the timing, you can build the library of elements and then pop them into a scene the way a compositor would have card elements of dust hits or something similar. But here they’re all volumetric. You can light them in Eddy so that they’re getting the same key light direction as the rest of your scene. You’re not trying to find a dust element that’s got the right light direction. You get the lighting correctly for free. And it renders really fast.
For the more hero Blips, those were done in Houdini. We’ve got the Leviathan that looks like it’s about to eat Rocket, and turns to dust. That was a fun one. And then of course there’s Thanos himself – that was a great shot to work on. That was our most hero blip I think that we’ve ever done. We developed our blip technology further just for that moment. We had a way of reviewing the timing of the blip with a graphic that flowed over the body that was generated from the same algorithms that would generate the blip itself – it was color-coded, so red when it was turning to ash and blue when it was drifting away. We’d review shots with these colod-coded graphics first with Dan DeLeeuw.
Another thing we did was add complexity to the air field that is making the blips drift away. Previously it would just interact with a static Thanos, but now we were able to actually adjust the wind field so that the collision objects were being updated dynamically as Thanos turns to ash and it drifts away.
That Thanos blip was one where we really explored with the filmmakers how to play that performance. We had motion capture from Josh Brolin, whose walk away echoes the walk that he does in the yurt, at the end of Infinity War. When he sits down and looks out over his field in Infinity War, in the shot that Digital Domain did, he’s happy. He’s satisfied. He’s achieved his goal.
But in this one when he sits down, he’s just really trying to come to terms with the fact that he has lost. And it’s tough for him because he’s so subtly arrogant throughout, that for him to change his thinking is a big deal. But there’s a subtlety of performance there that I think plays out on the stage where you see his dawning realization, and not acceptance, but he’s not fighting it. He knows better than anybody the finality of what’s happening to him. So he almost just ends up waiting for it to happen.
The impact on Tony Stark, and the MCU
When Tony staggers into the frame and leans up against the tree, that’s a digital prosthetic on his face. That was another thing that we really explored with the filmmakers – the level of damage there. I think that the great thing about doing it as a digital prosthetic was that they could do that in the post. Once they had the scene cut and they could see how it played out emotionally, then they could dial the level of damage to him rather than a practical prosthetic which would have been locked into the look of it right from the start. Instead we were able to really quite subtly explore the damage.
It had to be nasty enough to look like it was fatal. When we see him we have to know that he’s on his way out. So there’s blood oozing from his eye, from his nose and his mouth and there’s this big gash up his neck and onto his face and half his hair on his temple’s been singed off. We rendered this onto a matchmove of his head, but without matchmoving his facial performance because we were able to get away with the fact that his facial performance was quite muted in this scene.
It was a look we worked out with our own concept art. We had a team working in this little room, it was sort of under lock and key off the main Weta Digital production network working up digital concept art which we would securely deliver to a very select group of people at Marvel. And then we would get on the phone and discuss. We had some really, really gory passes that we had to dial back from. When it came to tweaking the look in the digital build, we had all that earlier concept art to refer to, to continue to explore the level of the look.
I’ve started to say, we blipped Spidey in the first film and we killed Tony in this one, so it’s a wonder that the fans are still talking to us…
Explore more of our in-depth Avengers: Endgame coverage during #endgameweek.
Great read!
Thanks so much, Noah.
this was really great!
Thank you!
Thanks so much for putting this together. First time on this site and really impressed. Lots of great insight!
So much hard work and they get lazy on the script.
So much hard work and they get lazy in the script.