Behind the visual effects of the battle in episode 207, with VFX supervisor Jason Smith.
Matters escalate quickly in the penultimate episode of season 2 of Amazon Prime Video’s The Lord of the Rings: The Rings of Power when the Orcs lay siege to the city of Eregion.
Amongst the fighting and assault on the city walls is the emergence of the 16-foot Hill-Troll, Damrod. Production visual effects supervisor put together a reel of scenes featuring actor Jonathan Banks as Mike Ehrmantraut from Breaking Bad for the Troll’s wry behaviour during the siege. On set, the Troll was sometimes represented by poles, foam parts and even an inflatable bag.
Then, DNEG (under visual effects supervisor Greg Butler) crafted the CG character. The studio was also responsible for the visual effects of the siege, which included environment work, some CG Orcs and horses, weaponry and destruction.
befores & afters caught up with Jason Smith to discuss how the Siege of Eregion was planned and executed, and how Damrod was brought to life.
b&a: For the siege, what were some of the visual planning aids that came in here, either boards or early previs or anything like that?
Jason Smith: We started building a part of the set, so we knew that we would have that. We were in V-cam and we were in previs, so we knew, okay, if we’ve got this little section of wall set, here’s what it’s going to look like, here’s where the trees are on the opposite side. We had an empty field that could play as the empty riverbed. The thing that actually was really critical, and I always try to do it earlier and earlier on every project, is just figuring out the mood and the look.
The main reason to do that was because we had all these orcs out there who can’t be in direct sun. And so, what that means is that there was a whole plot point to get to the point where the smoke from the siege is enough that it’s blotting out the sun enough that they can go out and they can fight. I think that when you put a whole battle like that, a whole epic battle scene like that, in overcast lighting, just imagine—it can look really bad.
So, what we found really early was that I had to be finding every excuse and every variation on overcast lighting that we could find. I worked on that with DNEG and with Greg Butler, and we did that with concepts with Einar Martinsen, who is a concept artist on the production side. We just did painting after painting after painting after painting, and then found, okay, some light breaking through the clouds here in the background can help, things like that.
b&a: As the battle progressed, did fire end up becoming that background thing that you could rely on?
Jason Smith: Absolutely. First we found that gray meant there was no color anywhere, but Eregion is designed with brass, bronze, copper—with these cupric metals that all have a verdigris, a green patina to them. They would pop greens, and then if you look on the forest, they would pop greens. That gives this kind of cool in the gray. And then when we added fires, that gave us kind of a warm counterpoint. You were absolutely on point that what we ended up doing was, in almost every shot, we would just be saying, can we add a little fire here? Can we emphasize the greens on the rooftops there? Because otherwise, it just became really painful.
We did also allow some direct sun in the background. We would do that kind of stuff where you’re getting a little bit of that alpine glow on the top of the mountain or on the cliff. It turned every shot into a little bit of an art direction project, which is, at the scale that we were at, it’s a lot. But I’m happy with the final result. Look, when you go into those challenges and you say, ‘Oh man, overcast? Really?’, and the story demands it, that’s what the story says. Sometimes it’s nice when you realize that actually that little constraint helped us to find a look that’s not in this show. Those constraints can actually be the thing that helps you to find a unique voice.
b&a: Where were you able to film this?
Jason Smith: We were lucky enough to have that wall set that was out on our backlot. Then for the field location there was a churned up muddy area to represent the riverbed and then a line of trees. The fact that we had those three things meant that generally we were out there I would say 95% shooting all the battle. There was, of course, some beautiful digital Orc work by DNEG.
Then for ultra wides we had crowd simulation work. I haven’t actually done those big, Massive-like crowd battle shots and being in charge of being ultimately responsible for the notes on the animation too. That’s a really interesting challenge. You have to sit there and watch each shot just many, many times, because you have to look at each part and see how everybody’s behaving. You have to ask yourself all the same questions you ask during an animation review, buT on each one-sixteenth of the frame. There were times where we would catch something where a character dies and he face plants into the ground, and if somebody looked there, it would be a comedic moment. You don’t want that. We would have to go and cross out some of those guys. I would send back lists of, ‘Let’s get rid of this guy, this guy, this guy, etc.’
b&a: By the way, as well as whatever digital Orcs DNEG was doing, the practical makeup effects for the Orcs was just exquisite.
Jason Smith: They’re unbelievable. I have to say, I feel really privileged as a makeup kid, which I was—growing up, I was going to be a makeup artist. I feel really privileged to be working with Barrie Gower, truly one of the greats of our era. His company, BGFX, him and his wife, Barrie and Sarah Gower, run this company. For our show, they produced over 1,500—and just let that number sink in, 1,500—full prosthetic, multi-piece, layered, designed makeups for Orcs on the show.
That means, somebody waking up at three in the morning, somebody sitting in the chair, multiple rubber pieces glued on, painted, etc etc, you’ve got an Orc. 1,500. That is a scale that I’ve never heard of before, and I think that that has to be recognized. And then, Barrie was really innovative in terms of being able to hit those numbers and be clever about it. He’s got silicone makeups, which we used last time as well, and the silicone makeups are more expensive and they’re heavier. There’s many reasons that they have a little bit more of a cost in every kind of way. What we used to use, let’s say back in the 2000s, was foam latex. On this project, he actually started doing some foam latex again and slip-on latex masks.
By doing all three of those options, and by the way, the 1,500 is only counting actual glued on makeups, so that’s amazing, but by doing some silicone, a lot of foam latex, and then some pullover masks, it just became a superpower. I mean, I don’t know how those guys pulled it off, but they really did a beautiful job. The beautiful thing about the foam is that you can mix and match. You can have a Zorro section, a piece of chin, cheeks, and you can have somebody sit in the chair and just decide, ‘I think maybe this suits you’, and then you can make a new character that nobody’s ever seen. It’s really a cool way to do it.
b&a: Since Barrie Gower and the team were doing so many, was there an attempt for the visual effects department to try and scan or photogrammetry all of these Orcs for extra shots?
Jason Smith: Yes, we couldn’t take all of them, obviously, because out of 1,500 applications, I don’t know if 500 of those are unique or something, but really, to fill out a battle, we only needed a few dozen to really do it in a way that nobody could see the duplication. We scanned them, and that became the basis for all of our digital works. I think that’s one of the things that I really love about this show is the inter-reliance between practical and visual effects in a way that is so constructive. It’s nuts. Barrie and I, I think, work very well together. Love working together. We’re both excited about creatures. We’re both excited about making the best thing, however we do that, and he’s open to visual effects when they’re needed. It’s just a really, really great situation.
b&a: At one point, the cavalry does arrive, and I thought some of the horse stuff there was fantastic. What was the mix of real horses and digital work there?
Jason Smith: Yeah, it’s intentionally confusing. We do that for a couple of reasons, but one of the reasons that we’ll do it that way, is to give ourselves better lighting reference. Let’s say you give us 20 horses and we’re going to make it appear to be 300, 200, 400, whatever multiple, what we want to do is, obviously, we usually want to fill out in front of the camera first. We want to fill out close to camera, the first row. But what we’ve tried to do on this project is try to keep some of those horses in our reserve to fill out the background and mid-ground, even the practical ones.
We’ll have a big group right by camera, and then kind of a sparse spattering on the sides and in the background there are real horses with riders, and those are in the right light. They’re running on the ground, they’re going through the cloud shadows, they’re going underneath the tree. They capture all of the nuances of filling out the rest of that world that we’re going to have to fill out with our crowd.
So, we don’t just have reference at the front of the crowd. The strategy that we use there is we really try to fill out the front of the camera, but then really sparsely populate them throughout the scene. I think, again, it’s pretty difficult in some ways, in some shots, to tell where we’ve added them. We’ve tried to increase danger. We’ve added more horses where it looks a little bit too safe. There’s a lot of work in the shots, actually. A lot of beautiful work.
b&a: And then of course there’s the Troll, which was just some of the most fantastic creature animation I’ve seen on a streaming show in recent times, I have to tell you.
Jason Smith: Yeah, he’s fun. Thank you.
b&a: Take me back to the beginning of that, tell me about the kinds of ways you wanted to bring that creature to life.
Jason Smith: Initially, I think that there were some rounds where it was different character. When it finally landed on this guy in the script, I noticed that the showrunners wrote him a little bit John Wayne-ish. They wrote him as a man of few words, but he was very intimidating, and he kind of had this mercenary feel to him. The more that we were reading the scripts and planning for things, I started getting visions of Mike Ehrmantraut from Breaking Bad. I went as far as to do a super cut of Jonathan Banks. I love him as an actor. I think he’s so great, and I hope I get to work with him someday.
I did a super cut of some of his scenes from that show with little key emotional moments that would feed into what our guy was going to be doing and presented that to the showrunners just to say, ‘Hey, are you guys good with this as an attitude for this guy? Because it gives me and the animators and the supervisor over at DNEG something to hinge on.’ We’re not making Mike, obviously, we’re not doing that exactly, but it gives us a little bit of a common fulcrum that we can help each other find the character. They loved it. They loved the idea and said, ‘Go for it.’
And so, we used that reference more as inspiration for mood and how he moves and how much he would move his mouth and stuff like that. Nothing was able to be used directly as reference, but we did learn a lot. When Mike talks, a lot of times, his eyes aren’t really doing a whole lot. His eyes get pretty deadpan, and the mouth is moving the minimum amount to threaten Walter.
At DNEG, our animation supervisor, Bartek Kujbida, he’s a physical guy, and we were doing our first few animation shots and we were iterating a lot on getting the walk cycle right. Finally, he started doing some of the shots as recorded takes, kind of getting his walk in there, slowing it down by just a little bit, and there was a whole exploration to find this, but we ended up with a lot of Bartek’s performances driving the posture and the steps and things like that.
I thought it was very appropriate for our guy. I even ended up doing some of the close-up stuff for episode seven. There’s footage of me dying as the Troll and doing some other of the laughter and stuff like that. Those were used pretty tightly, I think, as reference for those shots. We really followed the different beats of that, because I was very specific. When he’s dying, it’s not what you’d expect. He is looking at the situation, getting a kick out of it, kind of laughing and accepting it, and then dying.
b&a: I love that during the sequence he’s smiling wryly, and then of course, smiling and laughing when he is dying. It’s brilliant, so unexpected.
Jason Smith: That’s what I love about it. I was pitching it on set and I remember everybody that was looking at me while I’m pitching it. You can see in their eyes they were thinking, ‘What?’ I have to really give credit to Lindsey Weber and the showrunners because it was a resounding ‘Yes’. Even though they’re like, ‘We don’t quite know what you’re exactly doing, but we can see that you’re onto something.’
b&a: What kinds of things did you have on set, if anything, to represent him? Was it poles or a stuffy of some kind? Or a person on stilts?
Jason Smith: We had some cool ones actually. First of all, let me say that we used the fact, which is known, that the trolls were made by Morgoth in mockery of Ents. That was my argument that I made to say that we should make him pretty tall. He is around 16 feet tall. We looked into stilts, but the terrain was so rough, we backed out on that. We did end up having poles, so we did the same thing we would do on a Transformers film. We would have that pole and we could walk around and make eye lines with it. And then, we had specific rigs for things like the Orcs getting lifted up. We actually did shoot that in a stage with a special rig to lift them up and have them land against his body when he’s using the Orcs as the ‘inhuman’ shield.
The one I wanted to mention to you, though, is when he is dying on the ground, we had a big inflatable bag. It was fantastic, because you don’t think about the fact that when he’s laying on the ground, a thing that size, a guy that size, his torso from back to front is about six feet. He’s so massive that we had to have a big inflatable thing. And so, the people knew, ‘I have to look over this character right now.’ We had little hand pieces so that we knew when Arondir is stabbing the hand, we knew where to do that kind of stuff.
On the night of the shoot, we went out with rolls of tape—many, many rolls of tape—and we did an art project when he was dying on the ground, like a full on masking tape sculpture of the outline of the Troll so that people knew where to avoid. I’ve learned over the years that if you do a chalk outline on the ground, people will just immediately walk through it. But having that chalk outline on the ground, we were then able to stack in obstruction objects that would stop people from walking through it, and it ended up working out really well.
We also had some foam legs that we had used that could be stabbed when the Troll is finally getting killed and all the wounds are adding up and he’s dying. We used those legs when we’re stomping people too. That was fun. It was good times.
b&a: I really felt there was some super cool moments with him. One time, he grabs a horse as it’s galloping and he throws it. Another time, as you mentioned he picks up two Orcs and uses them to prevent arrows hitting him. How did those moments come into the show, whether they were in the script, or in the previs or just moments that you thought of while shooting?
Jason Smith: Those in particular were moments that I thought of with the blessing of our showrunners. I had a good amount of freedom in certain sections to figure out what we could do that was cool. He has this one shot when he’s first showing up where he walks by camera and he’s got his club and it’s flaming, and he slams it into a guy and throws him into the distance and the camera pans with him. We called that, in pre-production, his guitar solo. We’ve been introducing him. He’s been coming out slowly, and then now he’s walking across the battlefield just full on having a blast killing everybody.
So, for those moments, we were able to design those ourselves and think of all the ways we could make them better. Joe Henderson from The Third Floor did some fantastic contributions to those gags. The inhuman shield gag with the Orcs, that was one that I worked up when it became clear to me that the story was really hinging on him hurting Orcs and him doing it in ways that is so insulting that the Orcs are starting to get upset about it. I thought that that was such a great way to just add insult to injury.
b&a: There’s that great part where Arondir jumps on the Troll. How did you stage that, seeing as he would be interacting so closely with the CG character?
Jason Smith: That was a real challenge. Of course, the actor Ismael Cruz Córdova did an amazing job. He’s always so gung-ho, willing to get out there on the wires and go for it. As far as the interactions with the Troll went, we tried to frame the shots a certain way. You might’ve noticed this. We tried to frame the shots so that that interaction was secondary to the action of the shots. We see him running towards the Troll, but we don’t really see him land on it until we get to the side view. It’s only that one shot where we see some contact. And then, he’s already jumping off and flipping over the Troll at the end of that shot. We really tried to minimize with camera, and we tried to minimize with how long the contact was, how much we had to actually deal with that interaction. I think that was the only way that we were able to get through it.
And then in post, every one of those moments where he’s jumping, where he’s flying off things and landing huge distances where it’s kind of fantastical—it’s those Legolas moments, if you will, excuse the saying—every single one of those, we ended up basically lining up to some reference of people doing parkour. It’s a funny thing to do, but you can find people jumping off a wall from that height and landing on the ground. Those videos will show you what they do and how long it takes to fall. You can basically find something real, and then benefit from it. That was really the thing that I thought took us from, ‘Oh boy, this looks a little bit CG’ to something that’s like, ‘Okay, I can believe the physics now.’ It’s still fantastical, of course, don’t get me wrong, but I want all the audience members out there to know it’s all based on actual people falling through the air.
