A look behind the scenes at the extensive VFX work in Denis Villeneuve’s film.
At one stage during filming of Dune at Origo Film Studios in Budapest, visual effects supervisor Paul Lambert, having implemented a raft of sand-colored screens–rather than blue or greenscreens–on the backlot there, was asked by the line producer on set whether this meant everything would now need to be rotoscoped.
As they walked Lambert took a picture of the backlot covered in ‘sandscreen’ with his iPhone, brought this into a standard app and made a negative (inverted) image of the scene. “That’s when he realized that the entire thing was a bluescreen,” says Lambert.
While of course the VFX vendors on the film, led by DNEG (other visual effects contributors were Wylie Co, Rodeo FX, with MPC handling previs), certainly tackled sandscreen comps with roto where necessary, the decision to use those screens was indicative of the overall ‘grounded’ approach to plate capture and later digital effects work that Lambert and the VFX team took to Denis Villeneuve’s take on the Frank Herbert novel.
Lambert, a VFX Oscar-winner for Blade Runner 2049 and First Man, shares more on the making of the film, including crafting multiple worlds, massive sandworms, filming and creating flying craft, building protective shields and more, in this in-depth interview with befores & afters, alongside DNEG visual effects supervisors Tristan Myles and Brian Connor.
The overall approach to the VFX of Dune
b&a: Paul, it would be great to sum up your thoughts going into this huge project in terms of what you wanted to achieve, what Denis wanted to achieve and what you thought were the hardest things going in.
Paul Lambert (overall visual effects supervisor): From the outset, the goal for the visual effects was to try and keep everything as grounded and as photoreal as possible. We weren’t going to have any virtual cameras which could only be done in CG. We wanted to embrace all of the natural environments which we were going to visit. This was also one of the most collaborative projects I’ve been on, in that the DOP Greig Fraser was fantastic, Patrice Vermette the production designer was fantastic, everyone was. We all got on really well and had a similar goal to try and come up with the best procedures and the best techniques on set to be able to give us the best basis for our visual effects work.
We spent a good six months in pre-production, coming up with various schemes and various different ways in which we were going to approach the work. I knew from the outset that Denis is not a big greenscreen or bluescreen fan. If anything, it kind of depresses him to see anything like that. So we had to be creative to come up with different ways in which we could get everything we needed for visual effects knowing that we were going to be extending backgrounds and creating these amazing worlds.
For example, for the interiors of the ornithopters, traditionally, you would shoot these inside of a studio up against a greenscreen or a bluescreen. Now, the one thing which we weren’t going to do was to try to recreate daylight inside a studio, especially for when we’re trying to recreate the desert environment. I had done First Man with Tristan, and Greig Fraser had shot the first season of The Mandalorian, so obviously we talked about LED screens for potential sequences. But Greig knew that there was no way we could get that harsh, arid, hot, bright environment with an LED screen. So, that idea was abandoned pretty early on. Well, we did actually do one sequence with LED screens set in space above Arrakis which at one point was going to open the movie, but that never made the cut.
We knew that whenever we wanted something to be outside, we were going to shoot it outside. So what that meant, for example, for the interior ornithopter work, we went out and scouted and found the highest hill in Budapest, and we built our gimbal on top of this hill so we could get a nice flat horizon, and then around the gimbal we constructed a 25 foot high sand-colored 360 degree ramp. What we called it was the ‘dog collar’, which circled the entire gimbal, the idea being that when it was a bright, sunny day, the sun would bounce off the dog collar into the ornithopter giving us the ideal lighting. Now, the ornithopter was just like a glass bowl, so it was getting light from every direction. That’s how we shot it, with the glass in, we had the actors inside the ornithopter and we had Greig on the camera. Say Greig had focused on Paul (Timothée Chalamet) or Leto (Oscar Issac) and everything was out of focus in the background, it almost looked as if you were over the desert.
But then for the overall sequence, which included the flying, what we did was we had shot hours of footage flying over the UAE in a helicopter with six cameras mounted in an array to get super massive resolution plates. What the artists did back at DNEG was, rather than doing a full extraction on those plates, because we had such subtle reflections, and we had reflections of reflections, instead of trying to extract that and produce a composite–you would basically be trying to rebuild those reflections and just starting again–so it was more of an overall blend between the elements.
What that gave you was, you kept all the reflections, you kept all the subtleties which we shot, and it made for a far more believable composite. The general philosophy of the shoot for the movie was in that particular vein. For example, another one is when the Sardaukar are coming down in this massive structure and they’re trying to attack the Fremen. It’s a sequence which Brian had supervised.
Again, with this structure, which we called the Nexus, they’re actually coming in through a daylight opening in the structure, and Patrice spent forever trying to find a place in which we could do this, where we could have enough space, with natural sunlight coming through. But we couldn’t find anywhere suitable. So in the end, we took two of the studios in Budapest, and we connected the two studios with cloth along the top, basically made a massive box connecting the two studios, cut out a hole at the top, and we had sunlight pouring into this area.
Now, we could only shoot for two hours in the day, because obviously, with the changing position of the sun, the light would be pushed up against the wall rather than on the floor where we needed it. But being able to get the ideally lit plates, which we could then extend out, was instrumental to the believability of the scene. I was always saying to Greig, ‘Look, Greig, we’re in a position in visual effects that you can give me any footage and I can put any background behind it. I could put Disneyland behind you right now. But if the foreground doesn’t correlate to what the intent is in the background, there’s not much we can do.’
b&a: I want to ask Brian and Tristan about that from DNEG’s perspective. Brian, what is that like, getting as much real as possible, say for those environments that Paul mentioned?
Brian Connor (visual effects supervisor, DNEG): Well, for me, it’s a blessing, and it’s almost a luxury to have all of these environments and set pieces and machines built for real, and be outside, especially, but inside as well. I’m definitely not an actor by any stretch, but I don’t have to pretend I’m there on Arrakis or at the Space Port or just in one of the massive, almost cavernous sets. You’re definitely there, you’re immersed. The lighting is all there. The visual reference that I need to add more ornithopters, it’s all in camera. All of the answers for things we need to add or to extend things, which is a lot of what we did, it was already on a massive set.
It just makes our jobs easier, because we’re grounded in reality. Of course we’re doing all of the data wrangling to add to that. So later in dailies with the laser pointer, we can point to the real thing, it makes it easier. Usually we have to study the image and the sequence and try to figure things out. It makes it that much easier to add that higher level of photorealism to the shot, in general.
b&a: What about you, Tristan? What’s your perspective on that as well?
Tristan Myles (visual effects supervisor, DNEG): The answers are always in the plate. It’s been shot that way, so you have perfect reference to work with. And as Paul has already mentioned, with the use of the sandscreen, you’ve got the tone and the luminance in the plate for the background already, a little nudge here or there and you’ve got quite a decent edge around the guys to work with. So you don’t end up with these cut-out characters against backgrounds, which you sometimes get with greenscreen shots. I quite like that way of working. Plate is sacrosanct, keep as much as you can. If you do any work, it’s got to sit in with that plate and that lighting.
Paul Lambert: We were never in a position on set where the idea was we would ‘fix it in post’. That mentality never came up. Everybody was on the same page of trying to do the best they could on set. Yes, obviously, things did get through. Sometimes when you put up a greenscreen or a bluescreen, certain parts of the crew switch off, because it is going into post. So, certain things don’t get cleaned up because it’s going to post anyway, and we never had that.
Just to what Tristan was saying, the sandscreens were something which we came up with in pre production. We had the backlot in Budapest, just outside the studios, where basically we covered this massive area. Rather than a traditional blue or green, we actually covered it in sand color. So that was our screen, the idea being that because this was generally going to be Arrakis or it was going to be the desert, it was going to be that color. We were about a month into this, building all these screens, and we had big 20 by 20 screens on the backlot and I was walking with one of the producers, and he said, ‘Paul, how are you going to do this? This is all going to be roto…’.
What I did then was I took a picture on my iPhone of the set, and then I brought it into a standard app and I made a negative of it. And then I showed him, and that’s when he realized that the entire thing was a bluescreen. It’s just a little trick.
Even though the intent was to try and do it with an inversion, I know that some compositors actually swapped channels instead, but the idea was that as long as it’s a constant color, you can pull the key from that. The fact that it was sand colored was very relevant to Dune, obviously, because we had a sand colored world.
More on the sandscreens
b&a: Two questions from that, first for Paul. How did you come up with the right sand color? And then I’ll ask Tristan and Brian about the extraction.
Paul Lambert: Swatches, painting. Basically, we went through a process of doing various tests, having an idea what the best blue is to then invert that to get the color, and then having that paint made up. We put it onto cloth and we put it onto wood, as well, and then we also put it onto metal for some of the scenes. So it was extensively tested. What I did find though is that during the tests, there were certain times of day where, because of where the sun was, it actually was losing a bit of color. So it was always a bit problematic when we were going to be shooting in a particular direction at a particular time of day, and I would be stressing a little bit, but it worked out fine.
b&a: And guys, now is your chance to tell Paul it was all just roto…
Tristan Myles: (laughs) In some cases it was, it was definitely part of the template. So we worked in Nuke to build templates for various shots and different set-ups that we had for the sandscreen. Artists did use roto where it was appropriate, but there was a fair amount of multiplication and addition going on to get those edges working correctly with the correct background, as Paul was saying, based on the luminance of that plate, tying them together and boosting as needed.
Brian Connor: I was very impressed and surprised at how well the edges came out. I mean, I can order up roto, and we can take care of the core in different ways with different keys. But in terms of getting the edges and making that fall-off believable, and into our extension of the Space Port, for example, it worked a charm. It was very impressive.
Paul Lambert: I should add that we had brown dust and we had brown sand blowing up against those brown screens. It took a certain skill to be able to blend that. Not fully extract it but do blends and that type of thing. And I should say, Ian, that all three of us come from a compositing background so getting a shot to final needed to get through a barrage of 2d scrutiny….
Practical sand, and other effects
b&a: Was there any kind of particular on-set special effects approach for sand blowing and movement?
Paul Lambert: We were forever blowing sand around. The SFX team actually used 18 tons of dust/sand during the production. For the space port, where Paul and family first get to Arrakis, we were blowing sand everywhere. And that, up against the sand-screen, was actually quite a challenge to get that all to sit. And also, the sequence at the very beginning of the movie, where Chani is telling the story of when the Harkonnens are leaving, was literally just orange/brown plates with swirls of sand blowing everywhere. It wasn’t just soldiers walking by, it was a full environment we had to do, so the plates had to be skillfully composited together to put the Harkonnen sandcrawlers into this swirling environment behind the characters.
For the ornithopters, production had two 12 tonne ornithopters made, and these were both shipped out to Budapest and to Jordan, where we also shot, and these things were so big that it actually required an Antonov cargo plane to transport these things. Somebody told me the price of this, and I thought, ‘Oh my goodness, that’s like a million visual effect shots right there!’ But, having these things, which Tristan and Brian alluded to before, having these things on set, even though we were going to make CG versions of them, is the perfect reference, because it’s real. We blew dust and sand around these things. We even picked these machines up with a crane for the take-off and the landing, and we would add CG wings to it, or sometimes replace the whole thing if it needed to go higher, but we always had constant dust flowing.
When Paul and Jessica (Rebecca Ferguson) make that crash landing, Gerd Nefzer, the special effects supervisor, built a gimbal that could rotate 360 degrees. We had Paul and Jessica in there for the more simpler shots, but this particular gimbal was completely enclosed in this makeshift box made up of 5 20’ x 20’ blackscreens which Gerd filled with wind machines that swirled the added dust and sand just like a sandstorm.
For the crash, we had two stunties put into this thing for when it was really rotating around. The funniest image was seeing the first AD come out of this swirling box, and he’d be completely orange. He had Ronald McDonald’s hair, right? Just completely orange. And then there were times when we were out in Jordan as well, and we were kicking up dust, and a number of the crew were just covered in this orange color, which took a few showers to actually get rid of.
Then there was ‘shaking’ sand for the sandworms. Gerd came up with this brilliant idea of a vibrating plate. Denis wanted to see sand vibrating to give us a clue that a worm was coming, and Gerd built this massive 12 by 12 wide plate and buried it in the sand. He had a dial which he could dial it up and down, and what would happen is that there would be a certain point where the sand basically looked like it turned into rippling water, and if you stood on this area of sand, or you were on top of it, you actually started to sink into it. You see the effect clearly when Paul and Gurney are running away from the spice crawler which is about to be swallowed by the worm. What we did in post to help extend this 12 by 12 plate was we shot extensive photography of the effect so we could then replicate it to make it feel it covered a much larger area.
To get additional dust swirls and interactivity, we did a lot of helicopter work. We did helicopter to helicopter shooting, where we replaced the photographed helicopter with the ornithopter. We also used helicopters for landing and taking off, and sometimes we would use those elements and replace the helicopter with the ornithopter, sometimes we would just use it for reference. DNEG did an amazing shot of when Duncan (Jason Momoa) comes flying in to meet Kynes (Sharon Duncan-Brewster). We’d shot some reference of a helicopter which landed in this particular area, and it wasn’t quite in the area that we wanted, so Brian and his team then recreated all of those swirls in CG to put it in the correct place.
When you have something to match to, you just A/B it, you get something far more believable, because if we didn’t have that reference, it’s so easy to go down various paths like ‘Well, try this, try that, try this,’ but if you’ve got something to match to, it keeps everybody on the same page and it stops questions about realism.
Brian Connor: Paul wasn’t precious to keep things clear just to be able to see the CG vehicles. It’s like, ‘When this thing comes in and it’s massive and it has these huge wings, it’s going to displace a lot of dust, and at some point you’re not going to be able to see much.’ And that’s what I love about Paul and I think Denis too. It’s not a showy thing, it’s like, ‘Is that what would happen? Does it feel real?’
Paul Lambert: Yeah, that actually touches on something. You have artists who have spent a long time putting these incredible assets together, and then suddenly you have a shot where you barely see it because it’s covered in swirling dust. It’s a tricky thing to try and pass down through, but sometimes you have to do it.
Were there miniatures?
b&a: Just on the mention of full mock ups of the ornithopters and other things, Paul, did you consider miniatures at all in this film? Are there any miniatures?
Paul Lambert: So, just to put the movie into perspective. The movie was going to finish at a slightly different point at the very beginning of production, but there was a certain budget to hit, and to Denis’ credit, rather than keeping the point where the movie was going to finish, he made it finish earlier, which meant that by doing that, he got the money to where it was supposed to be, but he didn’t compromise any of the departments. We still had the money for the physical ornithopters and the physical sets and appropriate vfx budget, we still had that. So having that clear thing, being able to build the ornithopter for example, it just made it a great basis for all of our visual effects.
So there are some shots where you get to see, there was one particular shot where you get to see the Harkonnen ship explode, and there was extensive talk about wanting to build that on quite a big scale and get SFX to blow it up, because it was going to be made of all these little tiny tile pieces, so it would actually be a very beautiful explosion. But the trouble with that is that usually you need to augment the miniature work with CG, which means building that asset and replicating it, which basically doubles the cost.
There was talk of doing some big scale mock ups of some of the sets as well so we could use that to extend it, but again, it all came down to a budgetary thing, and I was really confident that the kind of designs which Patrice had made, they’re huge scale but they’re not overly intricate, itty-bitty kind of scale. So I was really confident that going down a CG path for this particular route was the correct way. But we did talk about it extensively.
‘You can never have too much sand’: the sand worms
b&a: You mentioned the sand worms. Could you talk about the design and what Denis and you wanted to achieve, but also how you felt you would communicate the scale of them on screen?
Paul Lambert: One thing I should just preface this with is that Denis and Patrice had been working on concepts for the entire movie for a good year prior to pre-production including working with freelancers and artists at Weta Workshop and Rodeo. Concept work is usually a springboard for different ideas. But not this one. Denis was super, super, super happy with the concept work, and he had already concepted the look of the worm. He had concepted how Arrakeen looked, he had concepted these amazing machines, these gigantic internal structures. And basically, on set, we built these structures, we built these spaceships, we built these virtual environments based on these concepts.
The overall look of the worm had been designed. Obviously it didn’t move, because these were just still frame concepts so the artists built the worm up from these huge rigid plates with soft membrane flesh in between. The idea is that when they turn, you get an accordion kind of feel. It’s not very agile. He wanted a very simple animation of this.
The way in which it goes through the sand, that’s where our real work came in, because he plays the worms very much like Jaws. You get a sense of them throughout the movie, and the way in which you get the sense is by seeing the dunes collapse and rise, and by seeing explosions happening on a dune, and I knew that that work was going to be the trickiest for us in post. And when I joined the production, and there was a good six months of pre-production before shooting, I actually instigated doing early FX sim tests, knowing that this was going to be a big part of it.
Now, as you know, Ian, the best visual effects are always based on a piece of reference which you can find, or if you can get something partially in camera. So we scoured the internet trying to find references of sand being moved, especially this amount of sand. And I came up with this idea of, ‘Look, we can ask special effects if we can bury some explosives in the desert and blow it up and take pictures and film it for reference.’ It was just then that I was reminded that we were in the Middle East and that was probably not the best thing to do. So I didn’t get that footage, which means that Tristan and the artists up in Vancouver did iteration after iteration after iteration. Tristan had the sand and Brian had the water in Caladan, where you have these massive Atreides ships rise out of the sea, and it was the same idea, a massive displacement of particles. It was a lot of effects simulation work where you didn’t quite have the reference which you needed.
And that always means that to try and move that amount of particles, you have to cut corners, because we don’t have the computational power to do it per grain, and that’s where things can go awry slightly, where things go a little bit too fast, or the scale doesn’t work in the right way, and you see it, something’s wrong. And these things take days to get through, so I knew it was going to be a slow burn to actually get this. In the end, the guys were doing quick turnarounds on shots, because they spent a good part of a year trying to get this actually developed. And what came out was something beautiful, because you almost get this sense that as the worms are far, are flowing through this arid environment, the worms almost feel like whales breaching water. You get all of these beautiful ripples as well as violent explosions coming from the dunes.
And the worm itself, its mouth is based on a whale’s baleen, so basically, a whale will sift through the water catching all the krill in its baleen, and the worm’s mouth is very similar. The idea is that it goes through the desert and it’s getting all the nutrients and then it’s digesting those into the spice, into the desert. So there was always a correlation to water which I found very beautiful, and it was a fantastic idea to have that to be part of Dune, this dry, arid land, you know?
b&a: From DNEG’s point of view, Brian and Tristan, what were the big challenges there with simulating sand but also moving such a big body through that kind of environment?
Tristan Myles: As it turns out, you can never have too much sand, which we found out fairly early on. It sounds obvious to say, but it really is a big indicator of scale. If those grains were too big, suddenly the worm looks small. And so does the environment. So we had to come up with a way of being able to represent that motion of sand as the worm moves through it and how it leaves, as Paul mentioned, troughs behind it, as it moves through the sand.
The FX guys came up with a system where you could do some simulations to get a rough idea of how it looks and then the additional sand would be simulated and filled in later, and you’d have simulations coming off those simulations and a bit of a cascade effect. There was a lot of hair pulling at the beginning, but it was definitely worth it. As Paul said, it was going to be a slow burn, we had to do this work ahead of time, before we got into production work, and it definitely paid off, I think. We had a great FX team that ended up producing some really amazing looking FX work in Dune.
b&a: Was it mostly Houdini, just in terms of the sim work, or is there any other proprietary stuff that DNEG was doing for that?
Tristan Myles: It was all Houdini, but the great thing about Houdini is it is completely customizable, so the guys came up with and wrote quite a few tools to work within their subset of tools to work with this volume of sand. With all those tools in place, it was still super heavy, and required huge amounts of disk space. I was always getting emails about clearing up disk space. Huge simulations. But it’s needed, right? There’s no way around it. There is a huge volume of sand that this creature is pushing around. So it is what it is. And it paid off in the end.
b&a: And Brian, what about from your point of view?
Brian Connor: Well, thankfully Tristan and the DNEG Vancouver team handled all of the worm and all of the sand interaction. We were tasked with creating a large sandstorm, where Jessica and Paul escape, which causes their demise and they crash. But the whole outside of the sandstorm, again, early on, Paul came to us with some wonderful National Geographic massive sandstorms. It was just incredible footage. And it’s surprising on how something that big, it doesn’t really move that much, and so there’s all sorts of different characteristics outside of it, and when you’re inside of it, and then we had to take a little bit of creative licence when they’re getting sand blasted, literally, their ornithopter. But it was all based on real footage, and Paul’s pretty much most of the time, ‘Just do that, just match that.’
Paul Lambert: ‘Keep it simple, man.’
Brian Connor: Obviously those kinds of things are easier said than done. When you have so much particulate and these FX simulations are so huge, that you have to distribute it on your servers and they create a lot of data. You know, it’s a somewhat painful endeavor, but it’s very gratifying once you get there and you start to crack that creative issue, if you will.
It was the same thing with the water and the ocean and displacing millions of gallons of water, in terms of making it look like it’s something huge. It’s hard doing all that when you don’t really have a scale of reference. Paul actually made us put a guy in a boat next to the rising Atreides ship in each render so we would review the shot, we always had that scale for reference, and it very quickly told us, ‘Okay, wait, no, those drops are huge, and those ships are huge, so it’s going too fast.’ It’s those kinds of things, where you have good examples and good direction that Paul gives us, I have to say, it really helps us focus and get there faster than we would have.
Shields up
b&a: I love the training shields and other shields — can you talk about the thinking behind the look and feel of the protective shields?
Paul Lambert: There were a handful of effects which I knew that we were going to have to do super-early tests on to see if what Denis liked could inform us prior to the shoot. And the shields, the hologram, the sandscreens were all concepted during pre-production. I had a couple of artists onset with me and we just did test after test. For how the shields came up, we got to a good place really quickly. Basically we had taken a clip from The Seven Samurai and we were just playing around with it, trying different effects filters on it just to play with it. And what we stumbled upon, because this kind of thing is always trial and error, was that if we blended past and future frames, it actually produced a very pleasing effect. It gave us the chaotic feeling of all the movement and all the fighting.
We did this in a very simplistic way and we showed Denis and he loved it. He loved the idea of what felt like a shimmer, but based on past and future frames. The one thing I did want to do with it, though, was, rather than have it so procedural, I wanted the artist to have an input into it, and what that was was basically painting frame by frame. So painting out some of the effect or painting it back in. That was a very conscious decision, because I wanted it to feel more analogue than it did digital. And that overall look was established. And that was improved upon.
It was only once we got to the edit, where we had to add color, because what was happening is that when you had Gurney (Josh Brolin) and Paul fighting, and they were getting really intense, it became a bit of a mess. It wasn’t conveying the idea of the hit and it bounces off and the slow penetration. So that’s where the blue–when it bounces off, it’s blue and when it penetrates, it’s red. And it’s only when it’s a hit which hasn’t been deflected by the opponent. So basically, if two swords came together, it wouldn’t shimmer. It wouldn’t have the effect. But if the blade had caught somebody unaware, then that’s where it would come up. And then Tristan and the artists up in Vancouver, for the attack on Arrakeen, when you see these snowflake bombs coming through and penetrating the shields of the ship, it’s a similar idea. It’s the same Holtzman effect, or personal shield, but on a much bigger scale and trying to keep the same visual of the blue and the red, basically.
Tristan Myles: We had to have the right type of crew to do the work, because it’s very visually based.
Paul Lambert: It’s so frame-by-frame.
Tristan Myles: And the patience of a saint as well, because of the painstaking process. But I think this attention to detail and artistry pays off. The effect in the end looks pretty simplistic, but there’s a lot of thought that went into where those strikes are, what the timings are, how the plate moves and responds, and what the color tint ends up being for the type of strike, red or blue. And also for the ships, as Paul said, for the attack on the space port towards the end of the movie, we had to redesign that look in 3D but get the same effect that you had from the personal shields, the Holtzman shield effect. Which again, we used the same team to do paint-ups but on a larger area, with some bigger maps and things to try and get that effect across, the shimmering effect of the shields being struck or when it’s being penetrated by these bombs.
There is one shot which was particularly tricky, but it looks good and it paid off in the end. The bomb goes through, the ship explodes, the explosion gets partially contained by the shield and then blasts out through the shield afterwards as the shield collapses. I think the guys did a good job in trying to portray that idea that the ship was trying to contain it before it fell apart.
Going into battle
b&a: For the battle sequences, was there a very general approach you had to shooting them, enhancing the environments and dealing with things like explosions?
Paul Lambert: There were two approaches, in that for the attack on the space port, we had actors and stunties running along the backlot in Budapest, and Gerd would produce these fireballs, these timed fireballs, which would give us the interactive light which let the guys match the CG explosions to. And we took some reference explosions too, but it was done in CG in the end. Again, keeping with the philosophy of the shoot, trying to get the best lighting, so that was for the attack on the space port.
But then we had an all CG extravaganza. Paul has a vision, it’s a vision into the future where you get to see him as leader of the Fremen, and it’s a good few years in the future, and they also have different suits and they’re fighting the Sardukar. And this was the first time that Denis had ever done mocap. So basically, we spent three days with 13 stunties, during COVID, so they all had masks on, and we could only do five minute sections because everybody was out of breath because they had these masks on.
It’s this really bizarre footage, seeing these guys exerting themselves and they’ve all got masks on. The actual shot had crashing worms in the background, we replicated the mocap and ended up having hundreds and hundreds of fighters in there, and at the end of the shot, we revealed Paul, which Tristan can talk more about.
Tristan Myles: It was Denis’s first time using mocap, my first time as well. Luckily we had Robyn Luckham on the team who’s done that type of work before, the animation supervisor. He talked us all through the process and came up with the set up, working with the action beats that the stunt choreographer created, for the different actions and how it needed to be captured so we could make it work as this one shot, the oner, as we call it. And yeah, as Paul said, there’s a lot of stuff going on, a lot of fighting and a lot of sections that the animation team needed to bring together to create this feeling of an intense fight.
There were Fremen coming out of the ground, sand flying everywhere. You had sand worms coming up in the distance, there was an explosion coming up in the foreground. Huge number of people running around. It’s two armies fighting. Paul is in the foreground, he runs along, the camera kind of follows along with him. His action is he fights these guys and in the end he pops his mask open and reveals that it’s Paul Atreides, which we had to work with. An element shoot which was done with Paul in a replica mask, basically, a section which would emulate the light as if it were a bright desert day, which is what it was in this sequence when the visor pops open. So we then had to track that and put it inside this helmet and make it sit with the plate.
The frame length should be burnt into my brain, I can’t remember the exact number, but it’s incredibly long, yeah. It was originally going to be a slow-mo shot. So it would have been longer. Luckily, it went to real time. A huge amount of work. More big simulations. Again, the dust team, the sand team pretty much stayed on sand the whole length of the project, looking longingly at the team working on the explosions, and at the same time the guys working on the explosions were looking longingly at the sand guys. Huge, huge set ups, either way you look at it, and explosions had to be, as Paul requested, had to be looked at before the shoot began so we had some realistic looking explosions to combine with the onset ones which we had to time with the onset lighting and SFX explosions as well.
Going blue
b&a: I want to ask about the eyes, the blue eyes, because it’s something I remember from the first film so well. What was the approach here?
Paul Lambert: Denis was adamant that he didn’t want crazy glowing eyes, which took you out of the movie. He didn’t want to be in an edit where you cut to somebody with blue eyes and they’re shining really bright, and then basically you’re not really hearing what the actor is saying but you’re just looking at what the heck is going on. So it’s a very subtle balance. And again, that was something that we did on set prior to make sure, to find out, ‘All right, do we use contact lenses, for example?’ And we played with that idea, but it being in such a dusty, sandy environment, and some actors can’t work with contacts, it could have been a real slowdown to the shoot. We did a test where we cut up some onions and we rubbed them under our eyes to try and get some red going on from the onion which we then tinted blue. And we tested that, Denis wasn’t interested in that.
We were just playing with ideas. And then what we settled on was we took various videos of the crew and we picked different colored eyes, and the idea wasn’t to try and make this one blue eye which was consistent for everybody. No, we would embrace whatever colored eye was the base, so if they had green eyes, blue eyes, it all got a very similar treatment, in that we did maps for the sclera, for the iris and for the pupil, and again, it was an artistic balance to make it so that you could read the blue in the eye but it wasn’t overpowering.
And it did change ever so subtly from sequence to sequence due to how the scene was lit. We also had shots where we had to do it behind flares and stuff, which was quite tricky, but that work was all done by Wylie Co (who also handled all the postviz and in-house for the film)
‘Everybody loved that little mouse’
b&a: There was a stuffy of the desert mouse in some EPK footage. I am obsessed with stuffies, I really love how they get used. Can you tell me a little bit about that?
Paul Lambert: There’s a scene where Paul comes out of the tent, where he is with Jessica, and basically, they’re there overnight and they’ve been covered with sand, so it’s basically where they were basically turned into a dune. And he uses a sand compactor to actually get out, and when he pops out of the sand, he sees this mouse.
We were going to have to do this mouse in CG. We had designs from Weta Workshop, which we tweaked and tinkered with. And once Denis was happy, we went to props and had that built. It was actually quite expensive as the level of detail was really high
So we had that built, and that was going to be our reference for when we shot in the desert. Everybody loved that little mouse. But it worked out really well because, again, you know, having a reference is what’s key to it, you know?
Hunter/seeker hologram
b&a: Tell me about the hunter/seeker sequence?
Paul Lambert: This is something where the tech developed for it was done by DNEG, but that sequence didn’t make it to the cut until very late, and Wylie picked up the actual post of this. But the actual shoot of it was we had done some tests, again, on set, to try and help us inform the shoot, and I was worried that Paul being immersed in this hologram, I was either going to have to do a close up digital of Paul to actually see the interactive light on him? So we started to experiment with different ideas, how can we actually get a real, interactive light pass on Paul based on a hologram? We couldn’t create a hologram, but we could definitely try and work out a way in which we could get the light from the hologram onto Paul correctly.
So what we did is that DNEG came up with a system where we could track Paul’s position on set, and from that position on set, we then drove a projector which would project an image of a slice of the hologram, so basically when Paul was in a certain position, he would have this slice being projected on him. As he moved forward, it would be a different slice, and as he moved, it would then go through each slice, and it felt as if he was going through branches, going through his face and body. What you ended up with was actual light hitting the skin and getting all that beautiful subsurface on his face which I didn’t have to do in CG, which I wasn’t looking forward to doing.
Yes, it can be done digitally, but it’s also very expensive and it takes a while to get a close up CG human, having done it a few times on previous projects. It takes a while to actually get that right. So having the ability to get that on set, and showing Greig and him playing with it, and him adjusting his cinematography around that, because in essence, it’s something tactile which he can play with, rather than being a virtual world. I think it worked out really well. So what the guys at Wylie did in post was then line up and add the rest of the bush around Paul, around that interaction. So it then felt as if he was truly inside this hologram. I thought it was a good way to capture something real on set, to then give us a good base for post.
‘Developing’ the digital film
b&a: Greig Fraser mentioned something in an article at some point about the shooting of this, where it was shot digital but scanned onto film and put it back into digital. Is that right?
Paul Lambert: We shot with the Alexa LF. It was the same camera used for the IMAX and for the 2.39:1. We used bigger lenses for the IMAX and kept the aspect ratio, and then for the 2.39:1, we used 1.65 anamorphic lenses. But still on the same sensor, so it was framed in that particular way. And from the start, Greig had talked about wanting to try this process. I think he had previously experimented with it. Dave Cole, his colorist, out at FotoKem, did an early test at the very beginning of Dune on some footage, and it was a look which I hadn’t seen before.
Basically what it was is that you filmed out the imagery and then you would put it through the bath, to expose it, and then you would scan that back in. Not the print of that, but the actual scan of the negative. And what that actually produced was you didn’t get something really grainy back. You could hardly tell the grain. But what it did, like it does on film, whenever you get those hot highlights, you get a certain filmic halation to anything bright, and that’s basically what I was seeing. I was seeing this analog form of halation, based on a chemical process and not a digital process. And it gave it such a quality to it. That test was something very much which Denis wanted to do, and there were questions towards the end of the post production whether we were going to have time to do it with the movie’s original release date. It was 50/50 whether that process would have been able to make it, because it adds weeks to process this, obviously, to the final film.
But yeah, because of the delayed release, they were able to do that. So it gives it a very distinctive look. It’s kind of like a hybrid between digital and film. I loved it, absolutely loved it. Also one thing about the formats for the movie, because it was interesting in that we were very focused about all the framing when we were shooting, and we framed it for IMAX when it was an IMAX shot, and, ‘Yeah, it’s going to be IMAX, we don’t need to worry about 2.39:1.’ Of course, when you get to the edit and the post and you’re showing the studio, 2.39:1 becomes very important. So there are 29 shots in the movie where we had to create something called a mega frame. We had the IMAX frame but you couldn’t get an animated 2.39:1 cut out of that IMAX frame. It just didn’t work. So what we did was we extended the sides out on the IMAX frame and then shrunk it down to 2.39:1. So we call those the mega frames. And a lot of them were the CG ones as well as a few plate based shots, so I do encourage people to go see the IMAX version as well as the 2.39:1 version because they are slightly different. There are framing changes.
And there’s one shot, and Tristan can probably correct me if there are other ones, but there is one shot in particular which was completely reframed for 2.39:1 and IMAX, because you just can’t fit it. It’s where Paul is at the bottom of the frame, and the worm comes up and you get to see the worm’s mouth appearing at the top of the frame, and it just doesn’t fit. You can’t get a 2.39:1 version of that, and you can’t expand it out, because Denis wanted the entire frame covered with the worm. So there were two distinct versions of that shot, but then 29 other ones where we had to extend either side.
b&a: What effect did that have of DNEG’s work in particular?
Tristan Myles: I can still hear the render farm groaning. Particularly for the ones that we had to reframe, they ended up being 7K in width at least, so it was just extra work, but it was worth going the extra mile to get that framing right, because as Paul said, you couldn’t do an animated rack or anything inside the IMAX frame. So it had to be done, and it was worth doing, I think. It was mostly digital extension work. I think for the worm shot we had to do this for it was re-using the plate, and then there was some plate extension on the side where they were standing on some rocks, looking up at the worm. Everything else was digital for the sand extension. But most of the sand extension work we did was based on plate photography anyway. So it looks like it should do, like plate photography.
Awesome!