Shooting in the desert, using infra-red, and making a moon rover melee

There’s an incredible sequence in Ad Astra when Brad Pitt’s character, Major Roy McBride, joins others on the moon en route to Mars. On board a procession of lunar rovers, the group is ambushed by space pirates, with devastating consequences.

When I first saw this sequence in the film, I initially thought it was possibly almost entirely CG, but after speaking with Method Studios visual effects supervisor Jedediah Smith, I was surprised to learn how much director James Gray and cinematographer Hoyte van Hoytema had acquired with practical photography.

Not only that, Hoytema employed infra-red filters for plates shot in Dumont Dunes to replicate, at least partially, the lighting on the moon’s surface. Method Studios then took that photography, extended the environments and crafted CG rovers, riders and many other effects – including visor reflections – to complete the final shots. Here’s how they did it.

Shooting infra-red

The location shoot for the moon rover plates was Dumont Dunes in Southern California, which stood in the barren lunar landscape. Looking to replicate as closely as possible the ‘kind’ of stark lighting that would really exist had they been able to shoot on the moon, Hoytema relied on a modified ARRI Alexa to film the plates in infra-red.

“Basically,” details Smith, “they removed the infra-red filter and the sensor so that it could capture infra-red light. For a lot of the wider shots, they shot on a stereo rig where they had a film camera on one eye and the infrared Alexa in the other eye.”

Method Studios would later take those two plates and align them, and then use the luminance from the infra-red footage and the color from the film footage. “Infra-red photography lends this really interesting other-worldly look to footage which made the dunes and the sands of those desert plates look really bright and washed out, and made the sky really dark,” adds Smith. “This was representative of the look and feel of the moon.”

“For a fair percentage of those shots that were shot on location, we actually didn’t have any film plate at all,” describes Smith. “We had a black and white IR plate without color that we had to colorize, so for those shots we had to rotoscope every material that was a different color on the pirate rover or the hero rover that was in frame, and basically colorize the footage, and then also do a replacement of the background for the moon surface. That was a huge challenge, and a massive amount of rotoscoping effort that my roto supervisor, Tony Como, and his team did an amazing job with.”

These plates were mainly for eventual wide shots, since production did not have the actors available for the Dumont Dunes shoot. A practical rover was built that was fully operational. “They were out there in the dunes driving it around in the blazing heat, so a lot of those plates are actually a physical rover that we then replaced the environment behind,” says Smith.

Making a moon surface

The infra-red footage was, however, just the beginning. Method Studios spent significant time referencing NASA Apollo mission archives and documentaries and ultimately a great deal of the lunar surface was, of course, built in CG. Method Studios developed a procedural shader system in Substance Designer to do this. Then, a specific approach to lighting was employed to achieve the final look.

“There’s definitely an aspect to the lighting on the moon that’s really, really alien and strange to us,” notes Smith. “If we look at a normal scene on Earth, obviously there’s atmosphere, which means that there’s atmospheric perspective, so the further away an object is the more hazed out and lifted it gets. There’s atmosphere that diffuses the light and adds spill light to everything, as well.”

“Those things, obviously, just don’t exist on the moon,” continues Smith. “So one of the things that we tried to very carefully recreate is that sort of harsh lighting feel where there’s not really any bounce light filling in shadows. It’s just either direct sunlight or black.”

CG rovers, astronauts and space pirates

In addition to what could be used from the live-action plates, a CG build for the rovers began care of reference photography of the prop rovers. A main digi-double for one of the astronauts who is shot in the head was crafted, while background CG characters and rovers were also placed in the scenes.

Once the space pirates attack, the sequence amps up, so much so that the rovers begin to crash. This causes wheels, equipment and other paraphernalia to go flying, which involved some nuanced simulation work from Method Studios in order to replicate the moon’s low gravity.

“The client was really interested in mimicking real physics for the moon,” outlines Smith. “We tried to create our effects simulations in Houdini using moon gravity and no air resistance. For the explosions and for a lot of the shots where we’re adding dust behind the rover tyres, the performance and behavior of that is unlike what you would see on Earth, just because the gravity is about a third of Earth’s gravity and there’s no air resistance or wind to create patterns and movements in the dust.”

The pirates, and the astronauts, also fire weapons at each other. The VFX team was tasked with working out what kind of futuristic look the bullets from the weapons would have. Says Smith: “We came up with this story of it being like a magnetically accelerated rail gun where it fires a heavy piece of metal that becomes molten and flies through the air and has this really strong sort of weight and kinetic energy to it. There’s quite a few shots throughout that fight where those bullets are impacting the ground and sending up a lot of debris and little pieces of metal flying everywhere.”

Space (visors): the final frontier

“Oh, the visors.” That was Smith’s reaction when I asked him about how visor reflections for the astronauts were handled. “Even though they are probably the least glamorous thing, they were certainly the most difficult. There were two categories of shots; the wider shots that were filmed on location in the desert, and then the more close-up shots, which were shot on stage.”

“For the shots that were filmed on a stage,” says Smith, “they set up this area with the actors. So, all the close-ups where we see their faces through the visors were shot there. They did their best to try to mimic the lighting conditions on the moon. They had a main light as distant as they could from this small stage set-up, and not much fill light, and they shot without any sort of chroma screens, so no bluescreens or greenscreen.”

Method Studios’ visor work was therefore all about recreating the environment – which simply didn’t exist – in the reflection of the visor itself. “Those visors are like mirror balls that see the whole world, and since the world around the plates that were shot was black, we had to create match-animation for all of the visors, and take that and place it into the world that we were building for the moon, and animate the rover through the world, and in some cases, actually animate digi-doubles for the passenger astronauts, and render all of that out, and then composite it and layer it with the plates visors.”

Smith acknowledges he was torn about what would have been a better decision for shooting – filming with or without visors, “but I think the fact that there were physical visors there for us to build on top of and for the actors to gauge their performances, added a layer of realism to it.”

One of the many challenges, notes Smith, was that astronauts actually wore different variations of visors. “The one that Brad Pitt’s character is wearing had a darkened visor that came down over the top of it, and so in the plates, the refraction – the light that you see through the visors – was tinted darker and greener, and then the reflections on the visor are tinted more of like an orangey yellow color. So it was an interesting balance trying to get the reflections that we were rendering looking the same as those reflections.”

“Layering them correctly was also tough because, in a lot of the shots, the reflections of the other astronauts were there. For those shots we would actually rotoscope all of the reflections that we wanted to keep, and actually composite them, basically removing the moon surface behind. That’s because in the reflection, the reflected astronaut would include the moon surface that we were compositing behind that reflection.”

For some of the other visors, which were clear, Method Studios had to mimic not only the refraction color darkening the actors’ faces and making them more green, but also the reflections, and in some cases paint out the reflection that was there in the visor. “This was because it might have been too bright and over the top of their face,” details Smith, “so we had to paint that out and then render CG reflections of our digi-doubles, our other astronauts, and also the moon’s surface.”

Stars, or no stars?

In addition to the moon surface, and the action on it, Method Studios also had the task of filling out the ‘sky’ (i.e. space). But, in those specific moon lighting conditions, would things in the sky, such as stars, be seen at all?

“We went back and forth on stars a lot,” comments Smith. “The director was really into realism, so it was always a question of, do we see the stars? Do we not see the stars? Do we see like a hint of stars? And I think we achieved a pretty good balance in the film.”

After the encounter with the space pirates, the surviving rover passengers find themselves on the dark side of the moon, at which point McBride lifts his visor up. “For those shots we did add stars in the background pretty subtly, like very de-focused, but there is a layer of moving stuff back there to communicate the fact that they’re driving over a rough terrain,” says Smith. “Even in the daytime shots there are actually stars in the sky, they’re just really dim so you can’t really see them, but if you expose those shots way up, they’re actually there.”

Organic and visceral

The moon rover attack in Ad Astra ended up being a sequence where so many different types of visual effects were required, in addition to what could be captured in-camera. Smith says the final shots in the sequence, when a missile attack is called on to take out the remaining pirate rovers, added yet another dimension. It, too, required looking at real-life reference.

“For that shot, we found this really cool reference of a scientific experiment trying to stimulate a meteor impact in a vacuum. It was obviously very small scale, but had this really interesting look and feel where there was this expanding cone of debris that just kept going up and up and up. We showed that reference to the client, and they really loved it, and we built this crazy effects simulation to mimic the same look and feel, adding a bit of fireball for dramatic effect. It’s one of my favorite shots.”

“I think in that shot and in the rover scenes there’s this visceral quality about it that comes through,” concludes Smith, “even though a lot of the environments have been replaced with CG. There’s a certain organic quality that really communicates an interesting mood to the sequence.”