How the director relied on ILM’s StageCraft virtual production set-up for the New Jersey tornado sequence.
Up ahead, the funnel cloud reappears, moving from left to right. Cars have begun to pull to the right and left sides of the street. Mitzi accelerates and drives straight up the middle of the street, now empty of traffic. Sammy sees that the traffic lights are swinging wildly from side to side. A couple of telephone poles begin to rock back and forth. SAMMY (CONT’D) Is this s-safe? MITZI (laughing:) Of course it’s safe! I’m your Mother! A transformer box on one of the poles blows with a bang and a shower of sparks. The children SCREAM.
That’s a short excerpt from a section of the official screenplay to Steven Spielberg’s The Fabelmans, in which the mother, Mitzi (Michelle Williams), drives her children toward a tornado in 1950s New Jersey.
On a larger ‘disaster’ movie or even larger Spielberg film, realizing a tornado sequence might be par for the course. But this was just a short moment in The Fabelmans, a film that also had a considerably less budget than his other blockbusters.
Furthermore, the filmmakers were not looking to travel to New Jersey to film this particular sequence, but instead remain on the west coast. The task was therefore set to visual effects supervisor Pablo Helman, from Industrial Light & Magic, for a way of bringing the tornado scene efficiently to life.
VFX considerations
Helman began by considering the different methodologies available to actually capture the sequence. “One way was to do the journey in the car for real or film it on a ‘process car’. But of course with Steven there’s a whole lot of camera movement and dialogue and storytelling going on in there, and children who need to be in the car. So process was out of the question.”
“We could also have done it on a bluescreen. I don’t personally like bluescreen work, mainly because of the lighting. It’s very difficult to get the lighting correct and to marry whatever you’re seeing outside with whatever is happening inside the car. For me, lighting is about 80% of the images that we make. If we don’t have the lighting, we’ve got nothing. I’ve also found that you can never escape what you shoot. In other words, you can change it a little bit and you can mitigate some of the problems, but it’s very, very difficult to make it look believable.”
Enter ILM StageCraft
Shooting on one of ILM’s StageCraft volumes was Helman’s next consideration, and of course the one the filmmakers chose. ILM has for several years now been perfecting and refining their StageCraft set of virtual production tools used on such projects as The Mandalorian, and they seemed to fit Spielberg’s needs here. That was despite the fact the filmmaker had never done LED wall shooting previously.
“We really wanted an opportunity for him to take a look at it and to see how he felt,” says Helman. “Steven is a very visual guy. Obviously he takes to technology really quickly, but, you know, ILM StageCraft is a completely different way of working. You have to do production design at the front and you have to be creative about budgeting because it’s a different way of working. You have to make decisions early. However, this was a perfect project since we didn’t have to come up with CG assets, per se, because it was a journey, and we also needed to get up and running very quickly. Which we did.”
Helman’s reference to assets there would normally mean a CG build requirement–perhaps an alien planet surface, or a complex environment in which to place actors on the StageCraft stage (which would also have to be optimized for real-time playback on the LED walls).
Instead, since this was a shoot that would involve filming the mother and children in a car on stage with the “journey” through the New Jersey streets projected on the walls, the “asset” was reimagined as live-action background plates augmented with some compositing. Helman worked directly with Director of Photography Janusz Kaminski—who had also never done an LED wall shoot—on this asset stage.
“We came up with a six-camera array to shoot a 360 degree background plate of a car driving through New Jersey. Then we had three weeks to stitch everything together and replace the skies so that this could be projected onto our LED screens. We had to paint out our plate driver of course because he shows up in the tiles.”
With the footage stitched, Helman and the filmmakers gathered at one of ILM’s existing StageCraft volumes at Manhattan Beach Studios for a pre-light session. Helman admits that, even then, Kaminski was hesitant.
“We had just the environment running on the LED’s and nothing else on the stage, and the first thing Janusz said was, ‘This is too bright.’ But I was able to say, ‘Well, let’s bring the car in and let’s start color correcting.’ And so we did all that and then suddenly Janusz looked at the car and he was like, ‘Wait a minute, I can see the reflections and I can see everything else!’ And then he started talking about where to place cameras and how to move around the car. He just got right into it.”
“Steven took it in right away, too,” adds Helman. “He immediately started designing shots, bringing the camera in, bringing it out. And then basically we were just shooting like a real set with 200 people there, just capturing the scene.”
Making it tornado-y
One of the particular challenges of this sequence was, of course, the fact it happens during a severe storm, and with a tornado in sight. That meant there would be a need for wind and rain (and lots of it). For the array shoot, the filmmakers chose not to incorporate water or rain in the plates so that they would have more control in stitching and final VFX, but they did include some special effects explosions.
“We planted an explosion on a pole on the street, and we knew we were going to go through it, so we timed it with the array plates,” describes Helman. “We also had sparks on the day on the volume, plus we added more in CG later because we wanted them to bounce in a specific way.”
On the volume, too, rain was orchestrated to hit the car and would be collected by a tray underneath. The car was also hit with wind machines and some extra debris.
During filming, the ‘stormy skies’ and street scene array plate would be played back on the LED walls, along with a special extra layer enabled by the StageCraft brain bar team depicting a tornado. “We were able to move that around to give the actors an eyeline to try to show where things were going,” says Helman.
Finally, as is the case with a multitude of LED wall shooting, the in-camera captured plates from the StageCraft shoot went through a round of visual effects augmentation, primarily to add in the tornado as well as the thrashing trees.
“That was much more traditional VFX work with roto and matchmoving,” notes Helman. “But the performances were all there, and the camera movement was all there. Steven cut the sequence before he turned it over, too, so we knew we didn’t have to replace all the trees in the whole three-minute stitch that we had, we only had to replace it on the shots that we were seeing them in in the cut.
A new way of filmmaking for an experienced filmmaker
Helman marvels at the way Spielberg and company ramped up so quickly on the StageCraft shoot, which was all designed to take place over just one day. “We did about 19 setups that day, which is an average day for Steven, I guess. But I mean, on a first time basis, nothing went wrong. It was a perfect day.”
The visual effects supervisor adds that the use of StageCraft proved key in capturing genuine actor reactions for this dramatic sequence. “When the actors come in, they’re like, ‘Oh my God!’ They are in the environment, they are in the moment, they are as close to reality as it possibly can be. That’s another reason why ILM StageCraft came about, just to give the actors the tools they need to be transported to wherever they are.”
This went for the Director of Photography and the other filmmakers, too, with Helman suggesting StageCraft allowed for a more accurate ‘lensing’ of the final frames, compared to say just shooting on bluescreen. “I do believe that when you’re on a bluescreen, you lens in a specific way and when you’re on location, you lens in a completely different way. StageCraft is as close as you can come to location without being there. You’re going to frame things in a completely different way, you’re going to start connecting things. With StageCraft, you connect things and you connect in a creative way, meaning that the process of connecting happens on the day while you are thinking about it and sparks ideas and it’s a very ‘in the moment’ thing.”
“I find that sometimes,” continues Helman, “when you have a lot of control—and, just to be clear, in visual effects we love control, that’s why we separate everything—the more time you spend agonizing over the choices that you make and you overthink things. Sometimes it’s good not to think about it too much, just go with your gut. That’s what Steven often does, and that’s part of the beauty of StageCraft. You go in there and then it is what it is.”
