How the ‘NYAD’ visual effects team helped turn Annette Bening into an open ocean endurance swimmer.
When NYAD visual effects supervisor Jake Braver first met with directors Elizabeth Chai Vasarhelyi and Jimmy Chin to discuss the film–which follows the true story of Diana Nyad’s attempts to make a 110-mile open ocean swim from Cuba to Florida–the directors’ observation was that water itself should be a crucial character in the movie.
“The water was Diana’s greatest foe, but also her greatest ally,” relates Braver. “That was something that VFX needed to bring to the ocean sequences. The water needed to have a performance.”
For obvious safety, budget and logistical reasons, scenes of Nyad (played by Annette Bening) in the open ocean could not be filmed for real. Instead, production made use of a 6500 sq ft tank at Pinewood Studios in the Dominican Republic, with visual effects tasked with a large number of extensions, augmentations and sometimes completely CG re-creations of the ocean, Nyad and her support crew.
Those augmentations would be necessary to essentially art direct the right kinds of waves and currents for the different conditions Nyad would find herself in. “We were very focused on, how can we make the ocean current directable and controllable,” notes Braver. “We need to be able to say, ‘Well, we want a medium current moving against Diana here, and we want a really fast current moving against her in this part of the scene, and then in this part the current needs to be carrying her.’ We built frames wider to take you out of the tank in order to give you these big sweeping shots that you would have if you were shooting this movie for real out in the ocean, boat to boat.”
In and under the water
Given that the ocean would be augmented in this way (both for wide and close-up shots, and underwater shots), Braver orchestrated an overarching approach in visual effects for scenes featuring Nyad, as well as her support crew, such as friend Bonnie Stoll played by Jodie Foster, when they needed to be in the water. This was, to take footage of them shot in the tank, rotoscope them from the frame, matchmove a digital replica, simulate waves and water so that it would properly interact with the actor, and then bring back the live-action to the final shot (digital doubles were also part of this).
In particular, a layout process was established for the purposes of ‘directing’ the performance of the water by building different ocean sets. Explains Braver: “There were four oceans we developed with a variety of amplitudes, and then there was an automated process that simulated and rendered them systematically across hundreds and hundreds and hundreds of shots. We could then pick not only an amplitude but also a speed for each beat in the sequence. We could say, ‘Okay, the water needs to ramp up here and get faster. Now it’s turned against Diana…’.”
Although varied due to pipeline between DNEG and Wētā FX, the basic recipe to complete a typical shot involving Nyad in the water, a vendor would, as Braver discusses, “simulate the shots with the ocean variant that had been chosen, then take Diana’s match-move movement, simulate it, then re-animate Diana to actually move in the water as if she was moving, then re-simulate again, and keep going back and forth until these things had a balance that worked for the shot.”
One particularly tricky kind of shot Braver’s VFX vendors had to tackle was for scenes where the camera was right there in the water with Nyad, with the waterline often cutting through the frame. “Those required a complete replacement of all of the water around Diana and sometimes the boat,” details Braver. “It required making the boat move and it required reanimating Diana to move.”
“Because the water in the tank most of the time was pretty flat, that also meant Diana’s refraction wasn’t really usable,” adds Braver. “So we had to do a digi-double for Annette, for the refraction underwater, and sometimes that shot was from the sky looking down, sometimes that was looking up–each shot had its own recipe in terms of what worked.”
For shots that were of Nyad underwater, several extensions needed to be made. “The tank was not very deep, so we extended the tank in just about every single shot,” says Braver. “We added all kinds of particulate and turbidity and marine snow to make you feel like you’re actually out there and to give a sense of movement. We are telling the story that Diana’s heading from A to B, and we needed something to help sell that. So that meant adding seaweed and detritus and fish and all of those things to really bring it to life.”
Ten different visual effects vendors contributed to the film. DNEG (VFX supervisor Mohsen Mousavi) was brought on for a round of ocean development and then worked on Nyad’s first two swim attempts, including the swimmer’s colorful falling stars hallucinations. Scanline VFX (VFX supervisor Dann Tarmy) handled the training swims, plus the violent storm Nyad swims through. Wētā FX (VFX supervisor Chris White) completed sharks and the entire final swim attempt. Rise FX crafted scenes on the marina in St. Martin. Phosphene took on the Los Angeles set extensions and various comp work.
Cadence Effects was responsible for bringing together archival footage from many different source formats such as 16mm and miniDV. Lola handled a de-ageing scene and some facial augmentation to make Bening more puffy and swollen from her allergic reactions and jellyfish stings. Proof provided visualization. Finally, Folks, and Studio 8 FX contributed prep and comp duties.
The art of a digital Nyad
For the CG representation of Nyad, Bening was photogrammetry scanned during post-production (other characters that required digi-double work were photo-modeled only). Braver says the intention was to always keep Bening’s original performance, not to re-invent it.
“Maintaining Annette’s performance was really a cornerstone of the ethos of the visual effects in the movie. So, the rule of thumb was if we were close enough where we needed a digi-double with her performance, we really wouldn’t use a digi-double– We would find a way to use Annette’s plate performance. There’s a few really wide shots of her swimming that are digi-doubles, but the animation for those is match-moved and rotomated from other shots of Annette swimming.”
“It’s an interesting thing, though,” continues Braver, “because there was this confluence of needing a performance from Annette that was moving with the water, but Annette’s performance wasn’t moving with the water. That’s how we got into this back and forth of trying first and foremost to always make the water work with the performance, not the performance work with the water, which was always the way we went about it wherever we could.”
One scene required the de-ageing of Bening. This was for a moment when she is watching a younger version of herself on TV. While archival footage of the real Diana Nyad plays a key role in the film, for this moment, says Braver, “we thought it would take you out of the movie if you were watching Annette watch Diana. So we wanted it to be Annette watching Annette. We took the this archival footage of Diana and captured Annette in Lola VFX’s ‘egg’ and replaced from the chin to the top of her head, using Diana’s neck with a performance of Annette re-doing the interview. That was the only de-ageing in the movie, and we wanted to make sure the performance driven by Annette, and fit in with all the other amazing real footage of Diana that is weaved into the fabric of the movie.”
Hatching the hallucinations
At one point, Nyad’s journey in one of the swims turns into psychedelic hallucinations involving falling stars and even the Taj Mahal, ultimately leading to her being stung by box jellyfish.
“For the falling stars hallucination,” advises Braver, “Jimmy, Chai and I wanted it to be the most beautiful thing Diana’s ever seen with this progression of the stars falling, swimming with her while glowing, and then going quickly into almost full horror movie mode when she gets stung. We ended up keying in on this FPV drone reference of fireworks, and we loved having a lot of color and it almost feeling like a star or a comet falling from the sky. DNEG did lot of concept art for this.”
Meanwhile, for the Taj Mahal, Nyad is first led there by tropical fish. “The thing about the hallucinations was that we decided logic has no place in them,” says Braver. “In the Taj Mahal, there’s a bunch of jump cuts and all of the water has no current, since it’s Diana giving herself a reprieve from what’s happening out there. We also ended up at fifth scale for the actual Taj because we wanted this balance of it feeling massive in some shots, but then we also wanted it to relate to Diana’s scale in other shots. It didn’t necessarily need to feel consistent, which was also a really fun freeing thing shot to shot.”
Stormy seas
Some of the most complicated water simulation shots were those in which Nyad was being pummelled with waves, particularly during the storm sequence. These involved significant amounts of interaction between CG water and Nyad, CG water and the support boat, and then for a moment that Bonnie also gets in the water. Scanline was responsible for this sequence.
“The storm was a challenge because our goal was for it to be completely crazy that someone would swim through a storm like that, but yet believable that someone was able to,” outlines Braver. “We looked at a lot of references early on with Scanline to find the right mix of an adequate level of danger, but still be believable that someone could swim in it.”
Live-action plates in the tank made use of a boat that was relatively static, that is, not aided with the movement of gimbal that might normally provide simulated ‘stormy’ motion, while Bening was also swimming without real waves or currents. “So, we had those elements on Diana shot, but now you have to add this thrashing ocean that’s really pushing her around,” says Braver. “It required a tremendous amount of re-animating these performances from Scanline in the shots where they weren’t fully CG. Ultimately, there’s probably more CG replacement in the storm sequence than any other sequence in the movie, just by the nature of the fact that they’re these punishing waves knocking into the boat, the kayaks, Diana and everyone else.”
A trademark of the storm sequence, too, was the red spill in the darkness of the water from the LED guideline hanging off the boat for Nyad to follow. “It’s tremendously cool lighting that helps sell the turbidity and churn of the ocean,” observes Braver, “and it was really something I talked to director of photography Claudio Miranda about very early. He was very specific about the color.”
“The challenge was that red light underwater is not the best for light transmission, actually it’s the worst!” adds Braver. “So this red light they had on set was so powerful that it had to be able to be exposed and balanced with everything else. I think it was an aircraft landing strip light. We were really careful in the storm sequence where we replaced all the water to therefore replace the light to really bring back with all these qualities of the light that Claudio had constructed for use on set . It was so beautiful.”
