Body of Work: Scanline’s latest adventures in fluid sims

From Heart of the Sea to Hydroman.

Mention the phrase ‘fluid sims’ in visual effects and one of the studios many people think of first is Scanline VFX. Using its proprietary Flowline toolset, Scanline’s artists have turned out some of the most complex work in fluid and particle simulations in the past few years.

For this new befores & afters ‘Body of Work’ breakdown, Scanline recounts some of its most recent efforts in fluid sims, as seen in In the Heart of the Sea, Miss Peregrine’s Home for Peculiar Children, Black Panther, Spider-Man: Far From Home and Game of Thrones season 8.

In the Heart of the Sea
Bryan Hirota, VFX Supervisor

Scanline was bought on to handle about 500 shots for In the Heart of the Sea. The company’s long background in handling complex fluid simulations made us a good fit for the extensive ocean work required. The most complex fluid simulation work on the show centered around when the whaling ship Essex sailed into a large squall as well as the aftermath of the ship’s encounter with the whale which left it on fire and ultimately sinking.

The primary challenge for the project was to maintain a sense of authenticity.  Overall VFX supervisor Jody Johnson had just finished Rush with Ron Howard and DOP Anthony Dod Mantle and they wanted to approach this project with a similar mentality. Obviously the film needed a heightened sense of drama but the overall goal was for it to convey being grounded in reality and not crossing the line into fantasy. To this end they went to great lengths to capture as much as possible in camera. This included building a full-size whaleship to use in the large exterior water tank at Leavesden with elaborate SFX wave/spray machines surrounded with blue screen and having a tall ship (Phoenix) stand in for the Essex for photography in the Canary Islands.

The squall sequence required us to seamlessly cut between shots of Phoenix out at sea, blue screen tank footage shot in the UK and full CG shots. This required a wide arsenal of solutions ranging from being able to replace or augment the rigging and sail configuration on all three ships,  replacing the water around the ships partially or entirely and creating the squall digitally.

Jody and Ron had a collection of real footage of large ships at sea during storms. We really pushed our Flowline teams to give us those kinds of large dynamic complex splashes. We also ran simulations for the rigging lines and masts with cloth simulations on top of that for all of the sails. Additional simulations were also run for the rain interaction on the ship, and cascading water that would drip off of the sails and surfaces. The accumulation of all of these layered simulations both small and large helped to add to the realism of the shots.

Once the ship was mortally wounded by its encounter with the whale, we were responsible for the sinking of the Essex. In real life it took the ship a couple of days to sink but in our film that timeframe was compressed down to a few hours and taking us from daytime, through dusk and into the night. To make matters worse, we also set the fire to the ship. The initial phase of the sinking was primarily photographed at Leavesden with the full-size ship listing in the water tank. We used Flowline to add digital fire to the onset SFX fires. Once the crew had retreated to their whale boats, the photography switched to an indoor tank shoot of the cast on the smaller boats with an entirely digital ship burning. We simulated the masts, sails and ropes as before but in this case they were on fire and collapsing.

One of the most memorable shots for me that we delivered was the final view you have of the Essex in the film. You see the ship underneath the surface, its structure having given way. We built the ship so its wood planks could be ripped apart. Bubbles from air trapped inside would come streaming out and the remaining whale oil barrels would explode further damaging and providing momently glimpses of illumination to see the ship falling into the murky depths.

Miss Peregrine’s Home for Peculiar Children
Jelmer Boskma, VFX supervisor/art director

For Miss Peregrine’s Home for Peculiar Children, our main challenges arose in the underwater sequence where Emma takes Jake to her secret hideout, a sunken ship deep beneath the ocean. Scanline’s history with creating water sequences meant we were tasked with the development of Emma’s custom bubble stream effect, including the giant bubble that surrounds Jake’s head and works as an underwater helmet. As Emma and Jake descend underwater and make their way through the body of the ship, Emma blows bubbles, forcing the water out of the environment and filling it with air so Jake is able to breathe without his helmet.

The SFX team built a series of different hydraulic air pumps onset that produced underwater bubbles of different sizes which we could shoot in camera as reference. While this was helpful to capture the behavior of bubbles and how they looked at different angles, the force at which Emma needed to eject the bubbles and how the bubbles needed to behave was so specific, they had to be built entirely in CG.

A crucial component of getting the bubbles to appear and behave correctly, was how they collided and merged as well as how they separated and detached from one another. Custom dev work was done utilizing meta balls within Flowline, which allowed the geometry in the simulation to merge and detach physically accurately.

Getting the lighting of the bubbles right underwater was also a challenge. Typically, bubbles have sharp highlights which are dictated by the light source and the environment they are found in. As the ship is set in deep, murky water off the Welsh coast, we introduced beauty lighting underwater to achieve the right cinematic look for the bubbles. Since the design for the ships interior and the location of all the light sources was mapped out during previs, we could see exactly where light would enter the ship and how it would bounce around right from the outset.

Another challenging shot came after Emma blows all the water out of the ship and Emma and Jake emerge dripping wet. This shot ended up being a hybrid of how it was originally conceived onset. Once again, the SFX team built an amazing lever rig for Emma and Jake that started off with them laying on their backs underwater. The rig then tilted Emma and Jake upwards at a 90 degree angle, bringing them out of the water until they were standing up straight.

We took what was shot in camera and replaced the background with the digital ship environment, enhanced certain transitions and added in residual dripping water on various background elements. This shot ended up being shot in one take and is a great example of practical ingenuity combined with a technical and creative approach in CG.

Miss Peregrine’s was a special project for me, primarily because I was fortunate enough to work with Tim Burton, a personal hero of mine, and be involved in some of the creative development and design quite early on, in particular the look of the ship and the design of its interior. One of my favourite shots from the film is a shot of Emma standing on top of the staircase in the atrium, filling the sunken ship with bubbles (see the still, above). As the sequence progress Tim had the idea to introduce this moment with Emma and suggested to have it set somewhere in the ship’s dining hall. I ended up up pitching this alternate framing, having her standing at the top of the stairs instead, which Tim ended up liking a lot and is what ended up in the final film.

Black Panther
Bryan Grill, VFX supervisor

First off, I would like to thank Geoffrey Baumann and the Marvel team for giving us an opportunity to work on Black Panther which is one of my personal favorite films to have been a part of in my career. I think the biggest fluid simulation challenge on Black Panther was two-fold.

The first challenge was that we had an environment that was miles long which needed to have waterfalls reacting and looking as realistic as possible. It was a back and forth between modeling and simulation that finally got us to the finish line. Achieving the realistic subtleties of how water moves over a surface was key for the correct breakup and look of the waterfalls. Technically, the higher the cliffs were, the more simulation that was needed. At the end we had simulated over 150 individual waterfalls that had their own unique look based on the way the surface beneath each one was meticulously modeled. Even the smallest variation was critical to the believable environment of Warrior Falls.

The other big technical hurdle was the Warrior Falls challenge pool. Even with the amazing engineering of the set by SFX Supervisor Dan Suddick and his team, to achieve running waterfalls and flowing water in the pool at the flow and speed director Ryan Coogler required for his storytelling, we ultimately needed to alter or completely replace the water digitally.

This was a huge undertaking which started with rotomation of every character that was interacting with the water in the pool. We needed to create realistic water interaction at both ends of the spectrum, as subtle as a the tribal elder Zuri just standing in the flowing water all the way to the fast-paced action fight between T’Challa and M’baku and T’Challa and Killmonger. 

The team at Scanline was able to nail the Warrior falls environment because of our ability to iterate as many times as possible so we could visually and economically perfect the look of our water simulations. The team at Scanline VFX is fearless, incredibly resourceful and smart.

For example, to solve the logistics of simulating and rendering the waterfall, ponds and river section’s visibility automatically, our pipeline department created a smart database called VisData which collected all the individual elements placed in the environment. These tools allowed us to optimize our work as we ended up having 102 waterfalls, and server wise all the caches involved in this show occupied 1.1 Petabyte!

Another advancement was the development and introduction of a new feature to our Flowline software called Timesplit (split particle generation by time) which made all of these huge simulations. This feature allowed the team to simulate and render waterfalls at high quality standards (approximately 1 billion elements per waterfall) while our multi-shot tool would automatically replace everything in one single master file i.e characters, their respective animation, cameras, simulation areas, technical simulation details (voxel size, solver properties, velocities etc). All these technical advancements gave us the room to achieve all the artistic and aesthetic adjustments the filmmakers were wanting to tell the story of Black Panther.

The scene that I was most proud of was the T’Challa and Killmonger fight. The director Ryan Coogler and his cinematographer, Rachel Morrison, shot the battle specifically to evoke a feeling of darkness and light. The brooding dark skies and golden sun trying to peek through created a beautiful pallet of color. Our job was to create an environment that was believable and realistic so the story of the fight for leadership of Wakanda was not overshadowed by sensationalized visual effects. In fact it was more important to make each simulation as physically correct as possible. Every splash, flow or interaction needed to be as natural as if it was actually photographically shot in a real environment becoming more of a supporting role so not to inhibit the drama unfolding between the two battling cousins.

Spider-Man: Far From Home
Julius Lechner, VFX supervisor

Our main sequence in Spider-Man: Far From Home was the Hydroman battle in Venice. Creating a character made out of water was both a technical and creative challenge. The basic idea was that Hydroman forms himself using the water from the Venice canals by attracting it from the surrounding area in the shape of a vortex. As the water is drawn upwards, it becomes like a turbulent and aerated, like white water rapids, with some of it releasing and falling back down, forming a repeating cycle. The main underlying technical problem we had to solve was to have the water conform to the shape we wanted, while still behaving naturally and being controllable, regardless of his animation.

We had to find a good balance between how much we forced the water to his body in order to get interesting looking simulations, while still being readable and recognizable as a character. To achieve this, we used an underlying mesh to guide the water through space although the water never actually interacted with this geometry. Rather, the mesh acted as a source of information for the simulation and can carry a multitude of parameters that could be driven by the artist with texture maps. This approach gave us an immense amount of control over the simulation, to leave as little as possible to chance. Using mesh as a base for his simulations allowed us to do treat his animation rig almost like a traditional character so our animation team were fairly unrestricted in how to animate Hydroman.

An important aspect was also the look of the water itself. It not only had to look real, but the color palette hat to fit well with the real green of the Venice water with the warm sunlight scattering through.

Scanline has a long history of creating complex water simulations and creating Hydroman felt like the perfect fit for us. It was great to apply one of our specialties but in a different form. Of course, there was still a lot of development involved. Our Flowline water simulation team around David Stopford, Tzung-Da Tsai and Scanline President Stephan Trojansky had to write completely new simulation logics and tools for the artists, to give them the control they need to get exactly what they are trying to achieve.

We also tried to push our simulation setups further, increasing the detail until our water droplets were the same size as in the real world, and reworked our approach for our mist simulations. Usually fine mist gets simulated as a soft atmospheric, comparable to smoke or steam. In our new approach it was actually millions of very fine particles which, when seen from far away, look like soft mist but are recognizable as individual points when viewed from a close distance. Not only is that more accurate to real life, but it also looked better and gave us the ability to fly directly through a cloud of mist without losing detail.

It’s really hard to pick a single shot that I liked the most as I think the team did an amazing job throughout the whole sequence, but one that sticks out is when Mysterio and Hydroman have their first interaction. It was in the first trailer, so it was the first time the audience got a look at our work, and it showed nicely what the setup could do and the detail it produced.

Game of Thrones, season 8, episode 6
Mohsen Mousavi, VFX supervisor & Marcel Kern, FX supervisor

The Iron Throne melt sequence was something that we started early on in March 2018. It was such an iconic moment and we were all very excited to put our best into play!

HBO wanted us to deliver the most complex and realistic melting simulation ever seen on the screen. We got constant feedback from the executive producers to push the resolution so we came up with some very advanced and clever shaders to get an insane amount of detail on top of the simulations. We were able to send dozens of the different shader looks at once to narrow down the final concept which was very helpful considering the immense tight schedule we had!

One of the biggest challenges was maintaining the detail and sharpness of the throne asset when combined with the fluid simulation. This was especially in the close up shots where we needed to see fine details like engravings, and the sharpness of the blades that make up the throne.
 We solved this with proprietary setups that allowed us to have adaptive detail control. High detail in areas where it was required, and less in areas that were less visible or not visible at all. This kept everything highly optimized and allowed us to iterate much faster.

From an aesthetic standpoint, it was challenging to have it behave like heavy molten metal while it is being blown away by the fire. There is no reference for this, so we spent some time finding the sweet spot. 

Everybody was very excited to be on this show, so we had high expectations of ourselves from the very beginning. This was a very iconic scene and effect, so everybody gave 200% and fortunately we have very talented artists across all departments! Also Scanline’s extensive infrastructure, resources and pipeline gave us a lot of flexibility and freedom.

Working on such iconic sequences over the last 3 episodes of the final season gave us a few short nights but most importantly it gave us the chance to work with the best of the best!

It felt like a big family with a unified goal and we will be forever thankful for everyone’s amazing dedication and passion. The energy of everyone involved, the entire team, and their motivation to make history really eased the process and made it a highly memorable experience!

One of our favorite shots is where you see the throne from behind, while Drogon is blasting it with fire and it starts to fall apart. It was one of the first shots in development, so we are somewhat attached to it. But ultimately we loved working on the whole sequence! Every shot had its challenges and requirements, like the super closeup shot of the molten metal rolling down the floor.