Leaf sims, bamboo sims, water sims: another day in the life of Scanline

The studio reveals their work for ‘Shang-Chi and the Legend of the Ten Rings’. UPDATED with new images.

The in-depth coverage of Shang-Chi and the Legend of the Ten Rings at befores & afters continues with this discussion with Scanline VFX’s digital effects supervisor Jessica Harris.

She tells us all about the studio’s significant simulation work required on the Wenwu vs. Ying Li ‘love fight’, the ‘alive’ bamboo forest maze and the trip the characters make through the Moon Gate waterfall.

b&a: What was Scanline’s brief for the Wenwu and Ying Li fight?

Jessica Harris: One of the big things that the director, Destin, and Chris Townsend, who’s the VFX supervisor on client side, kept reiterating was how they wanted this to be beautiful. It’s a fight scene, but it’s really two characters falling in love. It set the sense of emotion, the family dynamic and where it all begins. They also kept reiterating how it had to feel organic. It had to be exciting and have that fight energy, but it had to also have this beauty about it. Originally, we actually had a lot more effects involved in it. But eventually we stripped it down a little because it just lost that romantic feel to it when it was too overdone.

Our FX team and animation team had to work very closely together to make sure that any sim that was done worked with the movement of the actors in the plate. We were very, very aware of their body movement because it was choreographed so well. All the hand movements, everything had a meaning to it. We wanted to make sure that that was highlighted and stayed at the forefront, and it wasn’t something that got lost with a lot of effects and the sims on top of it.

b&a: I’m really curious about the kind of interaction that animation and FX and layout have, in terms of integrating the FX in the scene.

Jessica Harris: We basically had to make sure that we had animated geo that was completely following the character’s hands and legs and body twists so that we could make the leaves feel like a part extension of their body. So when FX actually started animating all of that, it came from a place that felt very organic and natural to their movement. It wasn’t just a matter of having to sim and say, “’Okay, do a wind force here and add a little bit of turbulence.’ It actually had to have the motion that the arm had. So if Ying Li did an upsweep with her arm, it followed through with that motion and it extrapolated from the very start of it. It didn’t just explode out of her hand, it matched the same intensity her arm was doing.

It was constantly having the animation team go in and do a tweak on it, and then having to send it over to our FX artists to try to get a sim that made sense for it. I think one of the biggest challenges that every FX team has is what looks aesthetically really great and what actually follows the laws of physics, and then trying to alter it so we can give the client what they actually want. Animation helped a lot with that—they could give FX something that they could start with that really felt like it was part of the scene when it was shot and it matched with the choreography.

b&a: What were the kinds of thing you had to sim? Was it mostly leaves?

Jessica Harris: There was actually a lot of different dust sims that were going on there and we had several different leaf sims on top of each other. It had to feel like there was an actual force pushing it through versus just a gust of wind hitting on it. There was a lot of almost invisible sim work going on in there, where you didn’t really necessarily see what you were looking at. But if we took it off, you would definitely feel it very differently. We had probably about four or five different layers of leaves that interacted with each other, doing their own thing, but dancing together.

b&a: In a sequence like that, where sometimes the action is ramping up to fast speed or it actually goes in slow-mo, what challenges does that bring in FX simulation and FX creation?

Jessica Harris: It’s definitely a huge challenge because FX always likes to do things in real time. We had a combination of doing some slow-mo actual FX sims in there and we also had our compositing team doing it, who worked hand in hand with them. It wasn’t unusual for a compositing team to do almost ‘previs’ for FX and then hand that off to them. Luckily we have a very experienced FX team who did a great job, led by FX supervisor, Hannes Kreiger.

b&a: There’s a lot of other stuff in the background to that sequence—trees etc. How was the scene filmed?

Jessica Harris: They filmed it on set where we had maybe one layer of bamboo. And then we had to add in an entire forest behind them. There was a wind machine on set. So we did have bamboo and a little bit of it moving. The big challenge for our lighting team, as well as our animation team and our compositing team, was getting the CG wind setup to match to the actual plate setup. When you have all these plants moving from on set, well, when you add the CG in there, we had to make sure it didn’t suddenly look like it was a static background or that things were moving in opposite directions or different speeds.

Another huge challenge we had was that you have to put a really dense forest back there, but of course you want to have enough light bleeding through. On set, they’re lighting it just for that center part, not necessarily for a 20 mile deep bamboo forest. So there were a lot of custom set-ups we had to do on that to get the lighting to work in with the plate, as opposed to trying to change the plates too much, to fit in with the real world bamboo we were doing. We didn’t want to take away from what the DP was doing on set, because he did a really fantastic job lighting everything.

And then of course there’s a lot of face replacement on some of that, as well as rig removals and a whole bunch of other things that you would expect doing this kind of choreography, especially the fight scenes with wire work.

b&a: The other main sequence Scanline worked on was the Moon Gate arrival, where first they’re driving through the bamboo and it closes up on them. How did you handle the bamboo there?

Jessica Harris: Well, essentially the bamboo forest was trying to kill them. Most of that was all CG, because they shot down in Australia, obviously not in a bamboo forest. We pretty much had to build an entire forest. It was pretty crazy the size we had in there. On some of the shots, our big overhead shot where you can see the actual maze of the trees opening and closing—that had over 600,000 trees in it. Close to a 100,000 of them were actually simulated. So they were opening and closing, and we had all different types of bamboo, we had basically created an entire library of it because what we found out very quickly was bamboo looks very similar. And if you don’t put enough variation in it, it looks fake.

We also had to get the leaf movement correct with the wind, because you had to find that balance of, well, you want the leaves to move, but you don’t want them moving too much, because then it just looks like shimmery aliasing, almost, even though it’s not, so that was a little bit of a challenge.

For this bamboo forest opening and closing, we had to try to get a natural feel of how bamboo would actually bend when it was moving. How much of the ground was moving on it was also another challenge that we found, because everything was simulated. So, if you move a tree, and then you have the ground, and how much of it’s going to ripple out from there? And how many plants are on there, and how they’re going to interact if the bamboo’s actually moving through it? Because there’s a lot going on with it. You had the tree moving, then you had the ground dirt moving, and then the plants on top of it moving.

b&a: I always think it’s incredible to see a sequence like this that I feel like I haven’t seen before. And then I thought, ‘Have I seen any great Houdini sims of bamboo forests attacking people or anything?’ Was there any reference like this, real or CG or otherwise?

Jessica Harris: Yeah, a lot of times when you’re doing something, the first thing you think of is, ‘Oh, what other films have done this?’ And let’s take a look at that for reference.’ Or if there’s any natural disasters, you can look at and say, ‘Oh yeah, this is what it looks like when a tornado was ripping through a forest.’ And there weren’t a tonne of things. I personally went onto YouTube and was looking up bamboo destruction, or cars driving through bamboo. We did do a lot of research on just seeing how does bamboo bend. Because that was one of the big things that we were trying to figure out is how do you make these trees move and make it look like it’s somewhat natural? We definitely learned a lot about how bamboo bends and breaks and how it splinters and what it looks like when it does.

b&a: And then when they head through the waterfall. What could production film for this?

Jessica Harris: They actually shot the Razor Fist car going over the water, a little creek bed, and that’s what we started with. We had to clear up the water so you could see through it. And then we had to extend that out as well because you want a certain amount of splashing. We also had them come through the bamboo. We had to do a whole splash on that through the water, as well as interaction with bamboo falling off the car and landing in the water bed.

The next thing was the waterfall. It’s always so fun doing waterfalls. Once it goes through the waterfall, that was all water sim, where it goes into the magical portal, where the water slows down and it’s just floating in the ‘void’, as we called it.

b&a: How did you approach that?

Jessica Harris: The look of that water sim was carried over from other scenes as well. We had to tie it into that, which was a little bit of a challenge, since you want to have this common theme and have the water react in the same way, but slightly differently. I think one of the biggest challenges we had on it was not making it look like one big, large floating blob. What happens is when you slow it down a lot, you start to really kind of run back and forth between, how does this water tear apart from itself? We were really trying to get it so it stayed uniform and it stretched itself, but didn’t tear. The client really wanted to feel like it was in a void as well.

A lot of times, there’s more of an environment that you can reflect through it with the refractions to give it a little bit more character and to tie it in with everything. We were pulling in some of the cave features through the water before they get to the cave. That’s how we started with this whole color palette on it, because they go from obviously this bright bamboo forest into a black void, into a magical cave. We tried to bring all those elements through it, as well as trying to get that water to feel like it’s slowed down, not necessarily stopped in time.

b&a: I want to go back to the waterfall itself as well. Scanline can do water incredibly, but I’m sure that you yourself had an opinion about what the water should look like. Chris Townsend must have, Marvel execs must have. How do you do what seems to be a relatively simple waterfall sim, which is never that simple, I know, and do iterations of it with people commenting about water, when sims are kind of random things?

Jessica Harris: I mean, that is always a challenge. Because obviously, you move one thing and it can completely change the sim. What we’ve definitely found is that a lot of times, at least on shows that I I’ve worked on, I like to approach it where we do layers of sims on it. The cool thing about a waterfall, especially something’s falling over a cliff is that it’s not one big piece of water all falling together. You can section it out to a certain degree.

What I have found is that when you do your sims in sections, they can all react together, but it’s much easier to confine it to a certain part. So if they want to change one thing, you can change that a little bit more than some of the others without completely changing the whole thing. We also have a fantastic compositing team, led by compositing supervisor, Micah Gallagher.

And obviously, Scanline’s been doing waterfalls for years. So we have in-house tools and setups where we’re able to take multiple sims and stitch them together. So we have a lot of control over how we do our water, but it’s also just not just one sim, which I think a lot of people who haven’t done a lot of waterfall work might think.

So, depending on what type of water the client wants, sometimes you don’t have to change the sim as much as you think you would. Because we might not actually have to change the water. We have to change the mist sim on top of it. And we can do a little bit of mixing and matching the sims if we have to.

It really is all physics, too. When it falls a certain way, you have to figure out the math. Now, if they want it to do something that’s not necessarily how it would work in real life, then there’s a lot of extra geo you can put in to move things around the waterfall.

You can say, ‘Okay, we want it to spout out a little bit more this way. Well, what about if we put a collision object in a little bit over to camera left, a little bit more behind it?’ That will pull out part of it without completely breaking the whole thing. And from the front, it looks like exactly what the client wants it to do. So there’s a lot of that as well that happens, but it’s definitely a delicate balance.