How motion capture for ‘The Stand’s’ raunchy scenes was all done remotely by a single performer

The pandemic required the show’s VFX team to quickly pivot the approach to these crazy crowd scenes.

Visual effects work on the nine-part limited series The Stand, which aired late in 2020 and in early 2021 on Paramount+, was just gearing up in March last year when the pandemic hit. Key sequences inside the show’s fictional Inferno hotel featuring crowds of guests carrying out a large range of debaucherous activities had been flagged as VFX shots.

There was just one problem—the planned method of crafting some of these performances was partly via motion capture, and that usually involves bringing a whole team together for a technical mocap shoot. Social distancing requirements meant that The Stand’s visual effects team, led by visual effects supervisor Jake Braver, had to think fast for an alternative.

Industrial Light & Magic, which was responsible for the creation of the hotel seen in ‘New Vegas’ and the crowds, came up with a methodology that saw a single motion capture performer take on every single crowd member role, all by himself on ILM’s mocap stage, while the rest of the VFX crew controlled the shoot remotely. befores & afters sat down with members of the team to find out how they did it.

Brendan Byrd performs motion capture at ILM’s stage.

When a pandemic hits

Jake Braver (visual effects supervisor): We had to build Randall Flagg’s Vegas hotel which is introduced to us as a total den of iniquity and just a really raucous, crazy, sex palace. We found this abandoned hotel that was an amazing location, but it didn’t have the scope and the scale of Vegas. So we basically put a digital set on top of the practical location. When you’re looking at the shots, about two-thirds of the set is CG. The layout of this place is such that there are essentially these balconies wrapping around a big atrium, many stories tall.

There are a number of scenes with different crowd volumes and reactions. We had a lot of extras on the floor and we had some extras on the balconies during filming, but in creating the set extensions we needed to populate the balconies, and more of the set. As we initially planned, everything from cheering and booing to all the extras’ action in the finale, was planned for a mocap shoot on stage in San Francisco.

As we were shooting, I’d get questions from the studio about dailies, and they’d say, ‘So there’s going to be people there, right? It’s going to be full of people?’ And I said, ‘Yes, of course it’s going to be full of people. It’s going to be great…’

Then, the week before the motion capture shoot, the pandemic happened and obviously we all had other worries. I know Stef and the production team were working like mad getting the team up and going and making sure that we were still working and productive, and Laurent was doing the same thing. The mo-cap conversation sort of fell into the background for at least a couple of weeks.

We showed the studio the cut of the finale sequence for the first time, the ‘Hand of God’ sequence, and said, ‘Here it is. What do you think?’ And literally their one note was, ‘It’s great. Again, we just want to make sure that it’s going to look like there’s hundreds of people there.’

At that point, we said, ‘Well, okay, we need figure out how we are going to add people. We looked at everything. We looked at Xsens suits and we looked at people doing mocap at home, and even animating them… Then, I got an email from Stef on a Friday morning saying, ‘Can you talk?’ He said, ‘Listen, we’re trying something. I’m not going to tell you what it is, but if it works, you’re going to be really happy.’

On the following Tuesday, at the start of our dailies review, Stef hopped on to say, ‘So we figured out how to do motion capture remotely with one person in the studio in San Francisco and everybody else remote.’

So, that’s what we did.

Stefan Drury (executive VFX producer, ILM): Firstly, I have to say, I haven’t seen a bigger change since March the 19th, 2020. The entire London ILM facility—I mean, we were anticipating lockdown happening, and we decided to get ahead of the game. On the Tuesday, we decided to move and get everybody working from home, primarily for their safety, because it was a pretty scary time. We made that decision collectively. And by Thursday evening, we had every one of the 450 people out of the facility.

It was disturbingly fun. We set up this entire triage unit. All the production crews took on this new mentality of helping everyone to get out of the building. We literally had people in tech and support handing out keyboards. You’d go from collect your keyboard to collect your mouse to collect your monitor to collect your chair. We had reception lining up cabs. I was with the other execs in a room, literally calling people, ‘Right, your turn.’

What was phenomenal was, we gave everyone the Friday to get their home setup working, and everybody was working on the Monday remotely. It was just astounding. All of these years of, ‘Can we work remotely? Can people work from home productively?’ In genuinely less than a week, it was Tuesday to the Monday, 450 people, no one left in the office, everybody working. Pretty much 95% of The Stand was executed remotely. We’d only turned over a few shots before that.

CG motion captured digi-doubles are populated into the scene. ILM also produced extensive set extensions.

How the remote motion capture worked

Jake Braver (visual effects supervisor): I think it’s important to say that in the COVID world, we were still figuring things out. In the week leading up to the shoot, Laurent and I, and Vincent Papaix, the associate supervisor at ILM on the show, were all pinging around what we wanted to capture during the shoot. The first time we see the hotel, there’s people having sex all over the place— in crazy positions, in elevators, and on balconies.

Part of the hilarity and ridiculousness in pulling this shoot off was that it had to be (due to COVID) one person, so Brendan Byrd at ILM was on-stage, there’s really no other way to say it, having sex with himself. He would do one side of the performance, and then he would go act the other side. It was born out of the fact that we couldn’t have two people on stage, or at that time even in the facility.

We probably spent half the day shooting very bizarre, weird things, of which I don’t even want to go into full detail. And then the other half of the day we shot more booing, chanting crowd people, drunk people, people falling over balconies, people running and falling, more standard stuff. Brendan didn’t have anyone doing props for him but he was like, ‘Okay, I brought this,’ and pulled out high heels to run in as a woman. It was really necessity being the mother of invention, we were just kind of figuring it out.

On a motion capture stage your days are usually productive, but never this productive. We just flew through it, because there were really no distractions. There were no technical glitches, either. But more than that, it was very focused. We had fun, but it was very focused. Because I was sitting at home, Laurent was sitting at home, all the support staff were sitting at home, and we were just sort of like, ‘All right, let’s go to the next item on the list.’

Laurent Hugueniot (VFX supervisor, ILM): Yep. It really was him on his own. We had a witness camera to see him. Really, it’s just unbelievable when you think about it, but he was able to do all this and just make it work on the day. Like a one man band, in a way, playing all those roles one after the other, and including the one about…you know, that one.

I think it was genius, the idea to extend that location [the Pacific Inn in Surrey, British Columbia]. I mean, it’s a big location without the CG, but with the CG, my God, it’s a proper lobby, with an enormous room.

Jake Braver (visual effects supervisor): And in a way, it was easy to pivot to this plan. We knew we were doing mo-cap and already had the bones of a plan. The part we had to adapt was the motion capture and bespoke elements. It was a credit to ILM’s work that it was just so darn seamless. There are some digi-doubles that are way closer to camera than I would typically be comfortable with for a background person.

Stefan Drury (executive VFX producer, ILM): We were also lucky on The Stand that 95% of the show had been shot, so we knew exactly what we needed to do. I think, normally, quite often we’re thrown in at the end at the end of the shoot and you’re kind of guessing what you’re going to need, but here, I think we could really go, ‘Here’s exactly what we need. We’ve got the measurements of the balcony. We can build that in advance and just roll it in with that one person track into the centre of the volume.’

‘This is going to work, right?’

Jake Braver (visual effects supervisor): I should point out that the studio, before they spent a bunch of money on a motion capture shoot, asked me, ‘This is going to work, right?’ And I said, ‘Well, I don’t think anyone’s ever done this before, so I can’t say, but when we tested it, it worked great.’ I wanted to be confident, but yet honest.

We just had such a great team, and they were so dedicated and on top of it that I went into that day thinking, ‘Alright, so we’ll probably get half of what we need.’ And again, just because everyone was so on it, we got everything we needed and more.

We also had bluescreen/greenscreen 2D elements to work with. The bigger shots with the sweeping camera moves, like first time you see the Inferno and everything at the end of the Hand of God sequence really lean on the CG. The more simple moments in the set— the people just passing through, people talking, some of the locked-off wide shots, those lean a lot more on a shoot we did three days before everyone had to start working from home.

Stefan Drury (executive VFX producer, ILM): I’d just like to say Jake was responsible for the greatest three minute conversation in a client review session ever. After we had done the mocap shoot, I was in on a review call, I’m there to be kind of invisible, but help out where I can. I sense Jake is struggling to really verbalize what he means. We were looking at a shot with lots of digi-doubles in there. Jake said, ‘I feel there’s a repetition of action.’

Now, there’s lots of very sensible adults on the conversation and I’m thinking, ‘No one’s saying it. No one’s saying it, are they?’. So, I go, ‘Jake, do you mean there’s too much doggy style?’”

Jake Braver (visual effects supervisor): [laughs]. It was one of those things where there were 50 people on that Zoom call. You don’t know who has their speakers on or headphones off with their kids in Zoom school. So, bless Stef for saying that, because he cut right to the core of my note.

A view of the balconies.

A new normal for VFX?

Jake Braver (visual effects supervisor): Looking at what we did in one day of shooting, we never could have done that if we were in-person. That was the remark from Phil Hoffman, my VFX producer on the show. He said, ‘I can’t believe we did all that. That was the most productive day of motion capture I’ve ever seen.’

I had absolutely no trouble communicating with Brendan and he had no trouble communicating with me during the shoot. However this was a very different shoot than one requiring more nuanced motion capture like directing narrower performances, specific performances, facial motion capture, all of those things. Those become a lot trickier because you’re really directing actors and having a conversation with the performer about what they’re doing and how it fits into the character’s perspective, not just from a visual effects perspective.
Is that possible to do over Zoom? 100%. Is there anyone in the world who would rather do that over Zoom than have a conversation in-person? I think probably not, but it’s great to know it’s an option.

Stefan Drury (executive VFX producer, ILM): I think we’ve got to just work really hard over the coming year, forgive me for saying, not to be lazy and just revert. I think we’ve got such a unique opportunity to try and mould the way that we’re going in the future.

Laurent Hugueniot (VFX supervisor, ILM): The pandemic has changed the way we work in visual effects. I mean, probably in many industries as well. And now we are in this Zoom world where we can access anyone anytime so easily and have a very quick conversation about things and just pop in and out of different spaces so easily. That’s great. But I think long-term, we need to bear in mind that we all started as a group of people that worked together before and then went in our corners, and what happened for the newcomers? What happened for these guys that have not been in part of the group, that only know us as faces on the Zoom call?

I think there’s unavoidable adjustments that will need to happen to this new world. And I think for visual effects, it would be we need to see each other. We need to see some big screens. We need to have chats and read body language more than just the top half, if you see what I mean. All those things matter, I think. But it’s amazing. I’m amazed how we managed to switch just like that, from our world to another world.

Jake Braver (visual effects supervisor): I agree with Laurent, we absolutely need big screens and to see people in person. That said, I would add that I found reviews and dailies much more efficient and collaborative. It’s no longer, ‘Well, there’s only room in the theatre for however many people…’ It’s, ‘Come one, come all,’ at least on my shows. I think one positive out of all this is that it really opened minds to flexible and remote work.

I also think it really has unlocked more collaboration. The biggest thing everyone says about working from home, is that you have to work to replicate the incidental conversations around the office. I completely agree. It’s popping by someone’s desk, standing over an artist’s shoulder. It’s all of those things that you have to work to replicate from home and make a point to check in and say, ‘Hey, how’s it going?’ or, ‘What’s going on with this?’ or, ‘Can I see…’ You need to make a point of creating those moments if you can’t have them in person.