Step-by-step with Framestore.
Virtual production is clearly a hot topic in visual effects right now, and it’s been making its mark in several television shows. This includes His Dark Materials, the series from BBC and HBO, where a virtual production approach was utilized for the key bear fight.
Series visual effects supervisor Russell Dodgson, from Framestore, used game engine assets and a system to overlay pre-planned scenes while shooting the live-action photography to craft the vicious fight. Here he breaks down the virtual production steps for befores & afters.
1. The kind of fight it needed to be
In the book, there’s a really violent finish to the fight where one bear eats the other bear’s heart. We were like, well, we can’t do that. But we still wanted it to have a violent feeling. We put it inside the palace rather than out in an arctic exterior; this gave it a more grimy and gritty feel and let the bears hit and fall over things.
Also, we took their armour off. That could be construed as making it easier for us, but it doesn’t make it easier because you’re just revealing more fur and fat. I spent quite a while watching polar bear and grizzly bear fights and the thing that makes these things really seem horrific is the way that their fat moves. I wanted to see as much of that as possible, even though it’s painful to do.
Furthermore, we wanted to film it like we were in amongst it, from the perspective of someone who’s in amongst it, like Lyra is, and that was another way of making it more intense.
2. Early fight beats
We also wanted a way of filming this fight that felt quite real and had that kind of happy accident quality that you get from actually shooting something.
So, we started off with roughing out the beats of the fight with me and the stuntman wrestling around. We used a combination of watching grizzly bears fight, we looked at some wrestling, we looked at the way polar bears break ice because that’s a really nice violent action that polar bears do that we could translate into fighting (polar bears actually don’t fight to the death. They just have a bit of a dominance fight and then one walks away a bit bloody and that’s it).
We sent that reference off to our amazing animation team in Montreal. They did a first pass of the bear fight using the reference of us wrestling and a representation of the sets built by production designer Joel Collins as a model. That gave us a rough blocking that we were happy with.
3. Game engine assets
Meanwhile, Painting Practice, who were the previs and postvis department for the show and also part of the art department for concept design, had already built a version of the bear palace in Unreal. That’s something that they had done as a way of virtually scouting it.
We thought, as we have that as an asset, we could port it over to Framestore and bring the bear animation into that Unreal Engine scene.
Then we bought director Jamie Childs into Framestore and did a virtual camera shoot of the bear fight to start getting a sense of the whole sequence. He would stand inside a capture volume with an iPad style-controller and he filmed the entire bear fight from all the different perspectives that he wanted, which gave us rushes to then cut together a fight.
4. Shooting on the set
On the set, we then used Vanishing Point’s Vector system, which gives you an AR-like virtual cam feed, a real-time comp of your CG assets over the live action set. It’s got quite a light footprint on filming and quite a fast set-up speed.
We had an operator who was operating that system, through Unreal Engine, and his job was registering the virtual set to the real set. Then we had a feed going from that which allowed the camera operators to see the bears overlaid in real-time over the set.
There was a small delay in the feed in a short number of frames, but it gives you enough to frame up and shoot accurate back plates that have a feeling of being operated by humans. We shot the whole thing hand-held, moving around. When you’re in that space that you’re starting to film, you’re able to react to the happy accidents that you get on the day.
We were able to shoot the whole thing in a day. I think it’s pretty clear that virtual production processes are the future of how we can efficiently and creatively manage complex visual effect sequences like this one.
