VFX roundtable: season 4 of ‘The Crown’

Framestore’s team discuss turning Spain into Australia, animating an injured stag and deep fake face replacements.

Over 230 shots in season 4 of Netflix’s The Crown, Framestore helped build out the 1980s world as inhabited by the royal family. The VFX work was diverse; from creature animation for an injured stag, to digital environments and crowds for Australia, London and New York, to using deep fakes for face replacement shots.

Here, befores & afters sits down with Framestore VFX producer Standish Millennas, 2D supervisor Ollie Bersey and animation supervisor Ross Burgess to break down the studio’s shots. We also feature for the first time a whole set of before/after images of Framestore’s work.

b&a: Can you give me a general overview of Framestore’s work for season 4?

Standish Millennas (VFX producer, Framestore): The kind of work that we were doing was primarily locations—changing locations, amending locations, modernizing locations. There were all the airports that needed creating, for instance. They were filmed on a runway but there were multiple airports and aeroplanes.

For the royal tour of Australia, there were lots of recognizable locations that needed to be reproduced [the production filmed in Spain for Australia]. Diana also went to New York so we had change the streets of Manchester in the UK to New York in the ’80s. Coming in and out of Buckingham Palace was another environment. Then there was the stag, one of the bigger kinds of effects we had to do.

b&a: Let’s talk about the stag. Tell me about the kind of conversations you had about the emotion and personality of that creature?

Ross Burgess (animation supervisor, Framestore): The most interesting part of the job for us in animation is that it was an injured stag. Normally when we’re animating CG characters we use a lot of reference, but here there was not a lot of that. There was a lot of reference of stags being tranquillized, but an injured stag, no. So the challenge for us was to make this photorealistic creature perform injured in quite a stylized way, flawlessly, so the audience think it’s realistic. Luckily we had a very good storyboard artist who boarded the whole thing out which was really useful for us.

Probably the most difficult shot that we did on the show was the shot where the stag gets shot. We were noticing as we were trawling through this disturbing footage, that when they do get shot stags don’t tend to get back up again. They just collapse, especially when they’re shot in that hind area, they just collapse and then they go down and that’s it. But obviously if we’ve had done that it’ll have ruined the whole story, because he had to get back up and run off. And that was really hard, because we knew that actually that was something we had to fake.

So having the stag that gets shot collapse, and then get back up again, in these tiny amount of frames that we had to tell that story, this huge story, and we had no reference for it at all–it was tricky and something that we had to just make up.

b&a: Was there a real stag filmed at all or any kind of stand-in used?

Ross Burgess: There was a blue wooden stag used that was placed where the stag would be in the shot. We call them stuffies, and they’re used just for placement purposes. There wasn’t really a real stag that we could use since we wanted to make him majestic. We wanted to make him regal, so that meant we had to make one up. You can’t just go and cast a stag and say, ‘Ah, can you limp a bit?’ It’s not possible.

b&a: For the Australia sequences, which were filmed in Spain, there are still a lot of Australian landmarks featured. How were these captured?

Ollie Bersey (2D supervisor, Framestore): It just so happened that Andy Scrase, our visual effects supervisor, was holidaying in Australia last year and was able to gather some reference. He grabbed some pretty extensive stills of Ayers Rock and also some stills of Sydney Opera House. They were extremely useful, particularly the Ayers Rock material, which formed the basis of the textures and information there.

b&a: So for example there’s a scene at the foot of Ayres Rock—was that filmed on a stage or outdoors somewhere?

Ollie Bersey: Outdoors in Spain. The environment they stood on was just color-corrected and massaged into the right kind of tonal area with DMP behind it.

Standish Millennas: That scene of them climbing Ayers Rock became much more tricky than we expected in an area we weren’t necessarily expecting, which was, it was filmed on a reasonably light colored sandy surface and roto’ing out the actors so they could then be against the red rock surface, and dealing with their foot interaction with the ground which was completely different as you would expect from a soft surface—their feet were disappearing into sand, but then had to actually be on a hard surface. That was just an added complication to it that took a lot more work than we anticipated. The team did a great job of actually creating the Ayers Rock, but integrating them into that was very tricky.

b&a: What about the Opera House scenes in Sydney?

Ollie Bersey: For one of the establishing shots, they did have quite a large crowd on location. There were a few passes of them and then we did some pretty conventional crowd replication. For one of the crane-up shots, we did a take-over to a particle-based crowd where we had sprites and then we used Nuke to populate them and randomize them into the DMP.

There’s a hero shot on the steps of the Opera House which used a similar process but we had to employ quite a lot of techniques and an awful lot of iterations to take out visible twins or triplets since we didn’t have a huge amount of sprites available to us. There were lots of re-times in there, lots of added flags and other stuff to try and busy it up.

Standish Millennas: I think that shot is our record for the amount of versions.

Ross Burgess: What did you get up to, Ollie?

Ollie Bersey: 120. I’m a bit save-heavy. I save everything. I don’t always render everything. When you’re in 4K you’ve got nowhere to hide in terms of these things and you get one repeated sprite, and then you see another, and the more you take out, the more you see. Eventually right at the end we asked everyone, please look at this, if you can see one for goodness sake tell me!

b&a: There were those entrances and exits from Buckingham Palace, too. How were those handled?

Ollie Bersey: On the backlot that they have at Elstree studios, they have the front gates and section of railings and just the archway of the palace frontage. And beyond there the sort of porte-cochère, the entrance that the cars would go to and a little corner of set. They shoot all of their palace material there and everything else is CG. There is a palace asset that has been used and developed throughout the series, and we embellished it a bit more this time around. For me, these are the shots I like doing the most where people don’t realize we’ve done anything.

b&a: How were the New York scenes filmed?

Ollie Bersey: They were all in Manchester. They required not just cityscape work, but also ‘road straightening’ to make everything look like longer streets. And there’s one key moment where the place they shot goes around the bend and to straighten that out was a large amount of work.

Standish Millennas: We’d add moving traffic on the streets of American cars. There were foreground cars but we’d need more in the background. We added CG fire escapes and American traffic lights and other CG set dressing. There was a lot of attention to detail.

b&a: What can you also tell me about the deep fake work to replace one of the rider’s faces at the showjumping scene to look more like Princess Anne? Was that considered a better approach than just trying a normal kind of 2D face replacement?

Ollie Bersey: We wanted to do it, to be honest with you. It’s pretty new, it’s something we’ve been talking about on a different project and when this came along we thought, actually, it’s ideal. We had plenty of candidate frames to give the A.I. to rebuild this face.

There were a couple of times where the performance of the stunt double wasn’t quite right and of course the deep fake reproduces that performance, so sometimes the face shape wasn’t quite right, or didn’t quite look like Erin Doherty, the actor. So we had to get in there and slightly change a few things or regrade a few parts where the face shape is ever so slightly different.

With deep fakes it’s really successful when you have a good impersonation of the person you’re deep faking, but if the face is completely different, you do need a little bit of extra massaging. It was a rewarding experiment for us and I think something we’ll look to use it again, certainly.