Key lessons the visual effects studio learnt on the impressive virtual production series.
When I talked to members of the Framestore team about 1899, the mysterious Netflix series set on the steamship Kerberos that was largely filmed on a LED volume at Studio Babelsberg, what stood out for them were a couple of key approaches and key sequences in the show.
So, I’ve shared them here as some key take-aways from the production, one of Framestore’s first as the lead creative studio on the virtual production side.
Here, visual effects supervisor Christian Kaestner, head of virtual art department Freddy Salazar and virtual production lead Connor Ling discuss jumping into the challenging show, how it differed from a traditional VFX project, and some of their favorite virtual production moments like the dining room set, real-time oceans and on-set rain.
How virtual production has changed the workflow in VFX
Christian Kaestner (visual effects supervisor): Virtual production does now involve us in the creative process more than I think we are used to. Historically, we are not involved in the costume design. We’re not involved in the production design. Yes, visual effects supervisors are there when the shows are being planned, but in this case, we had to finish visual effects before we even started shooting because they needed to be ready for in-camera work. I think that was not just new to us but also new to a lot of other departments.
Freddy Salazar (head of virtual art department): What was important early on was to figure out the goal of the virtual environments we were making right up front before anything else, in terms of mood of say the sky or the lighting. In VFX, you prepare a set, you prepare a scene, you prepare a shot. You put in the light, you put the things in, but then you can change your camera and you can change anything, really.
Also, we are really advanced in VFX–we are going for final photoreal, but in real-time, it has to look good and everything needs to be optimized. And then, it had to be approved ahead of time! You need to make bets, basically, about where to go and where you want to put the biggest effort in.
Christian Kaestner: Luckily, it was always clear that the wide establishing shots would not be in the volume. Maybe 80% or 90% of the ship we would never see on the volume, yet we needed to know how it all felt as a rough construct. As we progressed, and had some more production design decisions, and we had some camera decisions, and we had some storyboards to go along, we could refine and refine. We would refine up until the last day to pre-light. During the pre-light we’d work out, ‘Right, this is going to be painted this way, so now we can work with the right color against it.’ And also what became clear very early on was that we needed a lot of lead-in time to build the assets, but then once we’d built them, we also needed to optimize them to work in real-time.
Also, one thing that happened was that we made a decision very early on–which is something that as a visual effects supervisor you possibly don’t want to do–to get rid of all the markers. We didn’t have any markers on the LED wall. We knew that maybe we wouldn’t get everything in camera, but we would be trying to get as much as possible that way, and if we have markers everywhere then everything would become a visual effect. It made some shots maybe a little bit harder on tracking and in replacing some of the backgrounds, but it really helped the shot count that could be in camera because we did shoot with a wider and the longer lens simultaneously.
Connor Ling (virtual production lead): Having Framestore in control of the entire process, from pre-production into on set, into the post side as well, was a really beneficial part of this. What it meant was, in the brain bar we had artists that had directly dealt with the content and had been dealing with that for months. They really intricately understood it.
Framestore’s virtual production toolset
Connor Ling: Framestore has a product toolset called fARsight that covers multiple angles in terms of virtual camera scouting, and virtual reality scouting, and so on. Going into VR scouting proved really useful in terms of gauging scale, and working out the set that was actually possible to shoot in, in terms of the assets being to scale, and then displaying the LED imagery in there.
That VR scouting was done right up to shooting, and was a really integral part of getting feedback from the directors, and the DoP to make sure that everybody was happy with that this is actually going to work in volume as a lot of people within the crew hadn’t done it before in terms of actual physical shooting. It was critical that everybody just understood the limitations in terms of physical space as well as being able to properly embed yourself in it. The toolset proved really useful for this.
Freddy Salazar: With fARsight it was possible to assimilate what was on set with what was on the wall. It was interesting to talk about this with the DoP and director, to basically build the picture. For example, the engine room, you have a lot of source of fire with the chimney. But you play with them to compose a picture, and to make it natural. It’s not a source light, even though it’s bright. We were able to use these very technical tools to help art direct what was there.
Building a convincing dining room
Christian Kaestner: The dining room had to be open, and it had to feel grand. You want the audience to perceive that you’re in a first class dining room with a skylight. For me the most impressive thing was when the final set came together with the virtual set and became one, it was just so good to see the dining room extension reflecting on the practical floor. It married the scenarios together.
And, to be honest, it wasn’t really running in real-time until the very last few days before the shoot, because we were refining, and refining, refining just to make sure that last minute changes of colors, and matching pieces, and tablecloth, and decorations, and glass and everything, was working. Plus we had crowds in there as well in the background. We spent day and night making sure that it ran in real-time.
Connor Ling: I think the dining room worked really, really well. I think that the blend between virtual and real on that setup, you couldn’t tell. The lighting and the foreground set was stunning, and everything just worked really well for that. It was difficult, because the optimization for that scene was critical. That setup was expensive, in terms of frame rate.
Freddy Salazar: It was maybe one of the most scary ones at the beginning. Because we said wow, this is tricky. Even in VFX, we know when it’s an interior like this with wood, a bit reflective, it’s never simple.
On-set rain, but in a volume?!
Christian Kaestner: Bo said at one point, ‘Of course it’s going to rain, we need heavy rain!’ And everyone said, ‘Well, these are not waterproof panels, they’re not outdoor panels.’ However, we had a fantastic special effects supervisor who was tasked with the impossible, and he made it happen. He designed a very clever modular rain rig that was suspended through the skylights. It was designed in a way that we had very good control.
Just with the physics, you can calculate quite well how much the spread is going to be unless something horribly goes wrong, and a pipe completely bursts or something. Ultimately we had seven rain rigs that we could turn on based on where we would shoot. The biggest challenge there was that we had to be careful with the wind. We had to not blow the rain in into the wall.
Connor Ling: It really helped sell the immersion of the actors, and looked incredible on the stage. Doing it in a manner that meant that the LED was protected, and that it was easily able to be turned on and turned off, and everybody would be safe, was really important. The fight scene on the boat–spoiler!–that was all practical rain shot against LED and it looks fantastic.
