Here’s what made the 2D animation in ‘Klaus’ look ‘3D’

The art and tech of the film, including behind the scenes videos.

Ever since glimpses and tests began emerging from director Sergio Pablos’ Klaus, people have been wondering how the look of the film was being achieved. Was it 2D? Was it 3D? A combination of the two?

befores & afters was able to sit down with Pablos at the VIEW Conference in Turin to find out the secrets behind the production, which starts streaming on Netflix on November 15th. This included the new tools that were developed to effectively layer in light and shadow on 2D-animated drawings to provide for an almost volumetric appearance.

Here, Pablos breaks down how the unique style of the animation in SPA Studios’ Klaus came to be, its origins in wanting to surpass the current limitations in adding shadows to 2D animation, to tests at SPA Studios, all the way to the final film.

Updated below with behind the scenes video of the toolset developed with Les Films Du Poisson Rouge.

The quest to do something different in 2D

Sergio Pablos (director, Klaus): I knew I wanted to demolish the limitations that we traditionally had with 2D animation. I always knew there was someone out there who had a solution for this. I needed to find that person. I spent months reaching out to different programmers and developers. I found there were a lot of semi-automated attempts at doing it, but they didn’t tend to give you that control.

Meanwhile, one of my employees, Marcin Jakubowski, who at the time was a concept artist, was looking over my shoulder saying, ‘Hmm…give me a try’. And then he comes back and says, ‘I think I figured this out.’ He sent me a bunch of images of cubes and spheres overlapping. I said, ‘I don’t know what the hell I’m supposed to learn from this!’ He said, ‘Give me something to try it on. So I said, ‘Well, the simplest thing I can think of is a head turn.’

So I sent him an animated head turn – about 16 frames – and he brought back this thing that looked 3D. I said, ‘Well, I’m not sure how you’re doing this, but let me give you something more complex.’ So I gave him a walk cycle, and he knocks it out of the park.

Then I said, ‘I’m going to give you an actual shot, where a character walks into a room and there’s a different set of lights. If you pull it off, the question is, is this something you can do, because you’re a freakin’ mad-genius or can we actually teach other people to do this, too?’ The irony was, I was reaching out to people in the US and Canada and Australia and the guy who had the solution was in the next room.

A Bifrost journey

A production solution

That proof of concept was incredible, but it was a time consuming process. This is when we reached out to Les Films Du Poisson Rouge. They had been developing tools for 3D and 2D for a long time. We established a partnership. It was Marcin’s recipe and Les Films Du Poisson Rouge’s proprietary solutions – we merged those together.

Les Films Du Poisson Rouge figured out how to derive a tracking system from drawn lines. Not just vector drawn lines, but even bitmap lines. What came out of the partnership was this insanely intuitive tool that works in real-time and it allows artists to do an amazing amount of work in not much time. We called the tool ‘Klaus Light and Shadow.’

The other extremely important part of it was Toon Boom – the whole pipeline is built around Toon Boom Harmony. Everything from storyboarding through to ink and paint was all based on the Toon Boom pipeline.

The process before lighting

We would storyboard with Storyboard Pro from Toon Boom. The great thing was, from there we could go straight to layout. We did layout digitally on tablets. We would split layout depending on whether it was a 2D shot or 3D shot (see below). From layout we did animation directly in Toon Boom Harmony. No tricks. No puppet animation, it was all hand-drawn frame by frame. We had a clean-up team just like the old days where assistants and in-betweeners would do all the rest of the drawings and nice clean lines.

Then it would go to ink and paint where they would treat the lines. That was probably the trickiest part. We only kept lines where lines were required to convey information. So, you’ll notice a lot of outlines are gone because we felt that if the information with the contrast and the value of the color had already been given, then you don’t need a line to do that. But you’ll still see internal lines in hands or a face sometimes. So it was very selective about how many of those lines we actually needed and how many of them we could replace with shadows later.

The lighting breakthrough

We had a color bible for the whole film and we also had a color script. The color script didn’t have quite every shot in the film but I would say at least a shot out of every four. We needed to have a lighting reference for the background and the characters that was very cohesive so that both the background painters that were working on the backgrounds and the lighters on the characters were drawing from the same source.

The lighting process was essentially about breaking down the lighting of a scene in a convincing way, the same way that concept artists do every day. We’d introduce up to eight layers of lighting, it could be anything from ambient occlusion, sub-surface scattering, rim light, specular for the eyes, bounce light, etc. Each one of those layers would have a set of shapes that the artist would create underneath the layer for the shot. They will go through whatever number of layers they had using a tracking system that was part of the tool that Les Films Du Poisson Rouge brought to the plate. Then you merge them to get the ‘baked’ lighting look.

After lighting, there’s another step which we call ‘texturing’. But it’s not the texturing you know from 3D. It’s another tool from Les Films Du Poisson Rouge and it’s called M.O.E. It allows you to pick any painterly style like wash, watercolor or oil, and decide the behavior and size of the strokes and apply it to the image. We just wanted a certain level of grain because that’s the same style as the background. It’s very subtle – you almost have to squint your eyes to see it. What it does is, this grain actually travels with the character.

Then all those elements go to compositing. Comp decides how much of the texture we apply to each of the shapes. Because there’s not the same level of texture for the skin etc, the comp’ers can control that and they can decide what works for each shot.

3D still played a large role

We did many 3D elements in Maya. Sometimes elements, like the reindeer, were in 3D and sometimes they were 2D. I would say to my 3D team, ‘Can the rig do this?’ And they would say yes, or no, or say it’s easier to do in 2D. Sometimes we would animate the reindeer in 2D but the fur would look wrong, so we would just take the fur out and we would paint on top of that.

A tool for artists

Using the new tool was a joy for the artists to work on. It was actually hard to get them to stop going too far because they could easily keep adding bells and whistles to the animation. It came to a point where you had to go, ‘Well, that’s a bit much.’