The new tools that Walt Disney Animation Studios used to give ‘Wish’ an old-school feel.
Chris Buck and Fawn Veerasunthorn’s CG-animated Wish, from Walt Disney Animation Studios, tells the story of a teenager called Asha (Ariana DeBose) who makes a plea to the stars to help save her fellow Kingdom of Rosas citizens from the kingdom’s ruler, King Magnifico (Chris Pine).
Seeking to pay homage to the days of traditional 2D animation techniques at Disney, the studio employed several modern techniques to bring a number of stylized and illustrative looks to Wish.
These included line work additions, simplification of detail , edge pooling and color bleeding, layered effects to provide for a more textured and watercolor feel, broad washes, and an approach dubbed dynamic screen space texturing.
At the VIEW Conference in Turin this year, befores & afters sat down with Walt Disney Animation Studios visual effects supervisor Kyle Odermatt to discuss these techniques used on Wish.
b&a: Clearly the intention of the film was to throw back to some of the great Disney animation techniques of the past, and I think to adopt some of the more ‘stylization’ techniques we’ve seen in recent times. But how did you solve this approach to stylization? When you came on board, what were the first things you had to consider?
Kyle Odermatt: Looking at the illustrators that they wanted to be inspired by, the ones that inspired Walt back in the day–that style of watercolor illustration with inkline over the top–was something very, very different from the look of our current films. It’s always a back and forth with the production design team to figure out what it is they like about that inspiration and then trying to distill that out into a look that we were going to design.
We don’t just want to take a style and then recreate it exactly. It all involves subtle reinterpretation into something that’ll work for a feature film that you want to watch for 90 minutes. So you can come up with a really cool style, it might work for five minutes in a short, but might be fatiguing over the course of the entirety of a feature film. So, that was one of the early considerations.
The other thing we did is to find ways for each song to have its own little flavor within the style. So not only did we have to figure out the overall feel but how each of those were going to be treated in a slightly different way. Some of it’s just lighting, some of it’s just performance.
Probably the biggest challenge to solve was bringing in the feeling of watercolor paper texture. For each of these frames of the film, we wanted it to feel individually like a watercolor illustration, but the thing to solve was how could that work with dynamic cameras, with performing characters? It’s something very easy to do in Photoshop by putting that layer right on top of the layer stack, but very challenging to do on moving imagery.
b&a: Yes, you can get into the screen door effect with that, can’t you?
Kyle Odermatt: That’s right, the shower door or screen door effect where you see something moving either through a texture or behind a texture. That’s the one thing we knew we couldn’t have, and yet if we did it without texture we knew it wouldn’t feel exactly like a watercolor illustration that we were trying to be inspired by.
b&a: I’ll ask you more about that challenge later, but just on the idea of going back to a stylized approach. I mean, with Disney’s renderer Hyperion, we know you can generate incredible detail. But with the look of this film, I feel like you had to ‘take away’ detail to make it look more illustrative. Is that what you had to do?
Kyle Odermatt: Well, what really happened was that compositing became a much bigger part of the process. If I showed you a raw lighting render out of Hyperion compared to the final film, it would look very different. We knew that our end look would be very composite-dependent to achieve the overall look because we’re taking a bunch of calculated pieces out of the renderer and then combining them in ways through the compositing operation that gives us that overall feel.
So, an image that comes straight out of the renderer doesn’t look like that at all. In order to not fool ourselves throughout this process, we came up with a way to generate our best guess at a single pass render and composite that would make it look a bit like the style. We called it the auto comp process.
For every render we did on the film, whether it be an animation test or for this intermediate process, we generated this auto comp so that we could do intermediate single pass renders of all our tests and then have them go through a comp process that would put some of the line work over the top. That would do some of that lighting simplification and would give a bit of that textural feel just so we were not making the wrong choices about performances or early lighting passes before we got into all the sophistication possible with multipass rendering in lighting. We always tried to get away from the standard CG feel because we just knew that’s not what we were trying to do.
b&a: How would you describe that, exactly, ie. getting away from the CG feel? Because it’s ‘simpler’, I suppose, and you went through a simplification process, but it is not ‘plain’ at all.
Kyle Odermatt: It’s interesting because some of the original early reactions to the trailers were that we were doing hand-drawn animation or that we were doing 3D animation to make it look like 2D animation. However, what I felt like we had to do throughout this process was design a look. This had to be something that didn’t pre-exist. I think that’s what every movie’s doing these days where they are pursuing a style. I’m so excited about the state of the industry in this way. I’m a movie fan. I love going to see movies in all these different styles and every one of those teams is going down this process of trying to design a style. So, we knew we had to do that as well.
The degree of real lighting versus more stylized lighting was something that took a long time to get to the sweet spot of what we felt we liked. It is so easy to read character performance in this film that I think the teams did just a fabulous job of coming to a perfect sweet spot where it’s just wonderful to watch the characters. They have a lot of clarity like the hand-drawn characters had back in the day but yet there’s so much more nuance to them.
One of the differences, of course, between the hand-drawn films–which I worked on so many of–and this, is the integration between the characters and the backgrounds. It’s so much more tight. They fit well into the backgrounds. We’re using all the same techniques to achieve the style on the characters that we’re using on the backgrounds. This means they feel like they’re in the same place in this design world that we’ve created.
b&a: Tell me a little more about how you used compositing in a bigger way on Wish?
Kyle Odermatt: So, first, if you’ve ever looked at a node-based compositing, if you zoom out the whole thing, it looks like a bunch of tiny little specks because there’s so many nodes and so many operations strung together. We found that we made a very significant trade between compute time on the rendering side and compute time on the composite side. So our composites were far heavier, far more expensive than any film we’ve done before.
On the contrary, our render times were actually quite a bit lighter. There’s no motion blur used on the film except in very small ways when there was a little bit of strobing and that was done in comp. There’s no rendered motion blur on the film. There’s no rendered depth of field. Both those things make the renders go faster.
All the light simulation is calculated properly, but we actually take out so much of that in the composite actions. And then we do all these stacked and layered filters so that the image processing in the comp is used to achieve the final look. It tried the patience of our lighters and compositors for sure, because they’re used to waiting a very long time for their renders and then being able to quickly tweak the composite. Here they got their renders back quickly and then they had to figure out ways to be able to work with these very intense comps.
b&a: I was thinking line work might be one of the trickier things you had to tackle, because it is something you can dial in so much in comp. But I know that can be tough to achieve as solely a 2D solution.
Kyle Odermatt: For line work, we pursued two different possibilities. We actually added line work rendering to Hyperion. We built a whole new piece to Hyperion that is very exciting that we’re going to use in many ways in the future. We’re just trying to figure out all the cool ways in which to use it, where you can generate samples and then be able to operate on the samples in any way you care to. One of the things you could do at that stage is generate line work based on the sample data, which gives you so much more information than you have on the compositing side where you just have pixels.
So, we pursued that, thinking that, from a performance standpoint, having lines generated by a renderer would be more efficient. We were on the cusp of delivering that right when we were ramping up into production. I think there’s a shot or two in the film that uses rendered lines but most of them are generated in compositing. That gave us a great deal of flexibility but it also added to the expense and the intensity of the compositing part of the process.
In comp, we had four or five different thresholding operations that could generate lines. The difference between two different objects would generate a line. The difference in normals, so where curvature changes shape, would generate a line. Depth discontinuity would generate a set of lines. So we combined all these things to get an overall line pass and then we had a bunch of different ways to tune where we did and did not want the lines to occur, including attenuating them with increasing depth in the scene.
b&a: I guess that lets you also make very artistic creative choices about suppressing lines or removing lines to again give that illustrative feel, right?
Kyle Odermatt: Right. Also, at times, the simplest possible way to get the lines that an artist would have wanted was to just draw them in. We had a team of lighters do this. They were so artistic that they would just go in and add a few little indicative lines, exactly like the illustrators did back in the day. That was just the easiest way to do it because the alternative was to generate a ton of lines and then try and erase most of them through some operation. Indeed, a lot of times it was just easier to get rid of all the actual computed lines and just go add in a few. That made all the difference. In fact, we even did that on some character shots, particularly with Magnifico which gave some great emphasis to our villain.
b&a: There’s a very layered feel in some of the scenes and you mentioned something about screen space ambient occlusion, but not what everyone might know as ambient occlusion.
Kyle Odermatt: Yes, the difference between what we were doing from ambient occlusion was the fact that it had a directionality to it. When you think of a general ambient occlusion pass, it is just about the proximity of things and how close they are together. Here it was how they related to the camera. That’s the difference. It gives you essentially an outline of anything against the thing behind it, which we call screen space ambient occlusion. It allows you to make it feel like things are pulling apart from each other, which sometimes you want and sometimes you don’t.
That was the magic of what our lighter/compositors could do. They had all these things to play with and remarkably they were able to play with them in relatively efficient ways.
b&a: You mentioned something in your talk about using broad washes in scenes inside of volumes to give an atmospheric look. How did that work?
Kyle Odermatt: Normally, if we were doing a forest and a more realistic look, we would just throw a big atmospheric volume in there. That would take hundreds of hours to render. Here, instead, we would put in a card that had a colored wash with a bit of texture to it and that would suffice because it gave the perfect look, and it’s actually what an illustrator would’ve done. We allowed ourselves to really take that big step back and think about what was necessary to achieve the look and not just go back to the way we’ve done things before.
b&a: Then there’s that paper texture and watercolor look we had been discussing. What were the different techniques that helped you create that texture and have a feeling of it in the final frames?
Kyle Odermatt: We had what we called a dynamic screen space texturing system that calculated the right look per frame–this is what was aimed at removing the screen door effect. Imagine if you looked at a shot from a different point of view, if you had a camera looking at a ground plane that was going off into the distance and wanted everything to look the same at the screen, the texture has to get bigger and bigger and bigger and bigger the further it goes away, so that when it comes back to screen, it counteracts the perspective of things going away and getting smaller.
If you looked at it from a god’s eye point of view, you would see the texture that was quite fine close to camera getting bigger and chunkier and chunkier and big and huge in the distance, so that when you are looking at it through camera, it feels the same at the screen plane. That sounds easy but to get it to do it in a way that was seamless was a remarkable development.
b&a: I was curious with Wish if there was any time or particular development in some of these tools to implement any machine learning workflows. Obviously the machine learning denoiser might be part of that but was there any other kind of ML tech used?
Kyle Odermatt: One of our internal developers, who was very involved in our implementation of the machine learning denoiser and the way we use it in our post render process, helped us with some ML efficiencies. One of the aspects with the line work that we were generating and some of the texturing, even, was that we would get things that were temporally noisy. Although we were generating higher resolution passes to be able to do things like line generation, we would still find that there were things with temporal noise. Sometimes it was with architectural elements where they were just not quite straight on screen. So you would have things that were offset by two pixels over the course of the entire width of the screen, and as that moves, you can see it just alias and chatter. He was able to incorporate the concepts of the machine learning denoiser straight into the Nuke context. So we could run things in the compositing tree through a denoising pass that uses multiple frames.
Our current standard is to use the denoiser over a seven frame increment that uses motion vectors over those seven frames to know where pixels move over those seven frames, three before and three after the given frame. And so that denoiser in Nuke cleaned up all the noise whenever we had that chattering and allowed it to be of production quality.
b&a: Right at the beginning of your presentation you mentioned something about the unique aspect ratio of the film, going to 2.55:1. What challenges does that widescreen presentation bring to you as a VFX supervisor?
Kyle Odermatt: Not a whole lot, except that there was a concern about theatrical presentation and whether it would be presented properly because it’s less common to have that ratio now. Creatively, we wanted to make sure that once we made that choice that we then exploited it properly. It doesn’t make any sense to go extra widescreen and then have compositions that don’t take advantage of it.
Ultimately it was something that we had to push on initially, where if you want to feel a separation between characters because emotionally there’s that separation and you have that extra width, you want to make sure and push them out wide and use that. Thankfully, we don’t have to do a four by three version anymore for airplanes or anything like that, which always made using widescreen aspects challenging.
b&a: And just finally I think you recommended people go and see this in stereo if possible because you feel it really pops that way.
Kyle Odermatt: Yes, I think it’s fascinating to see this look that is effectively flat in nature and inspired by vertical illustrations, to be adapted to a widescreen cinematic format and then be dimensionalized. Yet, it still works perfectly from an integration standpoint, partly because of the innovations we talked about with the texturing and so forth. It also looks remarkable in theatrical 3D, and I encourage people to try and go see it that way.
