That was just one of the major challenges Sony Pictures Imageworks had to tackle on the film.
Having already dazzled audiences with Into the Spider-Verse, Sony Animation and Sony Pictures Imageworks upped the ante even further with Across the Spider-Verse, introducing more characters, more worlds and even more diverse styles for this new film.
Taking on the animation reins this time around was head of character animation Alan Hawkins, who befores & afters last spoke to about his work on The Mitchells vs. the Machines.
Here, Hawkins relates the fresh challenges involved on Across the Spider-Verse.
This included dealing with variable frame rates amongst different characters and even within characters themselves, achieving unique behaviors for these characters, new tools that had to be developed, and some fun moments where a few Easter eggs could be introduced.
b&a: What were the main early considerations, from an animation point of view, that you had coming into this film?
Alan Hawkins: Well, I didn’t work on the first one at all, so it was a lot of learning for me. When I joined, there had been some tests done–they tell you just as much what not to do as what to do. On some of the previous movies I’ve worked on, there is always a certain ‘style’ the filmmakers go for. Surfs Up was a mockumentary, and they didn’t want it to feel very animated, they wanted very realistic performances. With The Mitchells vs The Machines, it was a little bit more exaggerated, but also grounded in very real human acting and stuff. But with Across the Spider-Verse, this is the most, I would say, mature performance desires than anything we’ve had.
Things like little tiny tells that a human face will have were way more part of the process here, because they had anatomy that a cartoonish character won’t have. So when you have all these little facets, and angles, and muscles, and you do something like a little tiny squint with just something like that, that sends a message that maybe you can’t send if a character has big round eyes that don’t really exist on human faces.
It was really fun and exciting for me to get to delve into that degree of specificity. Not just that, but with every character in this movie, every performance is multilayered, no one is ever just happy or just sad. They’re happy, but they’re sad about not being able to do something else. Or, they’re sad, but they’re happy about some kind of secret that they know that no one else knows. No one is ever just being honest with each other, which is something that happens in real life, too. Those little complexities were an awesome playground.
b&a: Because there are so many different Spider-Men and so many different characters, they all not only have different palettes, environments, and colors, but they also have different movement and behavior requirements, even in the same frame. How did you achieve that?
Alan Hawkins: There’s obviously the looks, the rendering, the art styles of each character that are sometimes very conflicting with each other when they’re in the same space. I always felt like one of the strongest examples of that was the Vulture in Gwen’s world. He’s so parchment-based, and sepia, and Da Vinci-esque in his looks. And then her world has all this bleeding color, and glow, and bloom, and all these ever-changing light and color themes, and you kind of wonder if they’re going to mix. And thankfully, the production designer Patrick O’Keefe, and the backend teams led by visual effects supervisor Mike Lasker, were able to find the right balance of how these things could fit together.
As far as the motion goes, I think we were always trying to find a way to make it feel like these worlds are colliding. One good example is Hobie, who needed a whole other motion language than Miles, or Gwen, or anyone else. His world is a zine art style, a lot of Xerox stuff, a lot of patchworky collage, very scrappy. Just looking at his look, that all makes sense.
For how it moves, we tried a lot of stuff, there were a lot of tests with him at the beginning. We put it on the shelf for a while and then we came back to it once we had actual shot context to try to figure this stuff out. When you’re doing tests you can go in any direction, there’s no parameters. But having an actual scene, we’re like, ‘Okay, here’s what he thinks, here’s what he feels. Now let’s figure out how that translates to motion.’ It was necessary for figuring him out. He was one of the last characters that we figured out, and it was only when we were in shots with him.
Here's a (recreated) diagram of how the 2D elements are set up in one of my Maya scenes for my Hobie shots! Normally, you would do this in post with compositing software, but since we needed to have all the animation represented for approval in playblast, this is what we did… pic.twitter.com/mSmhZQlkOV
— Li Wen Toh (@LiWenToh) June 8, 2023
Ultimately what we found worked was a combination of different frame rates for different parts of his body, which no other character has. They’re either 2s, 3s, 1s, whatever. But his jacket is on 4s, his body’s on 3s, sometimes other parts of his body are on 2s. And then his guitar has the lowest frame rate, it’s a lot more, it’s 6s sometimes. That meant we had to have multiple rigs that all had those different settings on them so that it could be put together. He was probably one of the most complicated characters because he had so many different literal layers to him.
b&a: So the way you’ve achieved that look is variable frame rates within the character itself, but then he occupies a frame clearly with characters that might be on their own different frame rate. How do they fit together?
Alan Hawkins: You have to be careful, because if a character like him changes very suddenly it pops a lot more, whereas another character will have more inbetweens, you have more time to process it. We were all very sensitive to the fact that if we did too much with Hobie, he would steal the screen. I mean, he does in a lot of ways, a lot of other ways he steals the show. But visually we made sure that we don’t look to him every time he moves or does something because he’s popping so much.
We made sure not to change his render style too much within a single shot, for example. I think it works because people accept it. Once you establish a language for a character, as long as you stay within those rules and you put them next to each other, people just get used to it.
b&a: I think some casual filmgoers might not quite appreciate that the animation in this film tends to start as CG animation, because so much of it of course has a ‘2D’ look. Tell me about the workflow from CG to the final look.
Alan Hawkins: I mean, there is a lot of 2D involved. Some of it made it into the film, some of it didn’t. There’s actually one shot that was animated by supervising animator Humberto Rosa. There was a scene where Miguel is about to bite the Vulture, and you don’t see his face, it’s the fang shot where he’s going to bite him. Lighting wasn’t quite nailing what we had intended for this kind of shadowy vampire type look to him, so Humberto did these amazing draw-overs. And production just used it in the shot, they just comp’d it over the CG. So, Miguel’s face is 2D in that shot.
Some animators here will pencil test things out before they start blocking–those who are more comfortable with 2D, which means sometimes a 2D pass will get approved before it gets animated in CG. Sometimes they just sail right through after that, or sometimes there’s still notes.
A lot of shots in the film only went as far as playblasts in animation, and then they were actually completely handled by other vendors. The sketchbook sequence, where Miles is catching us up on, ‘I’ve been Spider-Man for this long’, which is intercut while he’s fighting The Spot, those were only ever playblasted, they were never lit. Then Titmouse and Buck did draw-overs on top of it. They made it look all nice, and markery, and sketchy, and it’s awesome. But those were not renders, those were actually drawn over. It was the same thing for The Spot visions, when it’s all black and white, they only went as far as animation playblasts and they were drawn over. So I think, if anything, this will probably confuse people more because it is more 2D involved than ever!
b&a: You mentioned The Spot there. That’s a character where I think you needed to find a lot of goofy moments, but then he also needs to appear threatening. How did you approach him in animation?
Alan Hawkins: Yeah. I mean, I love his trajectory as a character starting off goofy, kind of being a laughing stock. And then once he gets his power, he becomes really scary, and spooky, and intimidating. That always gives me goosebumps when he gets powered up and he becomes really threatening. So just having that range is really nice. We had a couple of mid-steps along the way. After he does his own mini-collider, we lose him for a little bit, we find him again in Mumbattan. We changed the way his spots are oriented on his body. They used to be very just kind of randomly, evenly spaced around him, but we consolidated them around his torso, and there’s more of them. So he’s kind of got this black band around his body, and that’s to show that his power is gathering. It also made him look a little sick with power. It made it feel like he’s inflicted with it, which I thought was interesting.
We also treated him with more confidence during those scenes. He’s saying basically that, too. He is saying, ‘I’m in the zone.’ And he’s cracking his own jokes and stuff. So we changed the way he acted there instead of being nervous in the early sequences. When he is in the bodega, the first time he meets Miles, he does a lot of awkward posing, which actually also came from Humberto. He was inspired by Egon Schiele self-portraits, where there’s a lot of contorted posturing, a lot of discomfort in his own body. We used that in the early stages, where, even when he’s trying to look cool, he doesn’t know how to do it, he’s kind of tense.
There was actually an early version of him that was developed before I came onto the project. He used to have this animated face that was very Venom-like, which I think was the main problem because it was very malleable and changing. And Bob Persichetti and I, when we started figuring him out, decided we wanted to really boil down his language, and it felt like it was too many things. So it’s like, ‘If he is The Spot, he’s just spots, we’ll get the acting with his body and his posture, and it’s almost better with him as a blank slate.’ Sometimes the animators wanted to do things like make an angry shape of the spots, and we resisted that almost entirely. That way it all came from his body presentation and the dialogue.
b&a: The Indian Spider-Man, Pavitr, I love the movements, as in the fighting style of that character. How did you come up with those movements, or look to reference, or take it even further for a more stylized look and feel?
Alan Hawkins: A lot of the guidance for him came from Joaquim Dos Santos, he was really into that character, in particular. He showed us all kinds of reference of this fighting style called Kalaripayattu. It’s all about flexibility and it has a lot of dance-like elements to it. But also it’s attacking, and swords, and punching, and kicking, and all that stuff, it’s almost like fighting yoga. That told us about fluidity of his motion, it told us about how to bend his body in certain ways, but then we also had to keep it consistent with somewhat Spider-Man type language, so it’s picking where to inject stuff and where not to. I mean, each character has their own little story that we used to then make them feel more different than everyone else, and that’s his, that’s where it came from.
b&a: When we talked about Mitchells, I remember there were some new tools that Imageworks developed specifically with animation in mind? What new tools were needed for animation for Across the Spider-Verse?
Alan Hawkins: The one that got used the most is the foundational one, was how to do things on 2s. When that happened on the first film, no one was really ready for it. And so the solutions for how to achieve that look, the variable frame rates and all of that stuff, were very scrappy, and very messy, and very inconsistent. So everyone walked away from that film saying, ‘Okay, on the next one we got to figure this out. We need a system.’
What we developed, then, was called StepSets, and it was a way to make what the animators were creating play nice with the CFX department, to play nice with the final layout departments, and all the lookdev and everything else. While that’s not a tool that you would use in a scene like, ‘Oh, I have this special character, I’m going to use this special tool for it, ‘it was a part of the process for every shot, so it was very fundamental for it. It’s used in every single scene in the movie.
When we publish a scene, that means to send it to the downstream departments. If we put everything out on 2s to the CFX department, and they start trying to do a cloth sim and it’s got every other frame, the cloth would jump every other frame as well. So when it gets published, it actually gets published every frame for that department, and then they run their version of the tool, which then brings it back to the 2s, or 2s, or whatever the correct number of frame rate is. So we had to have this way where we gave all of the information, but then only selectively so they could use it, but then selectively boiled it back down to what the intention was, and so it affected every department differently. It’s funny how fragile the pipeline can be, and something like there being a lack of information between frames can blow things up, and this was to fix that.
b&a: I wanted to also ask you about line work, which of course is something achieved in the first film, and in some other Imageworks projects as well. What were the challenges of solving line work on this film in an, I don’t want to say ‘automated’ way, but to make it as automated as possible?
Alan Hawkins: Yeah, I mean, as automated as any of these things can be, where there’s a system that exists and it has to be run through that process. There’s nothing that’s free that just runs through, it takes an artist to make sure it still works and looks good. But there are two forms of line work in terms of literal ink lines. One is what you might call automated, but it’s actually done by the FX department. And so those are angle-based things, where if you turn one way, a chin line might draw. And then if you turn away, it may shrink down or disappear entirely. Certain things that were all added in the render, these lines on Miles’s or almost everyone’s nose bridges, there’s certain ones that were done by the FX department. But then there were animation inklines when we wanted a special expression, when someone was particularly tense, so we would draw extra things there.
If it’s a smear frame, we would do those. So there are definitely animation specific lines, and then the kind of render effect lines that would happen too. And towards the end of the process, actually, we were so tight on time that animation had to take over all of the ink lines. FX handled so many different aspects of the look of the film, and as we were getting closer and closer to the deadline, animation was getting less and less busy, but FX was still very, very busy. And so we ended up inheriting all of that job. There are maybe a hundred shots in the film where animation did all of the lines, and you probably wouldn’t know the difference between when we were drawing them and when they were drawing them.
b&a: Alan, I think I saw on Twitter that your face got put onto a dollar bill. Is it when The Spot is trying to get the money from the ATM?
Alan Hawkins: Yeah, that was a gag that the animator did just for our internal animation rounds. And then Patrick O’Keefe thought it was funny and just kept it, and he put it on the actual money. And it’s blink and you miss it, but it’s just my Imageworks photo from the website, they didn’t do anything to make me look presidential or anything like that, it’s just my face.
OMG, they put Alan Hawkins, the Sony animation supervisor's face on the dollar bill!! Im crying this is such a great easter egg. 🤣🤣#SpiderVerse #SpiderManAcrossTheSpiderVerse pic.twitter.com/Xk5quciXER
— Arran Baker (@Arran_Baker) May 31, 2023
We love making generic characters of people, of ourselves, of bosses, things like that. So there are versions of [the directors] Justin, Kemp and Joaquim, and [producers and executives] Bob Persichetti, Kristine Belson, Pam Marsden and Michelle Grady, and then myself, all throughout the film as generic characters in the backgrounds.
Christine and Pam are in the coffee shop when all the bubbles and all the soapy water is on the ground, they’re standing back and watching. There are so many Easter eggs that we would put in there that I’m sure will all come to light, but the dollar bill is one of them, yeah.
One of our animators, Daniel Ceballos, has a Puerto Rican mother, and he animated a lot of the Rio shots. And when she snaps her fingers at him instead of an ink line thing, it’s a little Puerto Rican flag that he popped on there.
b&a: Did he do that just on his own accord, or was it something discussed?
Alan Hawkins: Oh, no, he just did it and it made it through. It reminds me actually of how things actually evolve, or stuff starts off as a gag that can then become more of a set piece.
For example, the little showdown that Miles has with the Web-Slinger, the gunslinger, as he’s escaping and they’re on the horses and they do the showdown, that was never written in, we developed that in animation. First it was, he just landed, and then they had a tense pause, and then there was a quick flip, and that was it, it was one shot. And then we split it out a little bit, it grew a little. I didn’t see this until it was done, I thought it was all going to be in the regular environment. But then the backend departments were inspired and said, ‘Oh, let’s put an actual Arizona set behind them.’ Then there was the music added and it just went further than was ever imagined. It’s not so much an Easter egg, but just an example of the collaborative process, where they’re combining everyone’s ideas to make everything as good as possible.
