How Imageworks made the linework tool Kismet. A new excerpt from befores & afters magazine.
During production on the first Into the Spider-Verse film, and its sequel, Across the Spider-Verse, Sony Pictures Imageworks leaned heavily into the series’ Spider-Man comic book origins, in particular, linework or inklines.
This was the case even though the films are 3D animated. But the fact that they are 3D meant linework required special attention, partly so it could help highlight character and location features, partly to retain a hand-drawn and sometimes messy feel for the film, and partly so that the linework could simply be achieved on top of moving geometry.
For Across the Spider-Verse, Imageworks developed a specific linework tool called Kismet inside of Houdini to deal with such diverse imagery as Gwen’s world of Earth-65, Mumbattan and the creature Vulture. The idea was to allow artistic control of different kinds of lines, giving the lines the ability to ‘redraw’ continuously, and to enable a procedural workflow.
Behind the tool was FX supervisor Pav Grochola, who shares with befores & afters how Kismet was born (including where the name came from), what it enabled artists to do, and how it helped generate a hundred million lines on the film.
b&a: Why is linework so important to a film like this?
Pav Grochola: We were tasked with matching the visual style of comic books, where linework plays such an important role. Comic books even employ a specialized artist known as an ‘Inker’ who exclusively focuses on creating linework. So we knew linework was going to be a really huge thing to figure out. I’m glad we took it seriously because it played a critical role in the look. I was in a lot of reviews where linework was missing from comps, usually because it hadn’t been completed yet. The comments were always like, ‘Something feels amiss…it doesn’t feel right,’ or, ‘It looks excessively three-dimensional and lacks a handmade touch.’ It was only when they realized that the linework was missing that the pieces fell into place.
b&a: You did work on some linework aspects for the first film, but tell me how things ramped up here for Across the Spider-Verse?
Pav Grochola: On Spider-Man: Into the Spider-Verse, we were spending most of our time in Miles’ world. We didn’t go anywhere else. On this one, we visited a number of different worlds, and each of those worlds had a different style. All of them had linework as a really important visual feature of the world.
On Into the Spider-Verse, linework was limited to the characters’ faces and hands. While on this film, we also had whole characters that required linework, head to toe (such as Vulture and Spot). If that wasn’t complex enough, we also had to generate linework for complex environments, while matching a particular style. All this added up to be a much bigger challenge, and meant that our toolset had to be very flexible.
b&a: How did you come to make a tool to deal with this linework?
Pav Grochola: My background is fine art (specifically drawing). So, I’ve always been interested in the challenge of making 3D look more like traditional 2D media. It’s actually been a hobby of mine for many years. When I saw the artwork for this film, I was very excited to test out some of these ideas and volunteered a solution.
Generating linework from a 3D model is not a new idea, but the way we approached it was novel. In the past this was achieved by an edge detecting gag, in comp or in a shader. This approach always falls short because this is not how artists draw! While drawing, they make very complex choices and prioritize artistic intention over simplistic rules. The goal was to create a toolset that would empower artists to fully express these complex creative choices. That’s why we chose a Houdini based solution. It gave us the controls we needed for artists to fully express themselves creatively. It’s the overarching goal of the film—to see genuine artistic expression on the screen.
b&a: What workflow did you come up with in Houdini to deal with linework?
Pav Grochola: One of the hardest parts about developing a tool like this is that there are multiple problems to solve. How do you get it to look good in the first place, and how do the lines update as the character is animating? I wanted to leverage all the stuff I know about Houdini and all of its procedural capabilities that let you add variations, while keeping things flexible. I knew any tool I built would have to be used on different types of characters, different types of environments and different types of situations with different animations. For example, the linework on Spider-Punk is very different to the work on Spot, which is very different to Abyss and very different to the India world.
Houdini was certainly the right tool because of the proceduralism and flexibility. The other advantage that Houdini had is that we were ultimately outputting curves to represent line drawings. That means our linework can sit in 3D around a character’s silhouette. We could very easily just offset it from the surface, which is a big part of what makes the linework look better, when it’s actually not perfectly married to the edge of the characters. It also meant that lighting could easily make adjustments to curve thickness or even remove curves on their side.
It takes a village. I developed the core tools, but once you get into production there is always room for improvement. Edmond Boulet-Gilly did an incredible job supporting the team and creating additional tools. He also spearheaded an approach to using Blender to hand-draw lines on set keyframes, while Filippo Maccari figured out a way to interpolate between these keyframes. Things really came together when we combined the hand-drawn Blender lines with the procedural Houdini ones. It was the best of both worlds. The hand-drawn lines gave us the beautiful design and hand-made feel, while the procedural lines gave us detail and complexity.
Pipeline is critical when you have 1000 shots to do. Filippo Maccari set up a template system. This meant an artist wouldn’t be sitting there every single shot dialing in linework. Instead, we could create these templates. We had a template for Spider-Punk and we had a template for Spot, and we would just automatically run these templates in the background when a shot came in from published animation.
It would be a command line prompt, and it would just get launched in the background—there was virtually no artist intervention at that point. It was really important to pipeline that stuff so we could get through the amount of shots we needed to.
b&a: You mentioned lines on character faces before, and I remember some of the discussion from Into the Spider-Verse where Imageworks used some automated or machine learning approaches to help draw some face lines. Was that part of the mix here at all, too?
Pav Grochola: That tool for the first film was trained on examples of the linework on the character’s faces. We still used this tool extensively on this film and Edmond found a better way to extract predictions. This tool was also used for all the one-off requests like ‘Oh, we really need a line along this fold in the garment.’
Kismet was a totally new thing, though. It’s much more about procedural creation, where you don’t really manually draw anything. It’s all procedural dials or sliders that the artist would control. Generally, we tried to template it as much as possible because there were so many shots to do.
b&a: And as you said before, it was extended to linework on environments and moving vehicles and whole bodies.
Pav Grochola: For whole bodies of characters, originally, I only thought it would be used sparingly, where we could get away with it. I was thrilled that I could actually get it working on an animated character. The problem dealing with animated characters is that, say I generated some outlines on the arm, and then I turn the arm, if the linework is stuck to the geometry, the linework is in the wrong place, it makes no sense. So the way to get around that is, you have to update the drawing in a way that doesn’t pop but also changes fast enough so that it looks correct.
What we did was, have the linework draw on (as though it was being created by an invisible pencil) and erase off. For animated characters, linework is constantly erasing off and drawing on, which gives it a really nice kind of organic feel, like how artists really draw. This also refreshed the drawings, so as the characters orientation relative to camera changed—drawing would always look correct. There is a really natural organic randomness that emerged from this. It felt right. Something we were struggling to achieve this with the comp method.
We tried to use it on environments and it was really successful there, too. In the India world, especially, there was a lot of linework in that because that was simply part of the look of that world, and in the comics that’s exactly how some of the lines would also be drawn.
Read the full story in issue #14 of befores & afters magazine.
