Also, behind the scenes of Framestore’s spaghettification and the Yggdrasil finale in season 2 of ‘Loki’.
In season 2 of Marvel Studios’ Loki, the central character played by Tom Hiddleston experiences the phenomenon of timeslipping. Here, he finds himself rooted in space, but not time. To Loki, and to others, it appears as if he is disappearing and glitching–sometimes painfully–between the past, present and future.
To create the ‘stretchy’ effect of Loki’s body effectively being molecularly ripped apart, production visual effects supervisor Christopher Townsend and the show’s creative team first embarked on a look development effort for the timeslipping effect.
On set, Hiddleston would then give an intense set of performances to portray the moment he was going into and out of the time slipping. Sometimes this would be as part of a normal live-action scene with other actors, and sometimes it was orchestrated against bluescreen or greenscreen.
In post, Framestore, under visual effects supervisor Matthew Twyford, then carried out an extensive matchmove, rebuild, effects simulation and compositing effort to bring the final gruesome timeslipping moments to life.
Twyford recounts in this interview with befores & afters how different Framestore departments–visual development, visualization and visual effects–were all involved in the timeslipping shots. Plus he also dives into the complex spaghettification scenes in the show, and the VFX involved in Loki’s efforts to save the Temporal Loom in the final episode.
‘Let’s see some gruesomeness’
b&a: Where did you start with the timeslipping effect?
Matthew Twyford: I think we did somewhere between 30 and 40 timeslips, so we knew that it was going to be a hero effect. So much so that Tom Hiddleston actually made time to sit down with us during shooting, which was fantastic, to explain how it worked, what he was feeling, how it evolved through the show. It was so valuable to us to get that direct feedback to know what he was going for.
We knew how important it would be, how it was going to relate to Loki, the character, what he was going through at the time, and how we had to complement Tom’s performances, because it’s a bit of a weird one. We know it’s going to be painful. We know it’s going to be violent. And also, it’s confusing to him as well. So all of these things we got direct from Tom, which is rare to get that really one-to-one insight, which is really valuable to us.
We also knew that every timeslip was going to be unique in its own way; different locations, different periods. So, we had to ask ourselves, ‘Can we pipeline this or are we going to treat it like a fully crafted individual moment?’ And in the end, it was a bit of both. It involved pretty much everyone in the team. I think the only team that wasn’t involved in this particular effect was probably environments. One thing that was nice was to really heavily involve the visual effects editors. They played a huge creative role in this in terms of the selects. We had three of our in-house editors working on this because there was so much footage and so much work to do.
b&a: What performances did Tom do on set and how did that inform what Framestore ended up doing?
Matthew Twyford: Tom did loads and loads of performances for this. Many unique performances that we selected from. Every one of them was retimed. They were in different positions. So editorial played a huge part in doing the selects, getting the retimes and the conforms all in place to give us what we called the editorial pass, which was all the passes we want which include even just like a limb. Every single part of his body, every different performance was carefully selected, positioned, crafted, to work in space and time to add to the value of the shot to go with the performance.
Then every one of those performances was body tracked. As you can imagine, it was a huge amount of body tracking for everything. It may look like it’s only eight frames long, but there’s anything up to 15, 16 performances that were body tracked with retimes, restaged in space. Every one of those performances became a 3D asset.
Then that was passed to FX. Those guys did the stringy skin, stretchy bits and pieces going on there. Then it was a huge comp task, because all those stretchy bits had to blend into all of those performances. Just the sheer dimensionality, the space of the strands passing through this and going onto that and shadowing this. It was all rendered to have shadow passes and to work in 3D. The 2D guys added on some custom strands. We had some tools that added on what we called the taffy puller, like the taffy sweets.
Then right at the end of that, we let our DMP artists in there to have some fun. They went in there and the brief from Chris Townsend was, ‘Let’s see some gruesomeness.’ The DMP artists were doing bloated eyeballs and ripped up teeth and blood coming up the nose–there’s beautiful little moments in there through every single one of those timeslips.
b&a: Had Chris Townsend given you any kind of artwork or development or previs, or had Framestore been part of the look of that as well?
Matthew Twyford: Framestore was involved with previs tests. And our visual development team headed by Owen Jackson then picked things up and they developed a whole string all the way through from first looks through to pretty much a finished shot, which was the elevator shot. That shot came out of vis dev looking really polished. They obviously used lots of cheats and fast turnaround stuff, but they handed over a look which we knew we could work with. We knew the timings, the speed, the violence of it, and we used that as the real basis. We also got involved in postvis. So, by the time we started on visual effects, everyone was fairly comfortable with what they were going to get.
The secrets of spaghettification
b&a: What about the spaghettification shots? Tell me about what the creators of the show and Chris wanted this to look like?
Matthew Twyford: We always knew that it was almost like a horror moment. It had to feel really ominous–‘This is seriously bad news.’ Because of that, we knew it couldn’t be a light show sort of thing. We didn’t really want it to feel ‘magic’. We wanted to feel like it was really practical and grounded and ominous, and it had this momentum where you knew, ‘There is no stopping this’.
We also knew that it sucks in everything, including light. That was one of the biggest challenges, doing something that still visually told the story in an environment where we’d actually lost all lighting and we weren’t allowed to glow anything.
Again, we had previs, and we did postvis. Our vis dev team was involved also. We did a hero shot which was the shot of Lyle who’s the record storekeeper running at Sylvie, and he spaghettifies with the whole shop going behind it. That went to a very polished state with vis dev.
b&a: How did you deal with the way the strands propagate?
Matthew Twyford: One of the things that we knew fairly early on was that we couldn’t afford to make everything into assets. We had three or four major environments and pretty much most of the major cast that we knew we had to spaghettify, and there wasn’t the budget or the time to make every single prop in these very complicated sets into assets that we could then spaghettify.
We were thinking we were going to use projections. We’d start by using a LIDAR scan, and then we’d project the footage onto it and then we could spaghettify from there. But we realized as soon as you spaghettify something, there’s nothing behind it. It looks cool, but it looks like a projection on a piece of geometry, which has only got a front face which is spaghettified. While it allowed us to get the color and really integrate with the set because it just pulled the natural lighting and the texture, we still knew we had to build more geometry behind.
In fact, we knew we had to build a deep environment which is called ‘nothingness’, which is a theme that we use throughout the episodes, like an infinite void of dead spaghetti. Then it was just a case of working out what is almost the minimum we can get away with in terms of what gets spaghettified, what gets revealed. Because as soon as you started putting lots of things in there and spaghettify them, it became too complex. You got into this horrible mess of things intersecting. The strands are getting in the way of each other.
In the end, every piece of that spaghetti has been choreographed to wrap around, to work with the environment, and to time with things that are disappearing to hide things that we don’t want to see. The choreography was driven by the FX team, but in a way where they were very much becoming animators. Their brief pretty much from Chris is almost this dance, where it feels like it is doing a thing. It’s got a certain life to it. It knows where it’s going. It knows what it’s doing, but it’s got this relentless momentum to it as well. It was fun almost turning the spaghetti into a character in its own right and giving it life and intent.
b&a: I was going to ask you about whether there was some gross animation done by an animator and then handed off to CFX or Houdini artists.
Matthew Twyford: In this case, the FX team was the one who took full control of that. Animators weren’t involved in the spaghetti. This was 100% the FX teams getting their hands dirty and giving it life of their own.
b&a: There’s a particular shot I remember from the second last episode where Loki’s in the control center surrounded by–it almost looks like cabling as the spaghettification happens. What were the particular challenges of that one because there’s so many different colors there?
Matthew Twyford: Every strand of spaghetti in theory picks up the material properties of where it emitted from. So you’ve got spaghetti which is glass, you’ve got spaghetti which is wood, you’ve got spaghetti which is cloth and leather and concrete. Now, some of that is 30 meters way back in the distance and it’s got a long way to come. When we had to do the streets, for instance, it’s a very difficult dynamic for the level of detail because the spaghetti in theory is slightly fatter than a spaghetti strand in real life. But that doesn’t work for distance. So it’s got a dynamic level of detail on it to work with as it travels through space. Obviously, some things may have spaghettified 30 seconds ago in a previous shot that is now traveling through the shot.
Once we got into that environment into the control room, the spaghetti also has to interact with Loki. There’s one shot where he’s actually putting his hand through the spaghetti and it’s breaking up into dust motes where he’s actually interacting with it. He’s getting really frustrated because he wants to stop it. And that’s part of the story where he suddenly starts clicking. He’s like, ‘Hang on a minute, I am interacting with this. Maybe I could control this.’
At the End of Time
b&a: Framestore also worked on the Temporal Loom. How did you approach that work?
Matthew Twyford: The Temporal Loom was a shared asset. We picked up the asset itself and matched to what Trixter had done earlier. Framestore looked after all the work in the finale where Loki steps out himself, he goes into God mode, shatters the loom, collects up all the time strands, and then forces them together to rebuild a new time strand, a new loom in the shape of Yggdrasil. In this particular case, there was previs and there was postvis going through FPS at Framestore, but there was no vis dev for this. So this is something that we had to develop in visual effects from scratch, with a little bit of help from the art department with the actual look of the detail of the time strands.
b&a: There’s a great profile shot of Loki walking and bits of his costume peeling off, which I guess is reminiscent of what we might’ve seen in some of the earlier episodes.
Matthew Twyford: That was a full Loki digi-double costume. It was actually filmed in the God outfit–the green robe outfit–not the TVA uniform. It was a case of keeping the performance of his face and his hands and then a full digi-double replacement for the body. We had two complete digi-doubles, one with the God mode, one with the TVA outfit. Then it was a job of transitioning from one outfit to the other. We basically shrink-wrapped the God mode suit onto his body and then put it underneath the TVA uniform.
This meant that when the TVA uniform gets shredded with the temporal wind forces, we could inflate Loki’s God outfit back to its regular proportions before we then had the key moment of letting the cape unfurl for the big reveal.
b&a: For Loki controlling the time strands, did Tom have to mime with something there on set?
Matthew Twyford: We first did some tests. We did tests with people actually pulling ropes and cables and seeing if Tom Hiddleston could actually use that and get the weight in there. We always knew they were going to have weight and that they were going to have a definite density to them. In the end, we shot his performance without anything. He was performing without anything in his hands but we did do practice runs so he could get a feel for it. He really did nail that performance.
Then it was a case of very tight body tracking on the hands and the interaction. They were supposed to be these infinite lines or cables and we had to give animation to them. We had to find a place to hide the points so they came into shot, they looked infinite, but they always had to have a start and finish for us, that is, for something for the animators to work with.
