Remember that live-action scene in the making-of for ‘Final Fantasy’? Here’s how it happened

‘Cut! OK, let’s strike the set.’

The opening section of the making-of documentary on the DVD for Square USA’s Final Fantasy: The Spirits Within sees the film’s main character, Dr. Aki Ross, seemingly ‘performing’ a scene in the fully-animated feature, then asking the director, who has just yelled ‘cut’, ‘How was that one, was it good?’ She then walks ‘off’ a hologram set—still as a CG-animated character—into the live-action world of the film’s motion capture stage, past actual crew-members, before ‘reviewing’ the shot on a Silicon Graphics monitor.

This meta gag from Spirits Within, which somewhat resembles the outtakes that once permeated many of Pixar’s blooper reels, has stuck with me for many years. With the 20th anniversary of the film now upon us, I wanted to find out about that live-action scene, why it happened and how it was made. So I asked two of the crew members directly responsible–Andy Jones, the film’s animation director, and Remington Scott, who served as motion capture director–how it came about.

The mocap of Final Fantasy

But first, a quick pre-history on how the segment came to be filmed on the motion capture stage for Final Fantasy: The Spirits Within.

Hironobu Sakaguchi’s film based on the hugely popular game franchise was four years in the making, with production taking place in Hawaii at two main venues: Hawaii Film Studio at Diamond Head, where a 16,000 square-foot sound stage housed the motion capture unit, and on several floors of the Harbor Court office tower in Honolulu, in which Square USA crafted the final animation.

At the Diamond Head stage, Remington Scott oversaw motion capture of several performers wearing black leotards and reflective marker balls on a grid-ed up floor. The system employed was from Motion Analysis, which involved 16 infra-red cameras with the performers wearing around 35 markers each and acting amongst proxy set-pieces and with props like proxy weapons. An in-house system allowed the resulting data to be fed into Maya. Only the bodies were motion captured; the faces were entirely key-framed.

“It would all start off as storyboards and then a template from the animatics team,” explains Scott. “It would then move into motion capture, where we’d fine tune it even more, and then it would move into the high-res model building, and then they would tune that even more. And then animation would take it and they would even add other elements to it that kind of gave it some more fidelity.”

A CG star

Spirits Within came at a time when the notion of a fully CG—and photoreal—character in a film seemed possible, and Dr. Aki Ross was being set-up as one of those characters. Indeed, Scott notes that Aki was being prepared as, at least, a ‘virtual influencer’, in terms of what we might call her today.

Motion capture performers on the set of ‘Final Fantasy: The Spirits Within’

“She had already appeared in Maxim magazine. The producer and the director were talking to a number of places where there’d be opportunities for her to appear. She was on the cover of Entertainment Weekly, you know, there was a trajectory for her, and I remember we were all thinking how neat it would be to see her in the real world.”

And that’s, partly, what led to the notion of the live-action segment, since it added to the burgeoning notion of Aki as a real, living-breathing, albeit digital, character. “You couldn’t do it in the film,” notes Scott, “because you’d break the aesthetic and you’d break the plane of the existence of the film, but you could take her outside of it. And it felt that she could be a part of helping guide you through what it looks like to be a part of the team.”

“We wanted to show our process by merging the two worlds of live action and our CGI together,” adds Andy Jones, who was overseeing the animation of Spirits Within. “Our process for motion capture was very precise and we could show that off a bit by calculating Aki’s walk from the stage to the mocap processing desks. Capturing and stitching mocap performance across multiple takes to get her to move from stage to desk seamlessly was something we did often throughout the film.”

So, spurred on by what they saw as the increasing fame surrounding Aki and the fact that they had the technology to do this, the animation and motion capture teams prepared this live-action scene.

‘There was so much more busy-ness than ever before’

To craft the live-action sequence—which played out like a oner but was actually a couple of shots stitched together—a motion capture shoot for Aki took place, as well as a shoot of the plates of crew members on the mocap stage.

“There was so much more busy-ness than ever before,” remarks Scott. “There’s those two guys walking by and carrying something, there’s an animator and a VFX artist and so much other stuff going on. Maybe on the mocap stage, on a good day, it was five people there. It was never that many people normally!  We captured the action in, I think, perhaps three pieces, and then we had to blend it together to make it work.”

Jones recalls that, despite the advanced motion capture and animation work being done by the team at the time, the camera tech to even shoot the live-action was fairly limited. “This was before the digital camera revolution. We didn’t want to shoot film, so we had to rely on a higher end video camera that recorded to tape. The low light quality of the camera was lacking indeed.”

For Aki’s live-action debut, Scott believes the motion capture was performed by one of the animators, Kelly Goldstein (for the film, Aki’s motion capture performer was Tori Eldridge, while the voice actor was Ming-Na Wen). Goldstein happened to also be behind one of the other Spirits Within gags; where several of the CG characters perform Michael Jackson’s Thriller.

“We definitely had a lot of fun with the live-action piece,” comments Jones. “Like getting our lead animator for General Hein, Matt Hackett, to be sitting on the couch discussing a scene with his character. We even threw in a Phantom sitting on the couch as an extra. Roy Sato, the lead animator for Aki, was the one sitting at the desk showing Aki her take as she approaches him to have a look.”

Ultimately, the segment definitely came off as a fun moment, but both Jones and Scott also reflect on how the piece shows the camaraderie that had been established amongst the crew, who had come from several different areas around the globe.

“Final Fantasy was really like college for everybody,” says Scott. “It was like an incubator of some of the smartest people in the world I’ve ever worked with, to this day. And somehow they all just ended up coming to Honolulu to go work on this crazy project, and there was this family, which you can really see in that video.”