Bullet Time before Bullet Time

The tests that led to the landmark visual effect of the ’90s

When The Matrix’s Bullet Time hit screens in 1999, things changed. Here was an elaborate visual effect that wasn’t just for…effect. It was intricately part of The Matrix’s storyline (being that Neo was inside a construct inside the Matrix and could start controlling his world, just as the prophecy had said).

But before the Bullet Time we know and love, there were tests. Partly proofs of concept, partly designed to get the film greenlit, these tests took place at Mass.Illusions in Lennox, Massachusetts. The machinations of how that company was involved in the VFX Oscar-winning The Matrix (as well as the Oscar-winning What May Dreams May Come), and then how the work was ultimately done by Manex in Alameda, is a story for another article. This article is limited to how the Mass. Illusions team delivered a Bullet Time test that would get many excited about the Wachowskis film. There were, of course, several other bullet-time-like advancements made prior to any of this work, most notably by Tim Macmillan and Time-Slice Films, as well as Buf.

“We did the Bullet Time tests to help producer Joel Silver raise money for the movie,” says Pierre Jasmin, who was at Mass.Illusions at the time, and says the Wachowskis had shown a core team at the company some incredible storyboards and concepts for what Bullet Time was. A shoot was organized that would draw upon what might be considered a ‘frozen time’ effect, something that had been seen before in commercials, music videos (see Michel Gondry’s Rolling Stones ‘Like A Rolling Stone’ clip) and even some feature films.

The test, shot outside Mass.Illusion’s premises, involved 27 cameras, with the 27th being a live-action camera – the idea being that the scene would ramp to the live action after the Bullet Time effect was completed. John Gaeta (who would go on to be the visual effects supervisor for The Matrix) directed the test shoot. Nick Brooks, who became one of the visual effects supervisors on What Dreams May Come, was the designer and developer of that Bullet Time test. Jasmin, who worked as lead software programmer on Dreams and would later co-found RE:Vision Effects Inc., assisted Brooks in working on the shot. In addition, Blake Holland had generated previs for the test shot, mostly using Softimage 3D to establish where to place cameras and where the bullets and their wake motion would be animated.

Since the painterly world of What Dreams May Come and The Matrix were somewhat intertwined in terms of R&D at Mass.Illusions, optical flow was a technique that seemed a fit for Bullet Time. “During What Dreams May Come,” says Jasmin, “we had made some arrangement with Kodak, who had the software called Cineon, which had a slow motion re-timer called Cinespeed. They gave us a copy of Cinespeed that was just outputting the motion vectors – the optical flow motion vectors, instead of doing the re-timing inside their software. We were able to use a process like that for the fire and explosions in the test, but it was really too hard to do. It didn’t work at all for the main actor, even on a greenscreen.”

“Our test was great for the fire explosion in the shot,” Jasmin adds, frankly. “But it was really hard to pull-off for the actor because there’s so much distance between camera views. They solved that in the eventual film with more cameras and optical flow and other things.”

To get the shot working as best they could, the team moved forward, ultimately breaking down the actor into ‘per-limb’ layers, stabilizing these layers, morphing the stablized views in Elastic Reality, and then unstablizing them for the final effect. “That meant it ended up being a morphing job,” says Jasmin, “and even that was way too hard. What we were doing to make it work was breaking down the actor into little limbs, every element, and stabilizing that with optical flow and re-building the character, basically.”

The result is what you see in the video above, which also shows other aspects of the work, such as using a 3D cube to line up camera positions, and even trying out some dense stereo reconstruction of the footage of the actor to try and extrapolate a CG model. The test helped solidify the ‘look and feel’ of what Bullet Time could be, but in and of itself, it wasn’t deemed the way ahead for how the effect would be realized. “Independent of that scene, our conclusion at the end was, this is way too hard,” says Jasmin. “We can’t make a movie like that!”

Eventually, as the production received the greenlight – and moved to filming in Australia – a team of people – including John Gaeta, Kim Libreri, Dan Piponi and George Borshukov, among many others – did manage to solve the Bullet Time effect with a denser camera rig, and then a combination of several techniques such as optical flow, frame interpolation and photogrammetry to make the Bullet Time you know and love.

Read more MATRIXweek stories at befores & afters.