Bullet Time before Bullet Time

The matrix Bullet Time

The tests that led to the landmark visual effect of the ’90s

When The Matrix’s Bullet Time hit screens in 1999, things changed. Here was an elaborate visual effect that wasn’t just for…effect. It was intricately part of The Matrix’s storyline (being that Neo was inside a construct inside the Matrix and could start controlling his world, just as the prophecy had said).

But before the Bullet Time we know and love, there were tests. Partly proofs of concept, partly designed to get the film greenlit, these tests took place at Mass.Illusions in Lennox, Massachusetts. The machinations of how that company was involved in the VFX Oscar-winning The Matrix (as well as the Oscar-winning What May Dreams May Come), and then how the work was ultimately done by Manex in Alameda, is a story for another article. This article is limited to how the Mass. Illusions team delivered a Bullet Time test that would get many excited about the Wachowskis film. There were, of course, several other bullet-time-like advancements made prior to any of this work, most notably by Tim Macmillan and Time-Slice Films, as well as Buf.

“We did the Bullet Time tests to help producer Joel Silver raise money for the movie,” says Pierre Jasmin, who was at Mass.Illusions at the time, and says the Wachowskis had shown a core team at the company some incredible storyboards and concepts for what Bullet Time was. A shoot was organized that would draw upon what might be considered a ‘frozen time’ effect, something that had been seen before in commercials, music videos (see Michel Gondry’s Rolling Stones ‘Like A Rolling Stone’ clip) and even some feature films.

The test, shot outside Mass.Illusion’s premises, involved 27 cameras, with the 27th being a live-action camera – the idea being that the scene would ramp to the live action after the Bullet Time effect was completed. John Gaeta (who would go on to be the visual effects supervisor for The Matrix) directed the test shoot. Nick Brooks, who became one of the visual effects supervisors on What Dreams May Come, was the designer and developer of that Bullet Time test. Jasmin, who worked as lead software programmer on Dreams and would later co-found RE:Vision Effects Inc., assisted Brooks in working on the shot. In addition, Blake Holland had generated previs for the test shot, mostly using Softimage 3D to establish where to place cameras and where the bullets and their wake motion would be animated.

Since the painterly world of What Dreams May Come and The Matrix were somewhat intertwined in terms of R&D at Mass.Illusions, optical flow was a technique that seemed a fit for Bullet Time. “During What Dreams May Come,” says Jasmin, “we had made some arrangement with Kodak, who had the software called Cineon, which had a slow motion re-timer called Cinespeed. They gave us a copy of Cinespeed that was just outputting the motion vectors – the optical flow motion vectors, instead of doing the re-timing inside their software. We were able to use a process like that for the fire and explosions in the test, but it was really too hard to do. It didn’t work at all for the main actor, even on a greenscreen.”

“Our test was great for the fire explosion in the shot,” Jasmin adds, frankly. “But it was really hard to pull-off for the actor because there’s so much distance between camera views. They solved that in the eventual film with more cameras and optical flow and other things.”

To get the shot working as best they could, the team moved forward, ultimately breaking down the actor into ‘per-limb’ layers, stabilizing these layers, morphing the stablized views in Elastic Reality, and then unstablizing them for the final effect. “That meant it ended up being a morphing job,” says Jasmin, “and even that was way too hard. What we were doing to make it work was breaking down the actor into little limbs, every element, and stabilizing that with optical flow and re-building the character, basically.”

The result is what you see in the video above, which also shows other aspects of the work, such as using a 3D cube to line up camera positions, and even trying out some dense stereo reconstruction of the footage of the actor to try and extrapolate a CG model. The test helped solidify the ‘look and feel’ of what Bullet Time could be, but in and of itself, it wasn’t deemed the way ahead for how the effect would be realized. “Independent of that scene, our conclusion at the end was, this is way too hard,” says Jasmin. “We can’t make a movie like that!”

Eventually, as the production received the greenlight – and moved to filming in Australia – a team of people – including John Gaeta, Kim Libreri, Dan Piponi and George Borshukov, among many others – did manage to solve the Bullet Time effect with a denser camera rig, and then a combination of several techniques such as optical flow, frame interpolation and photogrammetry to make the Bullet Time you know and love.

Read more MATRIXweek stories at befores & afters.

6 Replies to “Bullet Time before Bullet Time

  1. I was part of this team that worked on this effect. It was originally based on work done in Paris by BUF using hand-edited morphing software. This was used to excellent effect for Absolut Vodka. The senior VFX Dir. was originally Joel Hynek who brought John Gaeta to Mass.Illusion from R/Greenburg.

    There were a number of people who had much more to do with the development of this effect system than the ones you mention.

  2. Great article (esp. mentioning the classic Michel Gondry’s music video and What Dreams May Come), but it’s worth to mention that the original idea of the Bullet Time dates back to the 80s. Tim Macmillian invented the Time Slice technique, later Dayton Taylor patented a similar one in US.

    Some resources:

    Time-Slice Films archive, early works of Tim Macmillian:

    A conversation about the topic:

    Digital Video Hacks book:

    The companies of the inventors:

    American Cinematographer September 1996:

    And now I see that Tim liked this article on LinkedIn 🙂

    Greg Barta

    1. sciVfx… Pleased to see your pointer to the origins of the technique ‘Time-Slice’ technique. I had the fortune of working with Tim on several projects alongside his brother who has continues with the technique and pushing the boundaries on new techniques in volumetric capture.

    2. Not only did the timeslice approach predate MATRIX by quite awhile, but the whole frozen time idea was in some circles considered played out well before MATRIX reinvigorated it. For STAR TREK INSURRECTION, they had slowed-down time/heightened reality sequences and the VFX vendor — I think it was VIFX, but maybe Santa Barbara Studios, which was the other lead house — wanted to use their own rig to do a similar approach, but the TREK head honcho Rick Berman said no, saying the frozen time look was overused and a cliche from the successful commercials that had already aired and he wanted something new in the movie. Of course this was the same guy who dismissed a lot of original STAR TREK as being little more than aliens in togas and refusing to allow antennae on the NextGen series with the ultimatum ‘we don’t do antennae.’

Leave a Reply