Puppet passes, and the art of the nature doco, without animals to film. An excerpt from issue #61 of befores & afters magazine.
Armed with a wealth of planning and previs, those involved in making Prehistoric Planet: Ice Age moved on to capturing plates. During shoots in locations around the world, the visual effects team, in particular, consisted of a supervisor, wrangler and capture TD. They were, describes Framestore visual effects supervisor Gavin McKenzie, “tasked with collaborating with the directors and DOPs to select the optimal spots for shooting the background plates. We began our shots with a ‘puppet pass,’ which was crucial for determining the timing, framing, and focus of the shots. Once we were happy with this, we repeated the shot clean, without the puppet.”
Practic Creations built and performed the puppets and physical interaction reference assets for the various creatures seen in Prehistoric Planet: Ice Age. Puppeteer Brian Fisher explains that “the puppets needed to be dynamic assets which would help to solve a range of complexities. They needed to be lightweight and able to fully break down and fit into suitcases for travel around the world, and be easily assembled in remote locations without special tools. They needed to be made of resilient materials which could withstand temperatures ranging from sub zero glaciers, to full sun reaching nearly 50 degrees on iron sand.”
“From a practical standpoint,” continues Fisher, “they needed to represent the physical size and shape as close to the scientific material. Most importantly, because this show is natural history, the puppets helped to create a sense of spontaneity within the filming process, imbuing the story with character and a grounded natural performance.”
Across this series, Practic Creations used both traditional techniques and new technologies to build nearly 100 puppets and interactive assets. These ranged from representations of tiny lizards, up to some of the largest mammals ever to have roamed the earth—each of which could fit in a suitcase. When a puppet was not used, an interactive element would be relied upon, explains Fisher. “Some examples of the interactive elements included feet to leave footprints, a wooly rhino horn for snow interaction, a giant claw on a four meter pole to physically pull fruits from a high tree, and the list goes on.”
Visits to the various locations began with scouts to determine where the action would take place. “Depending on the environment,” says Fisher, “we needed to ensure we didn’t disturb the ground too much, as some locations had loose sand, some had deep snow, some thick foliage. When a specific location was decided, we would get the puppets in to explore the action, and block out the base elements of the story. Sometimes this would follow the previs as closely as possible, and sometimes while using the puppets to explore the space, a new idea would come up which we would flesh out and define for the filming.”
Then, once the blocking was completed, the puppet pass would be filmed. “Because the puppets were dynamic, and led by experienced puppeteers, the camera would follow our performance as if it were a real animal,” outlines Fisher. “This energy and intention that this style of puppet pass provided allowed the series to feel like it was living creatures that were being filmed. One or two puppet passes would often give the reference needed for the animators, and camera would get a muscle memory for what the take felt like to replicate without us in shot.”
Clean passes followed, but here the puppet performers would often remain hidden in shot with the puppet so they could jump in quickly to replicate a move, or give a size or movement reference again before hiding again. “Because this was filmed in a natural history style, often the camera would be on a very long lens, so us puppeteers could be quite far away from the rest of the team, sometimes for hours at a time,” states Fisher.
The puppeteers would also take care of any interaction passes if needed. Here, the teams were determined to be as accurate to the science as possible and obey accurate measurements of stride length, weight, gait and others. “While in Norway,” says Fisher, “we needed to do interaction passes for the wooly rhino’s horn shovelling deep snow to reach foliage beneath it. We had Darren Naish on location with us the day we were doing these movements, and he was able to help define exactly how the wooly rhinoceros would have moved its horn, and why. This is one example of the level of detail we gave for reference.”
These puppet passes were crucial in several ways, advises Framestore visual effects supervisor Benjamin Loch. “It made framing for any shot so much more accurate in terms of composition, camera move and focus. This ultimately enabled an edit to be populated very early and continually refined; this made it easy for the filmmakers to review and visualize the WIP sequences with the placeholder puppet passes that remained until the CG elements were ready to switch out. Even for the VFX team, it helped to understand the desired look of the shot, the creature’s size and movement, even the lighting on the puppet pass was a useful reference.”
The shoots were considered an interesting meeting of the minds; the BBC’s Natural History Unit who typically shoot documentaries for real, and a visual effects crew who tend to have certain requirements for acquiring data. “Naturally, the Natural History Unit’s usual workflow does not include the data gathering that we need for VFX,” points out Framestore compositing supervisor Chris Zeh, “so we had to learn a bit from one another about how we work, what we do and why, so we got the best looking shots and the data we needed.”
Framestore visual effects supervisor Francois Dumoulin shares his experience in how a typical shoot unfolded. “Once we arrived on location, our first priority was always finding the ideal place for the action to unfold, both logically and aesthetically. In vast environments like the Mongolian steppes or the glacial outwash plains of Iceland, this meant spending entire days scouting for the perfect spot. Once a location had been chosen, we would move on to camera placement and lensing. At that stage, I would step back and let the natural history specialists determine the most believable setup, based on how the animals would behave and react if they were physically present. From there, our puppeteer would start blocking the performance.”
“Oftentimes,” continues Dumoulin, “these performances turned into ‘all hands on deck’ operations, with everyone from our small on-set team jumping into frame to enact a character. For shots where the creatures appeared miles away from the camera, I would sometimes send one of our drivers out in a jeep as a placeholder. Other times we had no option but to walk for hours, carrying equipment and giving each other directions through talkie-walkies—most locations had no cellphone coverage.”
Once the puppet performance had been captured as the hero plate, the DOP would meticulously replicate the exact same camera move and lens behavior for a clean pass. “We also captured extensive tiles at multiple focal planes, giving us the flexibility to take over the camera movement later in CG if the animation called for it,” remarks Dumoulin. “When dealing with multiple planes of foreground occlusion we would sometimes shoot several clean plates with bluescreens isolating the different depth layers. When shooting in harsh sunlight we occasionally used cutout shapes or flags to capture a reference shadow pass. When we knew our character would interact with a specific element of the set, like a tree branch or scattered leaves on the ground, we would shoot plates with a reference interaction performed by the puppeteer and a clean plate without the interactive element so that we could replace it with a digital replica in post. We ended up calling that process the ‘gardening’ pass.”
Dumoulin mentions that he would try to collect as many practical live-action elements as possible, such as vegetation against portable bluescreens, or reference footage of blowing snow and sand. “This meant improvising with whatever tools we had available. Forget the Hollywood way—rain rigs, wind machines—we had none of that. We were a small documentary crew of eight, out in the middle of nowhere. At one point, for example, we ended up driving a jeep in circles like maniacs to generate a sandstorm reference.”
Framestore’s data acquisition workflow was largely comparable to a traditional VFX shoot. “We documented lighting conditions using HDRIs, chrome/gray balls, and Macbeth charts,” says Dumoulin, “while also gathering comprehensive reference for the textures and shading of the terrain our creatures would interact with. Surveying the environments themselves was one of the biggest challenges. The sheer scale of some of the scenes made full LiDAR coverage impractical, so instead we focused on capturing key interaction zones and representative ‘generic surfaces’ at high resolution rather than attempting lower-density scans across enormous areas. These surveys were frequently supplemented with photogrammetry captured through multiple drone flights.
Some environments proved nearly impossible to scan altogether, as Loch relates. “Many locations came with their own further challenges, involving fast flowing water, steep and unstable terrain or unexpected real wildlife such as an African elephant deciding to walk through the set, to list just a few. The result was a vast collection of LIDAR geometry being scanned and then being accurately pieced together off site.”.
Icebergs were too reflective for a LiDAR laser, while primitive forests with dense foliage moving in the wind generated nothing but a noisy mess, notes Dumoulin. “In many of those cases, we ultimately rebuilt parts of the environments in CG to ensure perfect contacts, using the live-action plates and a hefty amount of reference photography as the foundation for the final look.”
“Given the size of some locations, we used a combination of LiDAR scanning and drone photogrammetry,” details McKenzie. “Furthermore, due to the extremely long lenses used for shooting, we LiDAR scanned each individual camera position to aid in accurate shot tracking.”
As mentioned, Framestore’s Farsight GO virtual production app allowed the on-set teams to place scale accurate moving creatures into the iPad viewport and discuss shot composition with the director prior to sending the puppeteers into frame. “This proved to be very handy whenever the characters were close to the camera, much less so in wide shots where the limitations of the sensors would make the tool quite unreliable,” discusses Dumoulin. “A memorable moment happened while using this tool for the Moa sequences in New Zealand. One of the local farmers whose land we were shooting on had been fascinated by this extraordinary creature since childhood. Over the years, he had even collected bones he discovered in the cave systems beneath the forest. Watching the giant bird come back to life through the augmented reality lens moved him deeply. He got teary eyes witnessing it seemingly ‘re-emerge’ in real-time.”
