Something was very different about the environments in the photo-real animated film.
For its past visual effects work on films that needed to feature synthetic worlds or backgrounds, MPC has traditionally relied on digital matte painting, or DMP, as part of extensive 3D set builds.
On The Lion King, however, MPC knew it would need a whole new level of detail for the African landscapes in the film. That necessitated a new workflow, one that involved working procedurally, to deliver rocky formations, plains and lush grasslands.
befores & afters spoke to MPC set leads Marco Rolandi and Luca Bonatti to find out more about the process the studio adopted, which began with real photographic reference and ended with the generation of 66 CG sets that would cover 150 square kilometers.
Environmental builds: what MPC had done beforehand
“The usual CG workflow for environments,” says Rolandi, “is that you build assets and then you have another stage in which you take the assets, you put them together, and then you pretty much build the environment out of those. What we’ve done before, say on The Jungle Book, is focus on angles and focus on zones and then you up-res whatever needs to be up-res’d for the shot.”
What was different for The Lion King was the level of detail required and the flexibility to be able to see anywhere in the environment. That extra detail and the ability to also change things more easily came in the form of new procedural workflows, “not only from a purely technological point of view,” explains Rolandi, “as in the actual tools that we used in order to make some of these assets, but also in the way we approached the work.”
The new process
All of MPC’s environmental work began with an analysis of the photo shoot undertaken in Africa for the film (six photographers and four videographers were involved, generating 240,000 photos and 7,000 videos). “From these,” describes Rolandi, “we tried to decompose every single bit into components, and we tried to build the recipe that basically would re-create the environment itself.”
[perfectpullquote align=”right” bordertop=”false” cite=”” link=”” color=”” class=”” size=””]We didn’t necessarily have a build. Instead, the build is a dynamic build[/perfectpullquote]
From a workflow point of view, MPC’s environments department would receive the layout geometry from previs. “That gave us a base version of the set,” outlines Bonatti. “With that geometry, which wasn’t unique for the entire set, but was sometimes specific to the shot, we built a general build for the set, just the common areas that were shared for each shot of the set, or a subset of the shots.”
What MPC then developed was an expansion of its existing pipeline using Houdini and Katana to allow for the procedural generation of natural elements – things like rocks and terrain. This was coupled with a dynamic method for set dressing and scattering. To help make all this possible, MPC also built a new framework called PaX (Particle Xccelerator) that processed the large number of assets. It would then be passed to RenderMan for rendering, before final compositing in Nuke.
Also, whereas in past films the assets would follow a UV texturing approach, the texturing on The Lion King instead occurred procedurally (ie. in many instances, such as the canyon where the wildebeest stampeded occurs, there were generally no individual textures).
Photo textures acquired from real locations would become part of shaders that were procedurally applied to textures and materials that gave the locations the necessary real-world detail, and in fact were implemented so as to replicate natural phenomena such as weathering and erosion. That procedural approach also aided in delivering environments for wide shots, all the way up to incredibly close macro scenes in the film.
[perfectpullquote align=”right” bordertop=”false” cite=”” link=”” color=”” class=”” size=””]This is due to all the procedural rules that we put together.[/perfectpullquote]
It also meant there were pretty much no matte paintings in the film at all. “There are maybe 10 shots that are pure DMP,” says Bonatti. “The entire world is CG. So when they’re on Pride Rock and watching the pride lands, all of that is pure render. All the trees are scattered, all the terrain is filled with grass. Actual grass, not textures. It’s billions of instances of vegetation scattered everywhere.”
“Ultimately,” says Rolandi, “the main major difference is the way sets are conceived. We didn’t necessarily have a build. Instead, the build is a dynamic build. There’d be an indication – a sketch – of what the set should look like, but then the build is actually created on the fly when we make the shots themselves. And this is due to all the procedural rules that we put together.”
The challenge of trees
While The Lion King did include procedurally generated forests of trees (and often procedurally generated vines growing on those trees), the trees were generally not sculpted in the same manner as the rest of the rocks and terrain of the film. “This was because,” said Bonatti, “we had so much reference. After the team went on a photo shoot in Africa, they brought back a lot of references of trees, plants, grasses. And we had to match perfectly all that vegetation. So that part was done with photogrammetry and a lot of hand texturing.”
“On top of that,” continues Bonatti, “SpeedTree was used to build the main shape of the branches and exported for us all the rigs and we needed for like adjusting later on those trees in Maya to position them and animate them.”
A tool developed in Houdini allowed MPC to re-dress trees with procedural moss and hanging vines. The trees were also made to be dynamic for almost constant movement that could adjust for different wind inputs.
The advantages of working procedurally
To check the procedural approach would work, MPC embarked on a test bed of shots involving the canyon set. “It’s made of hundreds of separate rocks,” says Rolandi, “but there’s not a single texture that has been made for that set. Everything is procedural. Everything is derived from a combination of photographic textures that were shot in Africa and other materials we built. Every single rock has been scattered, and they’ve been scattered across using rules that took into consideration layering of the canyon itself, weathering of the canyon, and so forth.”
Another set, the pride lands, was able to be procedurally built based on satellite imagery, as Bonatti explains. “We used satellite imagery to guide the scattering tools and the tools that we have to dress the set procedurally. The satellite imagery was telling us where certain types of plants and vegetation should go. It would help us build the ecosystem. We had many ecosystems for each set.”
Bonatti details further: “We had specific rules that were able to dress the area close to the river in a separate way, because we studied how vegetation is different close to the water, or how in the desert there are specific types of plants. And all of that was made with rules – we decided what were the rules by studying the references. By then applying those, we could get a pretty good base of the ecosystem.”
[perfectpullquote align=”right” bordertop=”false” cite=”” link=”” color=”” class=”” size=””]By the end, we could literally output tens and tens and tens if not hundreds of shots in a week.[/perfectpullquote]
The ecosystems replaced the previous methodology of doing layouts by implementing rules, instead. “You might have a boulder,” says Rolandi, “and you tell to the system that around the boulder you have some specific type of grass done in a certain way. And then when you move the boulder, all the plants follow the boulder and basically you have a first layer of all of the layers done automatically.”
There was, however, still a great deal of manual artistry involved in The Lion King’s environments. This included hand placing some of the more broad elements as base shapes in the scenes (Pride Rock, for example), and setting up the rules of the ecosystems in the first place.
“Using our a procedural approach, there was a long, long time spent in research and development, in which basically the output was very little,” notes Rolandi. “The output ramped up very, very much at the end. By the end, we could literally output tens and tens and tens if not hundreds of shots in a week, simply because then you have all the rules, and things can actually go very smoothly.”
You can read more about MPC’s Lion King environments workflow in this paper accompanying the team’s SIGGRAPH 2019 talk (Luca Bonatti, Marco Rolandi, Julien Bolbach, and Kai Wolter. 2019. DMP without DMP, full-CG environments for The Lion King. In ACM SIGGRAPH 2019 Talks (SIGGRAPH ’19)).
