pixar’s Bob Moyer explains what all the fuss is about.
By now, you’ve probably all heard of Pixar’s open source Universal Scene Description (USD). It’s at the heart of the studio’s pipeline – and, increasingly, at other VFX and animation studio pipelines – in providing a common way for 3D data to be interchanged among different digital content creation apps.
Pixar used USD in a major way in Toy Story 4. USD is now implemented with a high level of support in the just-released RenderMan 23, and is supported as foundational data in Houdini 18 with its Solaris toolset. Other applications including Maya and Katana are also supporting USD.
So how did Pixar take advantage of USD for Toy Story 4, and how might other studios do the same in their workflows? To find out, I asked Pixar supervising technical director Bob Moyer about the implementation of USD at the studio and what impact it has had there.
b&a: How has USD impacted you in your work at Pixar?
Bob Moyer (Pixar supervising technical director): The main thing for us is that it provides a general and comprehensible format that we can use to transmit all kinds of information – geometry, shading, what have you – between the different portions of our pipeline, whether that’s our in-house software or third party software. It forms the backbone of our pipeline now.
The main advantage besides the ‘universal’ part of USD is the speed. So, say with the antiques mall in Toy Story 4, it was modeled and dressed with tens of thousands of props. Our camera and our animation departments could load the whole thing up and get pretty responsible responsive feedback. Those two features, both the universality and the speed, are really what have been the big wins for Pixar.
b&a: I was going to ask about the antiques mall. How did USD make that scene possible given it was such a data heavy scene?
Bob Moyer: USD creates a very concise format. It can basically bake down all of the transformation properties inside of our in-house software. For something like a set where animation isn’t going to actually need to interact with a whole lot of stuff, we just keep the whole thing as one flat object or even maybe with just a few levels of hierarchy. That really makes it easier for artists so that they can see and have everything loaded in the shot in a context that we didn’t really have before.
With our in-house tools, we’ve written something that allows artists, if they need to, to pop out individual objects. So, if Woody is running along a bookshelf but needs to pop out one of the books, which then needs to be animated, artists can pick that one book and re-grab it as a separate object, while leaving the other 99.99% of the set as a single object. That makes it so that they can still get everything they need out of it while letting things work as fast as possible.
b&a: Where was USD up to when you were working on Toy Story 4?
Bob Moyer: USD was a pretty solid portion of our pipeline during production. I think the most recent developments have been in the last couple of years where we’ve integrated shading into USD as well. We’ve tightened the knot more and more between the various parts of our software. It’s pretty much the format we use, unless we’re doing volumetric data to go between Houdini and our in-house software or from Houdini to Katana. So we’ve really tightened that loop and we try to use it as pretty much the format between tools.
And while we don’t currently use USD directly from Maya, we can actually use Maya as a front-end interface for assembling USD. So our set dressers will take pre-built models as USD references and drop them into Maya and set dress them, whether that’s by hand or procedurally. Again, it gives that same kind of instanced super-fast workflow where they can just have, say, a thousand sheets of paper stacked up and it feels just as fast.
b&a: What was done before you could rely on USD? What were the bottlenecks? What was the general workflow before USD was a thing at Pixar?
Bob Moyer: Before USD we had some in-house model formats, but I don’t think they were quite as optimized. So, you would bring in one model and then bring in another model and then another, and it didn’t really have the knowledge of, ‘Oh, this can be flattened down to something that’s much faster to work with’. So it treated every single model as its own piece. When we did that, we’d have to come out of an application and either convert it to some other universal format and figure out a way to hack that in. Or we would have to go through some kind of conversion process to get it into our pipeline.
So that’s been the nicest part of USD. It creates this flexibility where we can see along the way that, ‘Oh, we need to grab this piece from over here and this part piece from over here.’ Now we have these nice sockets and avenues to integrate those things together and keep it flowing down towards RenderMan.
b&a: In addition to the antiques mall, was there another particular sequence in Toy Story 4 where you were like, ‘Oh, thank God we’ve got this approach now’?
Bob Moyer: There’s a couple. For the carnival, every single light bulb is its own USD reference that has been referenced into a model or referenced in the set. There’s thousands of them in the carnival. The nice part about that was not just to keep things fast or keep things flowing through, but it also meant that we could have one model – this light bulb – that was then carefully shaded and calibrated between our sets department and lighting. So it actually gave off the exact right Kelvin color and illumination response for RenderMan.
That, in turn, allowed us to have a workflow where we could depend on knowing that all these light bulbs were set up the exact same way. So when lighting goes in, they’re not calibrating each and every light bulb – they know exactly how they’re going to respond. That kind of hierarchical procedural workflow was something that, even though lighting isn’t directly worrying about USD, it’s still having benefits for them in the long run.
I’m also thinking of the rain sequence. The rain surfaces, the splashes, the drops, were all procedurally created within Houdini, and then those were exported out, brought into the scene, and then lighting actually had those to work with live in-camera for their work.
b&a: As you mentioned, USD is being implemented into other products like Houdini. How’s that going to also affect Pixar’s workflow?
Bob Moyer: I think the exciting thing is that it then opens up the workflow. You don’t have to work in one place to get something to another place. For example, our set dressers primarily work in Maya. You could easily imagine that you might have something that would really be procedurally powered through Houdini. We had all the dusts and cobwebs in the antiques mall were built in Houdini.
We could have millions of curves that then got pushed through and basically layered on top of the set. But there’ll be other things; maybe you want a ton of books that have all tumbled out of a shelf, Previously it’s been a little divided between FX and sets. If you have USD and it’s fully integrated into all these packages, the move back and forth and the fluidity of that gets really exciting.
b&a: Where do you see USD going? What else would you like in terms of functionality with USD?
Bob Moyer: The more we see USD not just being about geometry and animation, and actually take on things like shading, the more you discover how different departments or different disciplines can work together and base their work off of each other. When it’s in that same format, it gives you those exciting opportunities that you hadn’t even thought of before.
I would love to see more applications take on USD as the shading model, the way we do it. And potentially lighting, too. That means that you can say, ‘I’m doing my shading in this application for this project, I can do my lighting over here in this application, and it’ll still come down the pipe the same way.’
Want to dive deeper into USD? You can read Pixar’s USD documentation here: https://graphics.pixar.com/usd/docs/index.html. And there’s a neat explainer on the RenderMan site that explores the different aspects of USD here: https://renderman.pixar.com/stories/pixars-usd-pipeline
