How Pixar made cheese and bread and grapes look good enough to eat.
In the past few weeks, befores & afters has been going retro with RenderMan, publishing previously published Pixar materials from days gone by that highlight some of the rendering highlights from Brad Bird’s Ratatouille. Our next piece on that film now covers the fine art of shading of food; preserving the required stylization of the film while also making the food particularly appealing.
1. Introduction
The challenge of shading food for Ratatouille was to work with a stylized look that fits into the film’s world, yet is still readable and recognizable as something appealing to eat. We, as humans, have a built-in sensory system to know what looks edible to our eyes and stomach. Finding that acceptable (and tasty) appearance was the main focus. To achieve this, Pixar used subtle illumination techniques that became a general approach for a variety of objects.
1.1. Overall Approach
Pixar’s food shading was based on skin and scatter computations, which have had internal improvements in speed and look since The Incredibles. The shaders were mainly a small variation on the studio’s in-house skin. Pixar took full advantage of this technology base, since there is a breadth of research and precedents in the industry [Jensen, Marschner, Levoy, Hanrahan 2001] [Jenson, Buhler 2002].
Skin has always posed a challenge, since there is a familiarity to what the surface qualities should be. Food had similar problems when interpreting into cg, as well as sharing properties of translucency and a living organic feel (or post-life, in the case of food).
For most shaders, Pixar wanted to expediently feed the models into its pipeline quickly. The studio accomplished this by taking advantage of Slim, which offered a great level of flexibility. By creating networks that could input a variety of shading elements into templates (procedural noises, textures, etc.), then using scatter on the results, artists could add an additional level of visual complexity to otherwise “typical”, slim shaders.
For items which do not fit into this system easily (such as bread and wine), it became a “special case” shader that was refined on an individual development project basis.
2. Art & reference – finding the stylization
Pixar’s challenge, working closely with art direction, was to make the food appealing but not entirely “real”, which could distract or stand out from the film. They did not apply any real world photographs as textures, but photos were often adapted, modified and painted over. [It was very easy to obtain plenty of reference, and there were many trips to the grocery store by Production]. Extensive photo references were used; including food prepared, cooked and lit for Pixar’s own documentation.
Consulting with chefs and watching them at work, enabled to document and control the whole process from raw food to meal. Part of the stylistic choice was described as the “doll house” look. This look is created by scaling up features larger than they would usually be in relation to the overall size, while maintaining realistic properties of the surface.
Working with art and direction, Pixar narrowed in on the factors that made it appealing. The key issues came down to three important qualities:
- Softness
- Reflection
- Saturation
The softness was an indication of light passing through the medium; things that do not exhibit internal light bounce tend to look hard and plastic. Saturation was an indication of freshness; deep rich colors signify quality food. Reflections (and specular) have a relationship to moisture; a characteristic that most edible and appealing food contains.
3. Using Scatter to capture translucency
The subsurface scattering system used on Ratatouille was an extension of the system developed for The Incredibles. Both use an IIR filter to blur sampled data through a voxel grid. For The Incredibles, different scatter lengths were used for the red, green, and blue channels, which allowed for the characteristic warm glow of backlit skin, while maintaining conservation of energy.
For Ratatouille, an exaggerated subsurface scattering look was required which the Incredibles approach did not accommodate. As the red scatter distance was pushed further, green and blue would dominate in areas where the red had been pushed out. This caused an undesirable “waxy” look. A different application of the technology was needed.
Rather than using different scatter lengths for each channel, the Ratatouille system scattered the irradiance uniformly, tinting the result both before, and after, scattering. This allowed for more artistic control in visualizing the light absorption, rather than differing scatter lengths per wavelength – more appropriate for the looks Pixar sought to achieve.
In addition, the studio non-linearly combined the diffuse and scattered illumination. Areas with no diffuse illumination would have full contribution from scatter. And areas where the diffuse illumination is bright would have little or no contribution from scatter. (essentially a max of the diffuse and scatter, smoothed to avoid discontinuities). Although not physically accurate, this allowed artists to increase the amplitude of the scatter contribution without “blowing out” the bright diffuse areas. It also allowed Pixar to maintain geometric detail in the diffuse component, even with large scatter distances.
Unfortunately, geometric detail was often maintained too well with this approach. To alleviate that, Pixar introduced the concept of “diffuse softening”. Diffuse softening was a second scatter pass with no tinting and a short scatter length. Artists used this in conjunction with the primary scatter contribution, by replacing diffuse with a mix of diffuse and the diffuse softening result.
By applying this softening only on diffuse, it allowed the studio to pick up displacement and specular surface information, thus maintaining sharp details via highlights and reflections. The overall visual effect of the diffuse softening and scatter illumination was a loose representation of bounced illumination.
4. Implementing the Look
Once artists found a general translucent look, the next step was to apply this to a variety of materials. Pixar had objects to shade such as crusty bread, raw fish, cooked meats, fruits, vegetables, cheeses and sauces. There were several techniques and implementation details Pixar discovered along the way.
Since there were often large ridges, dents and holes achieved through displacement, it was important to use the displaced position in scatter grids, especially for the diffuse softening element. The result being the studio could push the intensity of displacements: furthermore, as they were captured inside the scatter grid, they were softened at render time.
Controlling the “wetness look” became Pixar’s next concern. Artists controlled a lot of the surface shine via reflection. The surface roughness values were set up for sharp specular highlights, so they had a small contribution to the total illumination. Relying on reflection, instead of specular, to achieve wetness became a desirable part of the look. Trying to artificially increase the specular highlight would intensify color shaping, which would provide too much tinting, leading to a rubbery or plastic feel. Since reflections were more malleable and controllable by lighting, it gave Pixar a lot of flexibility. However, this did make it more difficult, at review time, to evaluate food shaders in still frames. Often they had to render test turntables, movement of light and reflections across the surface; to establish whether there was sufficient highlight response. It was sometimes less apparent depending on the angle.
Creating the shaders for each object then became a very artistic approach. The art packet was evaluated by the TD and often implemented by one individual, without an additional texture painter. This led to the richness of scatter and illuminations being balanced by one individual, helping facilitate better control of the materials. Translucency was an abstract element to define as a painted texture (especially in relationship to standard rgb surface color), and it became easier to use hand-painted control signals (float matte channels) in combination with a TD painted vertex, varying data to control subsurface illumination.
Pixar also often used procedural and tiling textures over custom-painted textures per model. This allowed artists to create interesting and complex patterns quickly, for reuse via slim. It became important to share these elements since there were many objects going from raw to cooked, or cooked to rotten. It would be prohibitively expensive to redo the work each time. The realization that the general shaders were the same across models (for example, with a grape and a tomato) and adapting a similar shader for both, became part of the technique for dealing with the volume of models.
4.1. Shading Grapes – the basic model
A bunch of grapes is a good case study for the components of contributing illumination. For the shader Pixar used tinting of scatter (with warmth) and lightly-patterned surface diffuse color, allowing more translucency to dominate over the diffuse. Each grape was offset individually from the bunch by a random color shift in hue and value. Artists also used a subtle diffuse blur and a sharp reflection. With this, the elements were in place for lighting to create an appealing image.
4.2. Cheese, Meats and Plated Food – adding complexity
Cheese shaders were helped by strong scattering light, in addition to having the displacement component in the scatter grid to accentuate ridges and holes. An intense diffuse blur amount was used for cheeses and meats to increase the fleshy look. Pushing the diffuse blur, to the point where details were lost, was actually beneficial for a waxy cheese look.
Techniques, such as paint integrated with procedural shading and painted vertex varying data to control subsurface contribution, helped provide additional control in the shaders. Another key technique was using cellular textures, based on voronoi patterns to capture small details when added as a multiplier to scattering amount. This broke up the light transmission look, giving the effect of subcutaneous details, such as striation inside meat.
The different material properties in one model (like this steak dish below) required multiple scattering grids to avoid bleeding of irradiance across discreet food types. It was undesirable to have reddish steak as part of the light potato shader. The exception to this came when shading a rotten garbage pile, where this bleed became a positive side effect for blending the shaders, such as a fake bounce irradiance.
Another key to complexity was adding flecks of spices to cooked dishes. Taking these flecks out of the diffuse blur was important, or else it would have blurry details rather than the desired parsley or pepper speck look. An oily or buttery coating for all cooked foods, was another level of detail. If the objects were cooked together, they would have a consistent coating. It was important to create that as a thin transparent layer on top of the shaders.
4.3. Methodologies: Rotten vs. Fresh, Raw to Cooked
After working on food for some time, Pixar realized the method of starting with raw foods first was an easier approach to shading. It was more difficult to go from cooked to raw, or rotten to fresh. A lot of detail in the cooked versions were helped by modeling simulation softening and adding displacement. Often artists reused the same shaders, with some small modifications; such as adding brushed details, cooking browning, or having the cooked colors move closer together toward a warm brownish hue.
4.4. Bread
The bread was an example of a special case project that fell outside the general approach, requiring a new technique. It still used scattering to create a soft look, but took it one step further into visualizing this softness through the formation of air bubbles inside the density of the bread.
To create this airy effect Pixar used a multi-step approach. The bread interior was a volumetric hypertexture (Perlin and Hoffert, 1989) defined using a procedural density function. The density function was made up of several layers of procedural voronoi noise.
In addition, the important features – such as the position of the crust and position of the torn off face- were encoded in point clouds, which the shader used to modify the density function. The entire density function was also deformed using a texture map. The texture map was applied to the density field using a planar projection. Before evaluating the density function, the x and y positions of the current point were replaced by the value obtained from the texture map. This allowed the studio to compress air bubbles which were closer to the crust, then give a slight twist to the pattern of air bubbles, as often seen in real bread.
5 Summary
We see and eat food everyday, so the challenge was to find this emotional impact of positive associations. Creating shaders in a realistic way, but still fitting into the film’s stylization, helped Pixar do this. By hitting the key subtleties of appealing illumination – saturation, softness, and reflection – the studio achieved its goals for shading.
Original notes from a SIGGRAPH course by Athena Xenakis and Erin Tomson, Pixar Animation Studios
