Go behind the scenes with Freddy Chávez Olmos on ‘BYE-BYE’ and the actual AI and machine learning tools he used .
VFX artist and director Freddy Chávez Olmos has shared a behind the scenes video for his new short horror film, BYE-BYE. Olmos is also the creative director for artificial intelligence at Boxel Studio.
The film utilizes a number of AI and machine learning approaches, including Foundry’s Copycat and GANs for face replacement, to help complete its VFX work. Here, Olmos breaks the actual tools he relied upon.
Leonardo.ai: “We utilized their customized dataset features and created our own personalized model to generate looks consistent with the style I envisioned for the decrepit old lady character. The use of controlNet features for image-to-image guidance was crucial, as it allowed us to stay true to the actress’s performance and have her movements and poses drive the generative process. By establishing consistent “style keyframes” in Leonardo, we were able to use Nuke’s CopyCat to generate in-betweens and map them to the plate.”
Nuke’s CopyCat: “CopyCat proved to be a powerful machine learning tool extensively used in this project. We not only employed it in conjunction with Leonardo but also relied on it to assist in the rotoscoping process and generate all the DI mattes required for the color grading session.”
Adobe Photoshop’s Generative Fill: “For the paint-out work needed on this particular project, Generative Fill was quite handy. It’s a tool I look forward to seeing evolve in the near future, especially once it’s released for the Adobe video packages.”
Check out the video below, plus a series of behind the scenes images in this gallery.
