What actually goes into ‘designing’ visual effects for a major feature film?

‘Venom: Let There Be Carnage’ visual effects supervisor Sheena Duggal explains.

Here at befores & afters I don’t always get a chance to ask VFX supes about ‘designing’ the visual effects for the films they work on. Oftentimes what I cover is how a particular shot is planned out, how it’s shot, and then what goes into post-production. All of that is incredibly interesting, but there’s also often an overall ‘design’ component as part of all those stages that we rarely talk about: where do particular character attributes come from? How might a particular scene look? How might a character behave? How will this all be lit, and how does this all impact on the story of the film?

I wanted to explore that side of the visual effects for Venom: Let There Be Carnage, so I decided to ask Sheena Duggal, the film’s visual effects supervisor, exactly what went into her VFX design, plus, of course, all stages of VFX production on the Andy Serkis film. Duggal was kind enough to outline the visual effects design components in detail, noting that much of it happens in a non-linear way and noting that a village of people were involved from all the film’s departments.

Where to start: thinking ‘character’ first

It might seem like an obvious point to make, but in Carnage the CG characters are lead actors. Duggal observes that this is something Tom Hardy, playing Eddie Brock and the symbiote Venom, understood deeply. “I had a lot of conversations with Kelly Marcel, the writer, and also with Tom Hardy, who literally is Venom, and has story credit. Tom was massively helpful in explaining to me what he felt and thought, as Eddie, in his interactions with Venom. Ultimately for Venom, Tom’s performance is driving what we animate to, so it’s incredibly important to know what he has in his head and is imagining Venom doing.”

Duggal says that then telegraphing what Tom and the filmmakers had intended extended all the way to extensively postvis’ing every VFX shot in the film, as well as to the development of key art and concept designs to show look and lighting intent. “Not only did this act as a first pass blocking for the vendors to follow but it also helped the filmmakers and studio visualize the look ahead of the VFX being completed.”

In-house concept artists Adam Burn and Jeff Read, who had been working in the art department during production, also joined Duggal in VFX. “I had a wonderful time working with them both to explore design ideas and key art. I know the vendors found all of this enormously helpful, as a picture does speak a thousand words!”

Creating all the blocking and concept art helped Duggal to explore a lot of ideas and to hold a very clear vision in her head of how everything looked and moved, how it should be lit and interact with light. “I think this may have made it more challenging for the vendors as it meant a lot of push back from me when things didn’t match what I had in my mind’s eye,” she admits. “I think there was more, ‘match the viz and key art’ requested than they normally expect, but with everyone working in their own bubble, I felt it was the best way to maintain a cohesive look to everything and hit the action beats we blocked with editor Maryann Brandon and previs studio The Third Floor. It was fantastic to work with Maryann again as we had a shorthand from working on the first film together.”

Indeed, previs would be crucial, especially, says Duggal, for the areas that required interaction between the CG characters and the actors they inhabited. “This involved working very closely with Jim Churchman, our stunt coordinator, Jacob Tomuri, Tom’s stunt double, and Oliver Scholl, our production designer, as we blocked out viz and edited it with our previs editors Chris Hunter and Richard Ketteridge. Kelly also worked closely with us to help when we had questions related to story. We would spend hours talking about what the characters were doing and why, and stunts would create mud maps for us to show us what they were planning and we’d stage the previs with this in mind.”

Designing a new character: Carnage

Duggal had worked on the first Venom film as a co-visual effects supervisor with Paul Franklin, and so was of course already familiar with how the central character looked and behaved. But the sequel offered up new design challenges, including with Carnage, one that the VFX supervisor prepared for in several ways.

“I read the fan sites and I also spent a lot of time looking at the ways different comic book artists had approached the character over the years, each artist had a unique take on the character and so it was a process of exploring the character abilities that fitted our story. I knew that we also needed to honor the intention of the comic book artists and speak to the characteristics that the fans were hoping to see in our interpretation. It was clear the fans had a certain expectation for this character in particular and I wanted to make sure they were happy, of course you can’t please everyone so I admit it was a lofty goal!”

Duggal launched into the design work on Carnage, working with principal VFX vendor DNEG’s art department to create designs, while production also hired concept artists to work on the character. Says Duggal: “I know artist Dan LuVisi, who is a fan of Venom and Carnage, so he came on board, and it was with Dan that we started to explore brutal anatomical transformations where we see the shared anatomy being broken and ripped apart. Andy Serkis was really excited and when Tom visited my office and left with the concept right off my wall, I knew we were on the right track.”

“One of DNEG’s artists created a wonderful piece of art,” continues Duggal, “that was developed from the idea that Carnage’s body–arms and legs–have equal priority to the tentacles and we called this the ‘vitruvian Carnage’, after Leonardo De Vinci’s drawing. We evolved this image with our in-house concept artist Adam Burn and he created key art that the filmmakers loved so much we see it in the film where Carnage is thrown back against the rose window in the third act.”

In tandem with the evolving concepts, Duggal also decided that it was beneficial to sculpt a maquette to give everyone something physical to interact with early on, rather than waiting for it to be built in CG. The maquette was sculpted by Sebastian Lochmann at KM Effects. “This allowed us all to see how the character was going to look,” explains Duggal. “It also gave me something to do lighting studies with. I used the sculpt and some basic lighting–literally just with a table lamp–to create some images of Carnage with dramatic lighting which all the filmmakers loved and I was able to share with the vendors to show lighting intent and direction.”

Ultimately the maquette was 3D scanned and printed and became the basis of the digital model of Carnage. In-house concept artists Adam Burn and Jeff Read also developed concepts based on the dramatic lighting design. Then, as is often the case in any film, the concept continued to evolve.

“It was especially hard to find the right balance for how he propagated himself and increased his biomass,” declares Duggal. “It looked awkward and weakened the character design when his biomass was attached to him by volume, so I decided to try incorporating umbilical tentacles that keep the biomass attached, leaning again into the birthing idea. As I explored ideas with the in-house concept artists and The Third Floor previs team, led by the amazing Martin Chamney–I realized that we could make this growth symbiotic to the environment. So I researched lindemyer-systems (L-sytems).

“This is a mathematical formalism for describing growth of simple multicellular organisms, that can describe the construction of a wide range of organic structures including trees and the networks of blood vessels in the body, also popular in the generation of artificial life. The recursive nature of the L-system rules leads to self-similarity and thereby, fractal-like forms are easy to describe with an L-system. You can encode a simple set of rules which, when applied iteratively, result in arbitrarily complex geometric structures. This led me to L-systems in architecture where you can create structures and systems from organic forms. Using this idea and leaning into how the ‘blob symbiotes’ moved we devised a similar simulation but based it on symbiosis with the structures in the environment, such as the obelisk in the Ravencroft escape and the spires in the cathedral sequence, you’ll see Carnages growth emulates these shapes.”

Carnage, his powers and more

With an overall design in the works, one of the first attributes of Carnage that Duggal identified needed to be designed was his transformational powers. It became clear that he was going to be a complex character to design and build, and this meant the idea of a single Carnage asset “went out the window very quickly,” jokes Duggal, noting he has multiple weaponized tentacles and he uses his biomass to grow and expand to any size he wants.

“Unlike Eddie who is hosting Venom,” identifies Duggal, “Cletus (Woody Harrelson) births Carnage and they share DNA so we wanted to sell that idea in the way he transforms. So I had to come up with ways to make him scary and dangerous but also add that extra bit of unhinged to his character. We definitely did that with the way he moved, held his body, the weapons, and the rage tentacles.”

Another point of difference was made with Duggal’s introduction of ‘birthing goo, amniotic fluid and membranes.’ “When Cletus transforms to Carnage, Carnage is breaking the anatomy of their shared DNA, violently breaking bones and tearing skin and his body takes over Cletus’ anatomy and is “born”. He’s a serial killer so it wanted to be something more unusual, a little bit gross and to be representational of his origins. I tried to make it about birth and regeneration and birth and regeneration within psycho is bound to go awry. So the transformation from Cletus to Carnage needed to be a bit disturbing and quite unlike the symbiosis that Venom and Eddie have.”

Then there was Carnage’s look and feel. The filmmakers were hoping to avoid having his body appear bloody or flayed, although Duggal does point out that they did explore meaty textures and different references for the different areas of his main structure. “For example, connective tissue was something we ultimately incorporated into his neck and body especially in the areas where we have spaces in his anatomy. Choosing the right tone of red so he didn’t look too gruesome and flayed was also important. We did explore a lot of ideas but we moved away from anything that looked bloody or left a trail.”

Continuing from early designs, some initial early dev work by DNEG (supervised by visual effects supervisor Chris McLaughlin) for how Carnage’s skin reflected light was created. “It looked really organic, having a wet dry look to his skin surface,” notes Duggal. “My goal was to try to create living breathing aliens, I think this ultimately was more successful in some shots than others.”

The behavior and look of Carnage’s weapons, including his tentacles, was developed in continued concepts and postvis. “I also collected a few hundred key comic book images and we went over these on Zoom with The Third Floor, DNEG and Image Engine,” recalls Duggal. “The goal was to share what resonated with our character and to show what poses and weaponization worked best for our storytelling. A lot of Carnage’s behaviors, when it comes to how he deploys his weapons, were referenced from the comics, including shooting projectiles from his body, sticking something into someone’s throat, accessing the internet and slamming tentacles into Venom’s face.”

“References from nature were also used for this, in particular the way a scorpion brings its barbed tail up as a pre-attack threat–you’ll see Carnage’s weapons do the same thing. Also, little details were added like the smaller rage tentacles that break up his silhouette when he’s really enraged.”

For Carnage’s movement, everything would be keyframe animated for final shots. Duggal reveals that in pre-production, performance artists explored how the creature might move. “Ultimately, we worked with The Third Floor to previsualize how Carnage would move and Image Engine and DNEG developed FX simulations that added another dimension to the way he moved and how the tentacles moved with him.”

Another new character: Shriek

Then there was the other new character, Shriek (Naomie Harris). The design challenge here was the look of Shriek’s scream. “We looked at objects crossing the sound barrier and cymatics–the study of visible sound and vibration, something I’d researched in the past and it was perfect for this as it gave us an added extra layer of complexity,” says Duggal. “I had watched a SpaceX launch at Vandenberg Air Force Base and saw some fascinating exhaust patterns from the reaction control thrusters caught by the sun over the horizon. This was an idea we incorporated into the look. Image Engine’s team led by talented VFX supe Christian Irles executed all the scream shots and lookdev.”

“In order to achieve the desired effect,” adds Duggal, “Image Engine relied on two ingredients–the visual representation of the scream, and its interaction with the environment. The visual representation was executed using a combination of FX layers and comp treatment. How it interacted with the environment ranged from replacing live action props to make them fly around a cell room, to fully replacing Shriek digitally so we could make her cheeks, hair and clothes react.”

Going back to Venom: the interplay between VFX and performance

As indicated, for Venom, Duggal and the VFX crew were able to leverage the character design from the first film. However, extra strides were taken to develop Venom’s personality and his relationship with Eddie. “It was all about giving him much more personality than he had and leaning much more into comedy with his keyframe animated performance,” details Duggal. “We also made upgrades to him under the hood, updating his muscle systems, facial rig and cloth interaction. One thing we improved that was important to me was that you see him breathing which he didn’t do in the first film, and I think that helped make him feel a bit more alive.”

Making the CG characters fit and interact in the real world relies on a combination of actors, on-set physical effects, stunt co-ordination, choreography and, of course, visual effects planning. One challenge VFX always faces is eyelines, that is, if the actor as a live-action character is looking properly at a CG character or thing. In the case of Tom Hardy as Eddie and his symbiote Venom, Hardy had not been inclined on the first film to use eyeline guides, as Duggal explains.

“Tom pre-records Venom, and he’s acting with Venom through the earwig in his ear, he’s also imagining where Venom is spatially. I realised after a chat with him that if I give him an eye-line that throws him off, because then he’s focusing on the eyeline and not the performance, it seems obvious in hindsight.

“What I discovered we needed to do was to wait until he had done the rehearsal and imagined where Venom was, and then we’d come in and place the eyeline based on where he wanted it, where he felt comfortable with the eyeline being depending on what was in his head. And that just worked. Once we’d figured that out, it was great.”

Before Duggal had worked out the best eye-line approach, there was one particular scene where Eddie and Venom are in conversation in the apartment in which Venom head butts Eddie. It needed some on-set interaction to be orchestrated. For that shot, Duggal asked Hardy’s stunt double Jacob Tomuri to stand in for Venom. “Jacob put this blue glove on and made a fist, (which was later replaced by CG wraith Venom), and then Tom was talking to the fist. When we did the head butt gag, Jacob did his thing and then Hardy could react to it. We got a really great performance.”

Duggal praises Hardy for helping to ultimately make the visual effects process for Venom’s character incredibly collegial. Kelly Marcel observed to Duggal, as the VFX supervisor relates: “To Tom…you’re Venom. You’re creating Venom, so to him you are Venom and you’re Carnage.” Duggal adds, about Hardy, “He was just so nice to me. Tom talked to me a bit like a co-actor. He’s just a very clever, very wonderful actor who’s got great visual ideas, and is just incredibly talented.”

Crafting the best kind of on-set interaction

To aid in on-set performances, Duggal also employed a range of stuffies, lighting references, stand-ins and other measures for the film. The visual effects supervisor pushed heavily to ensure that Venom’s specific dark features would be properly represented on set and therefore that the principal photography would take account of Venom despite him of course being added in CG later on. This began with Duggal sharing with DOP Bob Richardson what they had learned on the first film.

“Bob understood we’re dealing with this black character who is very hard to articulate in terms of his shape and internal form in any way other than with specular reflected lights. And so backlight and rim is better, flat light or direct light doesn’t help us. Bob took everything we had learned on the first film on board and was amazing, he created the most beautiful canvas for us to build on.”

Actual physical Venom props for use on set were key here, notes Duggal. “First we made a life-sized Venom bust, care of KM Effects. We also made one that you could put on. It could be worn on your head with extra support on your shoulders, when worn by our creature performer Adam Basil it was the correct height for Venom. We also made a helmet that had a tube coming out of the top of it, that had a Venom material ball on the top since the full head was a bit too heavy to wear all the time. KM Effects built the Venom bust and wearable head based on the existing Venom asset. It wasn’t like a perfect lighting reference, because it didn’t have the reflective goo layer attributes, but it gave Bob something really useful to work with.”

A head bust of Venom became another ‘stuffie’ employed as a lighting totem. Similarly, life-sized cardboard cutouts made of Venom and Carnage were used to help set up and stage scenes. The maquettes of Venom (and Carnage) also came in useful just as general reference. And then there was…the tongue stand-in.

“At one point I thought maybe there was a way we could use that for the scene where Hardy was pulling on Venom’s tongue,” recounts Duggal. “We ended up giving him a blue displacement object to hold onto which actually worked really successfully. Still, we had the tongue for lighting reference, and when we put K-Y jelly on it to give it more realistic wet attributes of mucus membranes, that caused hilarity on the set. Also when we were shooting the scene where he’s holding onto the tongue, we put K-Y jelly on Tom’s hands to give us that little bit of reality that he had just been holding Venom slimy tongue, it really helped sell the reality of this beat.”

Designing something a bit different: the animated sequence

For a moment in the film detailing the backstory of young Cletus Cassidy–in which he kills his grandmother, electrocutes his mother, among a number of other harrowing experiences–Duggal worked with Framestore on a ‘storybook’ animation approach.

Dale Newton at Framestore led the team on building up a sequence made of line work, paper textures and blood splatters; effectively all in 3D but in a line drawing style. “When speaking with Dale the hardest thing from a design standpoint,” observes Duggal, “was getting the level of ‘artistry’ correct. It needed to be childlike and naïve, but simultaneously imbuing that dark sense of malevolence and violence into the linework was a big challenge.

“We looked at Ralph Steadman for style ref as there was something aggressive at this style that appealed to us, I had studied cel animation under Bob Godfrey, the great English animator famous for Roobarb and Custard, it’s animated in a deliberately rough fashion known as ‘boiling’, often animated on 2’s and I had always wanted to find an applicable use for the style, we created key frames and stylistically decided where to add in-betweens. I think it worked really well in selling the psycho aspect of young Cletus.”

A complex task made more complex by COVID

When the COVID-19 pandemic hit, Duggal and her visual effects producer Barrie Hemsley were forced to completely re-think the way post on Carnage would be approached. Duggal says: “I hope I don’t have to do something like this again.” The task was enormous; reviewing thousands of shots, remotely, while setting up a technical workflow for that to happen. For example, here’s the tools that demanded Duggal’s attention everyday: emails, iMessage, Whatsapp, Clearview, Evercast, Zoom, Aspera, Microsoft teams, Slack, Shotgun, Q-take, PIX x2x, Moxian, RV, Cinesync, and Streambox. Then there were also the artistic, creative and personal challenges remote working brought.

“It was especially difficult to hold together a cohesive vision across thousands of artists each in their own bubble all around the world and I had to have an awareness that people had all kinds of personal problems to deal with in this tragic time,” shares Duggal. “I had to give a lot more feedback than I normally would expect to, and while we pulled off a very successful film, I feel there are still inconsistencies with the quality as a result of this challenging scenario.”

The big initial hurdle for Duggal was that Barrie and her VFX production side team were in the UK when restrictions arrived. At that point, she did not have a server set up in her house. “Coupled with limited bandwidth internationally, it meant it was too hard to download directly from the server in the UK. I would spend hours downloading aspera data packages from TTF, Editorial, Framestore, DNEG, the in-house team, concept artists and Image Engine and had to manage them on my laptop using only finder. You can imagine how quickly this became an unmanageable data management problem.So until the Morbius team turned me onto Apera drive I was downloading manually for months”

“It wasn’t until I got a team here in LA of VFX editors, Roxy Dorman and Claudia Jolly, and co-ordinators, that it became a bit more manageable for me to function effectively. I honestly don’t know how I did it for four months without this help on my time zone and server set up in my house. Fortunately my friend JF Panisset lives nearby and joined to help me take care of IT infrastructure in my house.”

Duggal and Panisset turned a spare room into a DI suite and a streambox connected to Harbor Picture Company in LA. “I was finaling shots this way until February 21st this year, after which I started going to Harbor in person to project and grade with our top notch colourist Elodie Icter. An added complication to our pipeline was that DOP Bob Richardson wanted to use a BLG pipeline for color–we had a plan for this but had to adjust due to WFH. It was so nice to go to Harbor several times a week after WFH alone for almost a year”

Reflecting on the COVID-induced post schedule and set-up, Duggal posits that “humans are not designed for this kind of input so there needs to be smarter workflow solutions if we are to continue working from home. It has required a lot of persistence and practical intelligence to meet our goals.
 
I haven’t looked at this recently but as of October 2020 I had read 22K emails and sent 6K. That was almost a year ago so I can only imagine it’s triple that by now. Then, for the 1,323 shots in the cut the approximate number of ‘unique’ subs I reviewed stands at 35,128, that’s subs I looked at and wrote notes for. It was quite overwhelming some days.”

An added challenge came from some additional photography acquired during the lockdown in the UK, NYC and Toronto. “Our actors were in different countries although sometimes acting in the same scene,” outlines Duggal. “So we did a lot of previs and techvis to figure out how best to shoot. I was lucky to have the help of first-rate additional VFX supervisor Marty Waters, who was our shoot supe during principal photography and helped supe in post, and he was local to the UK, for that shoot. Andy Serkis is based in the UK so that made it easier for that part of the shoot. Tools like Q-take and Moxian made it possible for me to be connected to set and give feedback on the VFX work. It worked amazingly well.”

“I think a big challenge for all the key creatives on the film was holding together a creative vision with all the VFX vendors working from home and with the new artists that joined our team and vendor teams as time went on,” concludes Duggal. “I learned that extensive postvis and concept/key art really helped us to keep a unified vision, which was essential during work from home.”