A breakdown of the ‘An American Pickle’ Seth-Rogen-as-two-characters shots.
You may have already seen the behind the scenes HBO Max video (embedded below) that showcases the ‘twinning’ work behind Seth Rogen’s An American Pickle, in which the actor plays both an immigrant worker, Hershel Greenbaum, and Hershel’s great-grandson Ben.
If you haven’t seen it, the video shows how the two characters appeared in many scenes together, often with moving cameras. This was thanks to the efforts of production visual effects supervisor Adam Rowland, and the team at visual effects studio Nviz.
But how were specific decisions about how to shoot the splitscreens made? And what did it then take to realize the final shots in compositing? befores & afters asked Rowland and Nviz visual effects supervisor Jason Evans these exact questions for a new look behind the scenes of the film.
Choosing twinning techniques
Rowland told befores & afters that a variety of different techniques were used for the twinning, the use of each was largely determined by the action, camera movement and the location. “Wherever possible we would have a static camera to enable a more straightforward splitscreen,” he says. “Of course, because of Seth’s beard, the two sides of the split were often shot months apart, so the word ‘straightforward’ is somewhat misleading—we were never able to lock a camera off and return to it, as you would normally hope to do.”
When the lighting and environment were controllable during typical on-set shooting, the ‘simple’ approach became the starting point for splitscreen shots. “As layers of complexity were added, so too the VFX approach would evolve,” notes Rowland. “Scenes were blocked out, and Seth would usually have the previous take playing back in his ear in order to time his responses accurately. We often added a bleep-track when he was required to sync to non-verbal cues.”
Then, in scenes where there was heavy interaction between the two characters or the camera was handheld or moving a lot—such as a fight scene—the twinning approach became one of face-replacements. Says Rowland: “These tend to work quite well as fleeting moments rather than long shots, so action set-pieces are ideal.”
Greenscreen shooting was only ever used as a last resort, reports Rowland, where there was no guarantee of being able to line up the two sides of the split due to changeable conditions, specific background action, foreground crossing or a moving camera. Most of these greenscreen splitscreens were for exteriors.
And with exteriors, a lot more planning was required. “We relied heavily on getting back to the same location and having identical, or as-close-as-possible, lighting and weather conditions,” explains Rowland. “Much respect to the production team for the complicated scheduling of this!”
“Because we were shooting in Pennsylvania between October and December when there is a dramatic shift in the seasons, this was not always possible,” adds Rowland. “In cases like this, when backgrounds were difficult to match exactly due to altered conditions, we would rely on a greenscreen.”
For example, one scene featuring both Ben and Herschel in the same frame see them visit a graveyard. It was shot very early on in the schedule when the trees were still verdant and the light was warmer. “As this was a moving splitscreen,” details Rowland, “we shot only the background plates, and the Herschel singles on location. The elements of Ben and Herschel walking were then shot in two separate sessions with Seth walking on a treadmill against greenscreen in the studio. The singles on Ben were also captured this way.”
Another scene was filmed in heavy snow—a scenario that would have been tricky to confidently replicate when returning for the ‘B’ side of the shots. Here, the second splits were planned to be shot in the studio against green.
Meanwhile, a scene where the two characters push over a tree blocking their path was shot on location. “We were lucky here that it didn’t snow in the intervening time,” says Rowland. “However, the leaves on the trees were noticeably different, so we used a greenscreen on location for the second split. When we weren’t able to return to the same conditions, John Gulesarian, the DOP, did a great job mimicking the location lighting in the studio. Without his attention to detail, a lot of these would have spent more time in comp and not necessarily looked better for it.”
Filming aids for splitscreens
Since the actors would be either filming scenes separately, or relying on an actor double (Ian Poake), Rowland instituted a number of things during the shoot to aid in eyelines, interaction and timing.
“The use of a double with a convincing likeness was very important, as it will often be the back of their head or the side of their face that you see out-of-focus in the foreground of a shot,” states Rowland. “Hair and make-up departments had a lot of work to do here to ensure continuity, and Seth’s double Ian had to very closely observe and mimic Seth’s body language, posture and general physicality.”
Using Ian as a stand-in was usually preferred, says Rowland, since the shadowing and bounce-lighting would often be noticeably different without somebody occupying the same space. However, when certain interactions were required that had to be pixel-accurate, the team would rely on a C-stand/tennis-ball so that Seth would be able to reliably hit his mark in space, be it a glance or something more physical.
“Returning the camera to its original position was sometimes quite complicated,” continues Rowland. “On set, we were able to tag the floor with UV pens to mark exactly where the camera had been for a specific shot. On location it was trickier, and we used a combination of rivets drilled into sidewalks, stakes in the ground and spray paint, depending on environmental constraints.”
For video playback, QTake was used (operated by Matt Gorbachov). “He had the camera feed dissolved with the previous take streaming to several iPads on set, so everyone from props to camera and lighting were able to check they were happy with everything,” says Rowland. “He was not only invaluable in helping us get the cameras lined up, but was also able to create live-splitscreens and greenscreen composites so that we could see straight after a take what was working and when something was not.”
Bringing it altogether
With plates in hand, the shots were handed-off to Nviz to integrate everything in compositing. “The challenges in compositing the scenes were normally in the things that were impossible to see until you actually lined the shots up in post,” admits Nviz’s Jason Evans. “The split shots that took place in Ben’s apartment ended up being quite challenging in this regard. The apartment was a set build which meant that the frame was constructed of timber studs. The timber naturally settled over time and so the set itself bent and warped with all the heavy gear moving around on it.”
“Normally, this wouldn’t be an issue as it’s only a tiny bit of warping, but in this case we were lining up obvious horizontals and verticals like window sills and brick lines; these were often quite noticeably out due to the plates being shot many weeks apart. The team used a mixture of patching and warping around the seams as well as choosing some quite intricate split lines between the plates to problem areas.”
Face replacements also posed a specific challenge for Nviz. Evans comments that the best results often came from where compositors could choose to join the replacement to the plate. “The simplest solution was to do a full head replacement using the collar as the join. This worked very well on over the shoulder shots, specifically, but on most of the shots we had to hold on to at least part of the head from the stand-in to make it feel natural. If the replacement element lined up nicely then the team could use the markers placed on the stand in’s face to simply 2D track and warp the replacement element into place.”
“Where this didn’t work,” says Evans, “a slightly more complex solution was required. On a couple of shots the elements didn’t align very well due editorial changes or to the element being slightly off in angle or height. Here we ended up using a projection method, projecting the element onto 3D scan of Seth and then tracking that 3D model to the plate. We then tidied up any missing texture using paint or using reference photography. This technique gave us the ability to subtly adjust the angle of Seth’s head whilst still retaining the lighting, texture and any micro movements, keeping us out of that pesky Uncanny Valley!”
One of the most complex scenes in the film sees Hershel and Ben strolling down a New York street sequence. It was a delicate mix of accomodating a dynamic live-action shoot while telling a specific story. Many things changed between filming days, as Evans explains.
“On this location they had actually put up Christmas lights in the intervening time between shoots! One of the challenges specific to this sequence, however, were the extras who walk up and down the pavement. They needed to be moving naturally along the street but not intersect with where Ben would be, even as he moves perpendicularly across the flow. This took some very careful planning and choreography and a few takes upon return to really dial in the speed of Ben’s movement.”
“In post,” details Evans, “one of the bigger challenges with blending the plates was integrating the reflections from the two plates, especially when the doors behind them are in use. Reflections needed to be lifted from the first plate and then added to the second and any reflections running across the doors as they opened and closed needed to make sense. The comps ended up being a very multi-layered affair with a few shots needing some rebuilds in places for extras and their reflections.”
