When NUKE is used for more than just compositing

‘I know this sounds crazy.’

We all know that Foundry’s NUKE is mostly utilized as a final compositing tool in visual effects, right? But what if it was used in pre-production?

Yep, pre-production.

At least, that’s what director and visual effects supervisor Victor Perez has managed to do on several of his projects, including the VES Award-nominated independent short film, Echo. Perez effectively simulates sets using whatever assets or panoramic photography can be quickly pulled together to help stage scenes before they are filmed.

He might not be the only person to use NUKE in this way, but here Perez tells befores & afters about how he has adapted the compositing tool for a different part of the visual effects process.

Victor Perez

b&a: So, why do you use NUKE at this early stage of the process?

Victor Perez (director and visual effects supervisor): Well, working on independent movies, one of the main issues is the budget. You cannot have entirely whatever you want. You just need to be very picky and say, ‘Okay, I’m going to need this, this and this because I have to spend my budget within what I have.’ That’s why I’m using NUKE for simulating scenes. I know this sounds crazy.

Get exclusive content, join the befores & afters Patreon community

b&a: For ‘simulating’ scenes?

Victor Perez: Yeah. I’m simulating sets. Whatever we are going to shoot, even if it’s a greenscreen stage or is a location. If it’s a greenscreen stage it’s easier because you have, usually, all the measurements and everything, but for being on location, that’s different.

b&a: OK, where do you start for location shoots?

Victor Perez: I can just give the location scouts a Ricoh Theta which takes a 360 degree panorama, or I use it myself. The camera is amazing, and it’s relatively cheap. What I do is I get that Theta in a selfie stick. You can find that on Amazon for five euros, a very long selfie stick of three meters. What I do is I just take a photograph from the highest point that I can. If I believe there are different points of occlusion, say something that I cannot see from one single photograph, maybe I will take one from another point.

Ricoh Theta frames from a swimming pool shot. You can download those below, plus the NUKE set that Perez staged with them, below.

b&a: And what are you doing with that panoramic imagery in NUKE?

Victor Perez: What I create is what I call an omni-projector. It’s a projector that is going to project in 360 degrees. I’ll use a meter rule on set – just from Amazon.com, that you can even fold and put in your backpack. You put it on the floor. You take the photograph and then you know at least at that position you have a reference for the size.

Then with the omni-projector you get a floor that is one card, which is the easiest geometry ever. You put a card in the floor, you align and you use the meter rule as your reference. Just by having that, you have the floor. Then if you need to create more complex geometry, you use more cards, because at the end it’s just aligning planes and surfaces.

So, for example, right now I’m doing a project and it’s an underwater project. What I did is re-create the pool in 3D using cards because cards are cheap, are easy to handle and I don’t need complex renders. What I need is just to see in the 3D scene an idea of the set. Then with that I can put cameras in there, that are going to be the cameras that the DP is going to use, the lenses he’s going to use. Then I can just create a basic virtual set of that. Then I’ll be talking to the director, and the DP, about the scene with something that I can show them.

The swimming pool set as staged in NUKE.

b&a: So, just to be clear, do you need access to the set before filming in order to build something up like this in NUKE?

Victor Perez: Well, initially what I would do is create a set kit, so you don’t need to have a proper set. What I also create with these set simulations is a set-up that lets you stage things with 3D humans, like basic mannequins, just to put the people in the environment and work out where they were going to move, or hide, or to work out the positions of the camera.

Then with that, out of NUKE you can create a document that gives you the position of the camera, height of the camera, lenses you are going to be using. A lot of this stuff you can get with other applications but what’s the nice thing about this is you can have a previsualization of how that is going to be looking on the final shot.

b&a: So, it is really a kind of previs?

Victor Perez: Yeah.

b&a: Would you call it previs?

Victor Perez: I don’t call it previs because it’s very rough and very cheap. Previs is something where you can see the shot more or less, even if it’s rough, but you can see animation. With my approach, I can see staging. For me it’s more of like a staging setup.

And then once you have that staging, it’s kind of a storyboard, but it’s technical and you have a lot of information. For instance, if it looks like your camera is going to need to be very, very high, you might then need to request, for instance, a crane, or you can just talk to the director or the DP and say, ‘Hey, do we really need that crane just for one shot? Can we do this slightly lower or can we just search for an alternative for that?’ Sometimes maybe they don’t think about the economy of the visual effects in terms of spending a lot for one shot.

b&a: What might be another example of where you’ve used this approach on set?

Victor Perez: This has helped me with greenscreen calculations on set. With this set-up in NUKE you have a previsualization just in the viewer, and you can see how big the greenscreen is in relation to everything else. It can be helpful with just getting a figure of reference, or placing and understanding heavy obstacles. For example, I am going to start using airwalls, these massive blow-up greenscreens. You need to figure out not only the position but also, because it’s an exterior, the pattern of the sun on the sky to understand if the big wall that you are putting there is going to project shadows over the set.

For that example, I’ll also use the app Helios which helps you understand the movement of the sun. With that I can re-create the pattern of the sun with a sphere in NUKE. Just by rotating the sphere I’m going to understand, ‘Oh, I’m going to have shadows in here. Let’s rotate everything by 20 degrees and that is going to solve a lot of problems.’

b&a: Why do you tackle this set simulation in NUKE? Is it because you’re comfortable in the tool? Could this also be something you do in Maya or another tool?

Victor Perez: Absolutely. But for me it’s NUKE because it’s the software that I know the most and, at the end of the day, with Maya or any 3D software, you have to generate renders. You have to hit render, wait for the file to be rendered. Even if it’s going to be really fast but you are going to rely on output, you need to go outside the software.

But in NUKE, you can see everything pixel for pixel, so, in real-time, because it’s just using a scanline renderer. And in the end, what I’m doing is just converting a pixel very roughly. I can have a 3D visualization together quickly. I can have everything together in real-time in the software. Sometimes I get just simple assets and then they are moving in 2D, so I’m not necessarily using fully 3D because cheating is better than have good quality.

Behind the scenes from the greenscreen and motion control shoot on ‘Echo’.

b&a: Does doing this in NUKE allow for any advantages when it comes time for post-production? Can the scene stay in NUKE as compositing unfolds or is it really just for planning?

Victor Perez: Well, I always provide this to the vendors. I can share this with them to give them an idea of what to expect. I like to have the vendors in place before I start the shooting because I want to get them involved in the process, asking them if they need something special. At least they know a bit of information from the cameras, for example.

For Echo, for instance, this approach was really good because the cameras were real and there was a lot of motion control. I could say, ‘The cameras are going to look exactly like this. They are doing this kind of movement. Can anybody help me to understand what are going to be the issues of this?’ Then we were discussing edges of the panels in the film or about being careful with the lighting of the panels.

b&a: Is this something you use for just your own projects?

Download the pool panoramic photos and NUKE scene here.

Victor Perez: No, I’m going to release it. I believe I don’t ‘own’ this technology. I just figure it out. I didn’t invent anything because the software’s not mine. I really love to share what I do. At the end of the day, what I believe is that it matters what you do with the software or with the setup, not the setup itself. I’m going to release it on Nukepedia for free so anybody can just take it, open it, implement it and…call me. [laughs]

b&a: What is it that you’re releasing? What actual components?

Victor Perez: What I’m going to share later is the main set-up, the omni-projector and maybe a very little tutorial on how to create that on set. I actually want other people just to find it and maybe even to tell me, ‘Actually, there is another way better than the one you have.’ It’s a way of improving myself and the community.

So, tell us in the comments if you also NUKE like this, or in other non-compositing ways.