Using Silhouette and Mocha Pro to paint out clipped highlights on 5+ mins of low quality footage

Hello,

What would be the best workflow and techniques for using Silhouette & Mocha Pro to paint out clipped area on an actor’s forehead? Here’s a 10 second example showing ugly clipping which comes and goes as the actor moves his head around:

This is the area I’m particularly look to paint out:
image

Background info:

I’m an amateur/hobbyist working on post-production for an online theatre performance recorded during Covid lockdown. Actors recorded their webcam and microphone locally using OBS. Source clips are H264 @ 30Mbit/s, but I’m editing on DNxHR HQ proxies. Footage is 1080p 30 FPS.

I’m using DaVinci Resolve Studio + Fusion Studio as my host. I have Neat Video for Resolve to do noise reduction. I’ve installed and am learning with the trials of Silhouette and Mocha Pro, and am considering buying a one month bundle of these two products to help me work on this project.

The actors performed in front of green screens and I will be using Fusion to key them out and composite them on backgrounds to create the impression of them being together in a real place. Being recorded on consumer webcams, the quality is poor. So too is lighting. As a result I have clipped highlights in several sections, like in the above example video.

I’ve done some experimenting using Drag brush strokes applied with Auto Paint, using Silhouette’s built-in Mocha tracking. For a first attempt it seemed to work reasonably well, although the tracking wasn’t 100% consistent.

I then experimented in Mocha Pro and by tweaking some of the Track parameters, such as switching from Luminance to Auto, I got a better track. I haven’t yet tried using that better track in Silhouette.

I also thought about trying Power Mesh, but so far have not had much success getting a mask to lock to his face without slipping. In particular the mesh points on the clipped highlight would jump all over the place, which I suppose is to be expected as it has little detail. I can keep experimenting with that.

I will have somewhere between 5 and 10 minutes of footage to correct (9000 - 18000 frames), so I’d like to automate it as much as is possible. It does not need to look Hollywood quality (and never could given my source footage!), I’m just hoping to be able to paint out the clipping in such a way that the audience don’t notice it, and to enable me to colour grade the footage to adjust the appearance of the lighting. For example, if I want the actor to appear to be in a darkened room, it’s no good having bright white lights bouncing off their head. Especially as this is a 17th century period piece.

Reshooting is out of the question unfortunately.

Any advice on workflow and general tips for approaching this task would be greatly appreciated. Thanks in advance.

You could stabilise that area throughout with Mocha Pro and then in Silhouette pull a rough luminance key to isolate the area in question. Then do a combination of grading and some transform offsets (perhaps with the detail/colour separated) to rebuild some of the skin.
If your tracks are good then it should be fairly procedural.

Thanks very much for the fast reply.

Could you elaborate at all on “grading and transform offsets?” Does this mean a patch - taking another part of the head and compositing it over the keyed area, then grading it to look correct for the target area? Or am I misunderstanding that?

I’d assumed I’d need tracked paint strokes and the Drag brush seemed to work much better than anything I’d tried in the past, eg cloned Paint strokes in Fusion. But if just a transformed patched could be enough, that could be even easier.

Thanks again.

@TheBloke Josh was referring to the Clone brush in paint. The Clone brush can transform and/or grade the clone source.

Ahh I see, that makes more sense. Thank you for clarifying that.

Due to my inexperience I’ve not so far had much luck with clone painting when I’ve tried in other apps or in Silhouette, as I find it hard to get the cloned patch to look right in the new area due to lighting changes. That’s what initially made me look at Silhouette - I saw the Drag brush used in several demos and tutorials by Ben Brownlee and it seemed much easier to blend in a replacement than with clone.

Is there a reason to use clone instead of Drag? Would clone make the workflow easier, once I’ve figured out how to grade the clone to look right?

Thanks again.

The benefit of the Clone brush is that if your source area is near the target area, you are painting from the same frame in time, and the Clone Source Frame is set to Relative (the default), any lighting changes will match in the painted stroke.

There is nothing wrong with using the Drag brush. It’s really what works for the shot.

Do have to be careful with the drag tool if you want to preserve skin details etc as its easy to smear and smudge them away. Using the drag tool on only the colour layer and cloning only on the detail layer can give nice results though

1 Like