Screen Replacement After Effects. Need Help!

[MikeSerks24]

Hey, was looking for some advice. Which one looks better. I know this isn’t the finished product, still need to tweak the edges but I’ve been struggling for weeks now trying to figure out what looks better for screen replacement for this tv in mocha pro I tracked.

Tried using Mesh and Bezier Warp trying to get the corners just right but the top is pretty difficult because of the depth and the right side of the screen is too. Was looking for someones opinion. Which one should I go with? Pic of original shot plus two pics of different inserts. On covering the whole screen but no “3D” depth and the other the opposite.

Opinions and comment to help improve it is welcomed!

[maryp]

For this, I’d use the grid warp BUT only on the lowest segmentation. Then I would use articulated roto to fit the edges perfectly into the screen with a few pixels of edge feathering to blend. If you’re in AE you can do this with a track matte OR you can do it with the insert module by selecting the mask you need and applying a feather in edge properties. Make sure you use motion blur if there’s noticable larger motion in the scene. Render the insert only into AE in the plugin interface and make sure you isolate it with the mask options. Next, I would apply a filter to the image in the screen to make it look more like old TV, there’s some good picks in Continuum for this and then add some color correction to make it fit in the scene better (Scanlines and Damage TV are filters you might consider, or noise if you don’t have that, and Sapphire also has S_Scanlines). Finally, I would screen (or overlay if it looks better) the reflections from the TV over the top to help make it fit.

I think that will get you a much more “real” feeling insert.

[MikeSerks24]

When you mean lowest segmentation do you mean 1 or 4?

Another question if you don’t mind. This might be a novel so apologies in advance!

With that in mind, I have a person moving into the shot. The camera doesn’t move at all but I tried rotoing around the person using the same formula in your videos. Where you track the closet objects to the camera first and work my way down. I also used the puppet method where I track him like an animator and separate body parts.

To point out, I actually watched pretty much all your video tutorials, even the lectures, trying to figure out a different way to make it look more realistic and overall a better roto. I even thought about the ae script lockdown because I know mocha is a planar tracker and thought that thee plugin would help bc you can use mocha with it too and it’s pretty good at tracking difficult shots but it’s like $240 and that’s my last option.

However, I finally got most of the body masked and it looks pretty good in my opinion. I create AE masks with the tracking data and used different variations of feather on the separate layers but my tracking data for my hair is off and looks awkward.

I redid the hair layer hundreds of times and if you watch the video I have the below with the final render, even with a 12 point feather and no expansion, it’s still is noticeable. I do not have Continuum or Silhouette either. Just the stand alone mocha pro but my question is that my hair seems to be moving in and out quite noticeably. How do I fix this?

For one thing, I tried rotoing first then tracking. I also tried using less keyframes and only new keyframes in between to make it look more fluid and not so many adjustments but the hair is tricky because some parts stick out more than others and it changes shape.

Furthermore, It makes the hair look weird jumping in/out when rendered and even with motion blur on both the layer and entire comp. I figure to just cut off some parts of the hair so it stays one shape throughout the entire shot thinking it looks better that way after I rendered a bunch of diff variations, I figured that’s what looks best.

Problem is the clip is 10 seconds so it doesn’t help that the guy doesn’t move out of the frame. He’s actually touching the screen as if he’s trying to grab the hand. That’s the look I’m trying to go for. I know it would have been smart and save loads of time to just use a green screen but I guess I know next time.

It’s annoying because the glare on the tv comes in at some spots of the hair so leaving the points out too much from the shape layer makes the yellow/brown screen highly noticeable and not amount of feather can get rid of it.

I know this is a difficult shot and I’m not asking for a perfect solution but something good enough I can just render out and use because I know it’s possible. If you have a process you would go through with the little amount of knowledge you know about thee clip would be really helpful! Will send the rendered clip, original and screen shots of my layer system to help you grasp what I’m saying!

Looking forward to hearing back because this has been bugging me for weeks and I just want to get on with this project and edit it.

[MikeSerks24]

Here is a link to the rendered clip

https://drive.google.com/file/d/1rV44O5v6NBg4fYjfYUuc1ncXW81t8F0r/view?usp=sharing

[MikeSerks24]

Clip w/o screen. Ik edges are noticeable but just ex.

https://drive.google.com/file/d/1fubDStBgUGOaaI38tVQ3eFxDj3YujAoY/view?usp=sharing

[maryp]

OK, so I mean the grid warp where it’s a grid in four squares.

Unfortunately, as you say, green screen would have solved a lot of this pain for you because you have hair and transparency from the lend going over the top. Live and learn, next time.

Fingers are easy to roto, but transparency is not.

For your roto, you will need to roto the edges and then you need the color data for the edges because they’re semi transparent. This means rebuilding the edge or finding a way to erode the color out. I find the poor mans method of compositing a blurred copy of the original over itself a pretty good way to erode the color out. You may also consider simply layering your rotoscoped person on top of themselves several times to give yourself enough background color to have transparency and then compositing the actor back on top. Any time you have to roto out a transparent object you have to do something like this. In Nuke, this technique is called a color smear. In AE this is a layering process.

You will need to animate per point edge feathering for the complex edge of his shoulder and you will need blurred and tracked in strokes to get better hair mattes. You can ALSO simply try to add hair and face and finger details using a compositing trick called difference matting, and since this is a lockoff shot, it should give you something for the hair and something for the edge of the face to add to your matte you made in Mocha. You can basically make a compoint mask in a precomp and use that as a track matte for your color eroded pass of your guy.

Your screen is also way too crisp to be on a TV like that, you need to add grain and dirty the image up, it looks too perfect for tube televisions of the time.

Hair is simply tricky. Especially transparent hair. I’d try the difference matte or hand painting it for the entirety of the shot and hand animating the blur.

The glare on the TV is something you can get away with if you cut it out of any difference matting you use with a garbage mask.

Try basically creating a fine composite mask in black and white and using that as a track matte and see what you get with those eroded edges. I look forward to seeing what you come up with.

And definitely composite the reflection back on top. But knock it down to where you are happy with it.

[MikeSerks24]

Ok thank you!

I understand what you’re talking about with the TV screen/making it look realistic.

It’s getting the edges back within my guy is what I’m confused with…

When I duplicate the guy on top several times to give myself enough for the BG, do I need to adjust those masks? My screen replacement would obviously be in the middle leaving the original below and the duplicate, however many times, on top right?

Another thing, When you mean difference matting do you mean creating a still image like a psd file? Or like a freeze frame. I’m a little confused with that too.

Am I using the effect difference matte? If so, the stacking order for that would be the main footage with the diff matte effect on top with the mask, my screen replacement below that, the main footage again with no effect below that and last the diff matte psd as a reference.

Couldn’t I just also, duplicate my layer that I roto’d in he plugin and move my screen replacement in between then add curves to create contrast and select use luma or inverted matte?

So please correct me if I’m wrong, I click the apply button in the mocha pro plugin to create AE masks or export ae shape data and copy to clipboard and paste it on a duplicate layer? When I do that I can’t really can’t adjust the masks and see a change as an articulated roto. Am I hand animating the blur in the plugin itself or outside? Should I create a difference matte on that layer I created tracking data or on another and only make adjustments to the masks in the mocha plugin itself?

I’m so sorry for the confusion and bombarding you with further questions but looking forward to hearing back from you.

[maryp]

You’d need to feather the mask out for your transparent edges.

For difference masks, yes, I mean use a still and then use difference to composite edges. You’d be using that to composite WITH your roto together into a black and white matte. You can find difference matte tutorials for AE online.

But there’s many ways to skin this cat, so to speak. You could simply track hairs and composite them on the mask too.

You can use the Mocha plugin to apply the matte, since it’s Mocha Pro you can add motion blur to that rendered matte and use per point feathering instead of using the AE masks. You can do all that in the plugin but rendering the B&W matte and compositing hairs or difference mattes/masks on top would work.

You can also use your luma inverted matte if that works better.

Basically, experiment and see what looks best.

[MikeSerks24]

Hey Mary, sorry for not responding for awhile but. I think I finally got something that I can put out there and I would like to ask for your opinion/ what do you think. It’s below so if you want to view it before you read the rest of this.

So, what I ended up doing was I created two pre-comps of the Body and the Edge.I duplicated the footage for each before I precomp’d them and used Luma Matte on both. The Edge comp is above the Body comp.

Inside the Body comp I copy/pasted the mocha mask onto a white solid layer. Inside the Edge comp I created 8 solid white layers and each with their own difference matte to separate the body parts out because it was easier to get the edges back. When I just used one layer some of the edges were disproportionate. That pretty much got all my edges back.

For the screen. I tracked in mocha. I used it as a screen cutout and rendered inside the mocha plugin. I precomp’d that and added grain to it inside the comp. I also applied a black solid layer above the screen and mocha pasted the shape that gave the screen it’s edges to fit inside the tv and subtracted that and applied alpha matte. All that inside the comp.

I duplicated the screen comp twice and the very top comp has normal blending mode with 61% opacity. The bottom two I used Screen and set both to about 20% to get the reflections back instead of using a garbage mask.

I just have two quick questions. What could I do to get the Red camera dot in the right corner from disappearing. I mean it’s not visible when I don’t bring the reflections back but the reflections look more realistic. Secondly, I think the screen insert isn’t actually a video because her hands are suppose to be moving. Pretty sure it’s a still. I need the actual video. So is it possible to apply an actual video inside the plugin render?

Looking forward to hearing back from you!! Below is a link to where my final video is because it’s too big of a file to send here.

https://drive.google.com/file/d/1PDgXJ7Dn47SRQvgc9tOXUli5f_p7IDeO/view?usp=sharing

[maryp]

This looks a lot better! And yes, you can simply replace your still with video and Mocha will process it the same way. No problem.

I do not see the red dot. But you can just mask it out.

[MikeSerks24]

Thank you!!

[MikeSerks24]

The insert clip import isn’t allowing me to select the video. It’s only allowing me to select jpgs or still images even when I have images and movies selected.

[MikeSerks24]

Actually still having that problem. Changed the MXF file of my original screen insert to a Tiff and Dpx sequence but when I render it inside the insert module inside and outside the plugin it’s still a still frame. I’ve tried reading various articles with a solution but to no avail.

Pretty sure my frame rates the same for each, bit depth too so I don’t know exactly what I’m doing wrong here…

[maryp]

Bring your footage into AE. Under the insert clip option in the Mocha Pro Plugin Interface in AE, you just need to select the footage layer as your insert clip. Have you tried that?