Problem with tracking and adjust track on 360 nadir

I’m having a very difficult time with what should be a simple track for Mocha.

Here’s the setup:

The remove layer is rotoed so that the dolly and camera is always covered.

The background layer is tracked in full resolution with perspective turned off because I want Mocha to treat the ground as a plane rather than change perspective as the equirectangular frame distorts during the dolly move.

I’ve set my surface to four high contrast corners on the mat and made a big background shape under the remove layer to give Mocha lots to work with. I’ve animated the bg shape as needed to keep the area consistent and am tracking at full resolution.

The track needs to be absolutely solid and accurate for this remove to work. There’s a little bit of drift but it doesn’t look too bad. Here’s after tracking:

I would think this would be easy to fix with Adjust Track, but I’m just not able to get it to work. After setting all master frames at frame 0, what exactly is the proper workflow for a scenario like this?

Do I set an adjusted keyframe at the end, then work backwards? Or add keyframes along the timeline from frame 0 out?

On top of that, the behavior of keyframes is very inconsistent. Here’s the upper left corner. It’s solid with 4 keyframes:

Here’s the upper right corner. It’s not even done and it’s a mess, sometimes needing many keyframes one after another:

Why would the surface jump immediately after a keyframe? I don’t understand what’s going on here. If I didn’t know better, it feels like keyframing adjust track on the nadir is buggy.

This is a very simple shot. I feel like this should be way easier for Mocha to do and for me to adjust. What am I doing wrong here? Any help much appreciated. Thanks!

Not really sure how to help specifically.

To test your track, use the Stabilize to test your track. If there’s a significant amount of lens distortion, it’ll show up in a very obvious way.

If I can’t get a solid track, I hide any drift by utilizing the large ‘feature-less’ areas. The eye doesn’t notice drift as much if there’s no nearby reference points.

Often I use the splines in Nuke as a wrap source instead of a 4 corner pin.

T

I think you’ll need to make your rig holdout shape larger, the seams are pulling towards the nadir and that will definitely affect your track.

Hi Mary,

Thanks I’ll give a larger holdout a try.

Could you please say more about how to properly use adjust track? I don’t understand why I’m getting such wildly inconsistent results with it on the nadir.

Is my track so far off that adjust track won’t work?

Is the nadir problematic in general for tracking?

Aside from a larger holdout, what advice would you give for tracking this shot? It seems to me it should be super easy, but maybe it’s more complex then I thought.

Thanks for your help!

If the original track is all over the place, using adjust track isn’t going to give good results either. It’s meant to be a correction over the top and only for things like drift, so that is probably why you are getting such results.

The Nadir is only problematic when the stitch itself is problematic, the better the stitch, the better the ability to track.

I think the reason you are having so much trouble is because of the pulling and inconsistencies in the nadir. Mocha often sees texture issues we do not. Try retracking with the larger holdout and then see if adjust track works any better for you. If you are still having issues it me know and I will probably need to take a look at the shot on my end and see what I can do with it to troubleshoot. I am working off of stills and that’s an incomplete picture of what’s going on.

Hi Mary,

My stitch is clean and the track actually is not all over the place. It does drift but the drift is relatively minor, far less than in your “Troll Face” demo on YouTube.

I did another test attempting to use a larger hold out shape as you suggested. In the process, I found that shape keyframes are definitely broken on the nadir. I made a screen grab video to demonstrate:

In this demo I attempt to correct the drift of the lower right corner by setting a keyframe with the correct position at the end of the shot. As you can see, after setting the keyframe, the shape is scrambled in the transition between keyframes. Note how at 29 seconds, the feather moves inside the shape:

Removing the keyframe restores the correct in-between behavior, though I still have the problem with the lower right corner. How do I make this work?

This buggy keyframe behavior also happens in the Adjust Track module. This makes getting a working track on the nadir impossible in many situations.

Please take a look and share with your development team. I’m also happy to share my project with you for further analysis.

Looking forward to your thoughts, thanks for your help!

Good to know, let’s definitely take a look at your shot then. Can you email me at Mary P at Boris FX dot com and let’s see if we can’t figure this out? I’ll download it and troubleshoot.

I will be happy to loop the dev team in on this and see if it’s a problem we can solve on our end.

Thanks!
Mary

Thanks Mary,

I’ve posted a .zip file with the assets to a shared dropbox folder and sent you an invite. Will also send you an email with the link. Thanks for your help and please keep me posted!

best,
–David

Hi David,

My emails keep bouncing, but I tried to send you a file. Please put me on an approved sender list so I can reply to you. :slight_smile:

Cheers!
Mary