PI Mocha track failure depending on resolution of clip?

Using Vegas Pro 17 and tracking some 1920x1080 59.9 fps footage and have a pretty solid track.

I decided that 60fps was higher than I needed and because the VP pipeline is cpu heavy it also slows things down unnecessarily so I converted the clip to 30fps.

Because Mocha doesn’t interpolate tracking data when used with VP, I had to re-track. No big deal, I knew that was coming.

But a few seconds into tracking the world center, the center point and search area jumps to a completely different location from one frame to the next. The track is flawless every time with the original footage and jumps with the converted footage. [see below for an exception]

While Mocha can be sensitive if I don’t include enough variation in detail with the 59 footage, it’s much more sensitive with the 30. I can include the fire hydrant or it and grass and at some point the search area will just jump somewhere else. Sometimes I’ll get a “tracking lost” message but usually just a discontinuous jump.

I’ve upload a sample project with working and non-working tracks if you want to try it on your end.

https://dragonseye.com/Tracking.zip

[the exception]

While I was writing this, I was giving AE a trial and tracking the original 59.9 fps footage and that track failed once. I haven’t been able to repeat that failure yet.

Very interesting shot Michael. I am reproducing the same results that you’re getting and I’m using AE on the Mac, which rules out platform and host as a potential factor. I’ll run this past some of our resident Mocha experts to see if they understand the source of the problem. Could be related to he wobble that I’m seeing in the shot but that’s just my guess.

Thanks,
Peter.

@PeterMcAuley I didn’t think to try this before, still getting used to all the knobs available in mocha, but if I change the input channel to Auto, the tracking works fine all the way to the end. I can see that mocha is also dynamically changing the % pixels used.

In fact, auto channel is giving me an even better track than I had with the original footage. It’s rock solid on exactly the point I selected, all the way through the clip

Hi Michael,
I am downloading your clip now and yes on first view before trying myself, I would suggest increasing the % of pixels tracked (found in “classic mocha” workspace). Also since you do not need perspective or skew for Particle Tracking, tracking only transform & rotation would also help. Generally if a Mocha track stops or falls off, you can typically solve it by changing the shape and increasing the amount of pixels tracked. Sounds like you are on your way. :wink:

General rule of thumb for VFX is do do your tracking on the original frames before any speed ramping.

@PeterMcAuley Ignore that last, other than the comment about the point staying rock solid.

So, just before my last comment, I put the playback point just before the jump and tracked using Auto. It worked great all the way to the end.

So I decided to track from the beginning of the clip and get a jump at fr 115. Previously it had jumped at 110. I’ve been able to repeat this by opening the original project, finding the jump at 110, going back a frame and auto tracking forward and all is good. Jump to the beginning and tracking again and there’s a jump at 115. If I track backwards, things look good until I get to fr 60.

I’m also seeing some behavior that makes me wonder if there are invisible keyframes. While playing around with this some more, I’ve been able to get mocha to pop up the track failure dialogs and I can see keyframe triangles on the timeline until I dismiss the dialogs and then they disappear. If I scroll through the timeline, my motion selections will change as though keyframes are set, though none are visible on the timeline.

I can get a video for you if that will help.

@rosss Thanks. There’s no speed ramping here, just a conversion of the video from 60fps to 30. Since Mocha won’t interpolate tracking data with Vegas Pro and I didn’t need 60hz for this project I converted the video.

I’ve gotten varying results with different motion params and as you imply, I’m working my way toward a solution.

I keep finding small issues but hope I’m earning my keep as a tester :slight_smile:

The latest is this…

This happened when the track failed and I tried to resize the search area by dragging one corner. I’d only moved it about 10 pixels when the curve went crazy

Hi Michael
I did a test with your 30fps clip and was able to successfully track the entire clip through without any errors OR having to stop and reposition.

The first test was done with transform, scale, rotation only and tracking the hydrant. This works fine if you wanted to attach a particle to the hydrant.

Looking at your image, I assume you want to attach to the spot on the ground, so instead, I was able to get a solid track but using perspective and focusing the search area on the spot and also including a small area of the grass to help the track:

I hope this helps.

And here is an AE project if that helps you see what I did. (Sorry I do not run Vegas).
https://drive.google.com/file/d/1CAmPbeZbNxrB8Umyl5AS7W9TeHgupW8R/view?usp=sharing

@rosss Thanks, I’ll give that a try.

What tracker were you using above? Corner pin?

I used the Mocha inside PI and tracked with perspective. While PI only needs transform and rotate, enabling perspective helped keep the track on the ground.

The shot is pretty compressed, so this along with the frame rate conversion might have effected the results you are seeing. But generally I was able to track the clip fine.

Note I did see some odd behavior animating the splines after tracking this shot, but this was not a necessary step to get the track back to PI. HTH

@rosss It’s interesting. I can load your project and think I see what you did but can’t reproduce your results no matter what I’ve tried. I’ve set all the parameters as it appears you have and even rotated the search area slightly to match the start in your project.

When you tracked that in AE, did you start at frame 0? I notice that the search curve is already rotated/skewed a bit rather than a horizontal oval as though the tracking had been run in reverse. Or did you rotate it slightly from the beginning? If the latter, I’m curious about your thought process in doing so. I’m still climbing the learning curve of mocha so any insight is appreciated.

One other thing that keeps happening is that the motion parameters keep changing on their own. Every time I run a track on this footage at some point in the timeline the motion params appear to change to tran/scale/rot where I’ve been starting with all 5. What’s doubly frustrating is I can’t drag the current position and see the parameters change, it’s only when I’ve clicked in the timeline past where an invisible keyframe sits that the parameters change. There’s an example here. Ignore my parameters in this video as this is a capture from much earlier and I’m just sharing this to demonstrate the changing parameters during tracking.

One last thing. Since some of the work is being done on the gpu, could this be a gpu/driver issue?

My current system specs:

Processor Intel® Core™ i7-8700 CPU @ 3.20GHz, 3192 Mhz, 6 Core(s), 12 Logical Processor(s)
BaseBoard Product ROG STRIX Z390-E GAMING
Installed Physical Memory (RAM) 32.0 GB
Adapter Description NVIDIA GeForce GTX 1660
Driver Version Studio 27.21.14.5177
Resolution 1920 x 1080 x 60 hertz
Video Memory 6G GDDR5

@rosss I know I’m throwing a lot of details at you. Sorry.

One more. In reference to my gpu question, I rebooted, started only AE and tracked and quickly got this

Got a similar failure with the original footage just now in AE.

If I make the search area very much larger with the original, I was able to get a solid track. [edit later, a much larger area also works with the 30hz footage]

I was hoping to come up with a more deterministic process, but the lesson I’m coming away with at the moment is “try it until it works.”

What are your thoughts on that?

and thanks for all the support. It’s really appreciated.

Hi Michael,
Sorry to hear you are still having issues. I know that Mocha can be a lot to learn at first. I’ll try to answer all the open issues:

  1. GPU - It is possible you are having GPU issues. I would suggest opening Mocha (inside PI) ,then under File>Prefs> GPU. Disable GPU processing:


    You do have to exit Mocha then relaunch it.

  2. Keyframing. Please try with a fresh plug-in instance. You had inadvertently keyframed your tracking motion options. This can happen is you stop and start your track and change the tracking options. Mocha’s default keyframe mode is aut0-key, so you are always writing KF. It was not a bug, the fact that these KF did not show up is normal. When you select a Mocha layer, it by default will only show you the shape KF in timeline. If you click on a panel, such as the Motion parameters, then it will show you the KF for that particular module tab. You actually showed this in your video recording by accident.
    image
    If you click just to the right of translation or scale, the interface will highlight that panel and you would see the keys related to those parameters. Typically this is only something that an advanced users comes across.

  3. Tracking start. In my project I think I started my track part way into the clip and tracked backwards, then forwards. This should not really make a difference as I did not add any keyframes or manually edit my shape. If you wanted to attach a particle to the hydrant, you can get a good track with translation only. Because the driveway has very little detail and the camera move has a large perspective and parallax change, perspective tracking is more necessary on this particular shot even though we are only using the X,Y track data.

Here I will detail another way to think about solving the track.

  1. I rest PI and applied a Smoke preset:

  2. I closed PI, set transform to World and and launched the Mocha tracker:
    image

  3. Instead of using the Circle shape, I want to create my own tracking shape, so turn off the the World tracker (gear) and hide the visibility.
    image

  4. Now on frame 1, I create a new shape with the X Spline tools and focus this on the driveway, avoiding the hydrant because I know this will occlude the area I am tracking later in shot. Enable perspective, keep % of pixel high. Track shot forwards. For me this tracks with no errors with GPU on or off.

  5. After I track, I like to make sure the track is valid. I go back to frame 1 and enable the SURFACE tool. I position the corners of the surface to match the ground and the center point to where I want my particle to attach to. I could also enable the GRID tool to help visualize the tracking:

  6. If that looks good, I then select the Word Center tracker in the layer list and link this track to my shape layer (Layer 3). You can position this layers offset cross-har at the same place as the previous track.

I hope this helps. Mocha inside Particle Illusion has the preset CIRCLE Trackers to assist new users. However, when tracking an object that has very little detail (like the driveway) it can help to use custom shapes and define the “plane” that you want to track. Mocha is a planar tracker (not a point tracker) and once you get your head around it, you can solve lots of shots.

HTH!

@rosss Thank you very much for that detail. I’ll have to run through the process based on this in order to fully grok it but I’m really glad to see your last detail.

With other tracks I’ve done, I’d gotten into the habit of creating my own splines, tracking those and then linking the emitter to those layers. One reason for doing it that way is my previous tracks had more difficult (I thought) objects to track and I needed to track edges of moving objects in the scene.

This thread came about because I wanted to create a beginners tutorial on using PI in VP and tracking that way felt “wrong.” My thinking was that since Boris had set things up to use those specific shape and this was such a simple scene, I should use those shapes for tracking and not try to do an end run around what the software/developers wanted. That thinking was reinforced a bit when I saw your version of this work and there weren’t any custom shapes.

My takeaway now, barring any new details, is that I should feel free to use the tracking methods that get me closest to the desired results. If the program lets me do something (like linking the emitter to another shape) I should just do it and see what happens. If it works, it was the right thing to do.

If I’m right in that thinking, it kinda reminds me of a product I worked nearly a decade ago. Had a dozen different ways to do most operations and none of them were “wrong” just some way were more “right” in certain situations.

It really depends on the shot. We provide the default circle track shapes because in many, many cases this works fine. For a brand new user, this is 85% the thing to do, but on some shots (like you have found) you do need to resort to more advanced tracking techniques.

Since you shot this clip yourself, I would also recommend thinking about the trackable areas. While at first glance tracking a driveway might appear to be simple, but if the footage is blurry, compressed, or does not have much texture, it can be difficult to track. The pure black and white of driveway = less texture to track. Hope that makes sense.

Certainly, for all users, it is good to have a sense of the advanced techniques to solve tracks.

Last note - it can be fun to play with both the world and emitter transforms to get different looks, depending on what you are going for. I attached just the world emitter to get the general motion but you can attached both the world and the emitter (for a moving object) to get very realistic, natural motion happening.
Best,
Ross

@rosss Yeah, I’ve learned a lot since I originally shot this. When I started prep for the tutorial I immediately thought “I wish I’d known about tracking markers when I shot this” but I decided to press on since I won’t be the last newbie to make that mistake. I also need to develop a good process for removing the markers in post so for now, this is “good” footage given where I am on the learning curve. I have Silhouette but that’s a whole nother curve :slight_smile:

Your comments about the driveway make complete sense. Attempting a track on this footage also gives me a more concrete sense of what mocha can and can’t do. It often seems like magic but it’s helpful to see first hand where the magic stops. I’m also starting to develop an eye for what works well in mocha and what doesn’t.

My tutorial is going to touch on using world, emitter and both since I found them a bit unintuitive at first. It also took me a bit of trial and error to figure out what makes for a good world center.

Thanks again.

1 Like

Finally got the tutorial out. It’s not original material per se but I hope it’ll help other people starting out.

Hi @michaelh. Thanks for the follow-up and for posting your video tutorial. I am sure it will be useful for other Vegas users. Looks like you are getting on well with the Mocha tracking.
Cheers, Ross

@rosss Making progress though I still get caught out sometimes and left wondering “why isn’t this case just like the last one” but I truly appreciate the level of support from Boris.

1 Like