What's the best roto workflow for non-planar shapes?

Looking for advice on best practices when doing roto work on difficult, non-planar shapes such as:

  • faces or other organic body parts that are in motion or are turning in space
  • fast moving non-planar shapes with motion blur
  • moving shapes that can’t be broken down into a planar surface

In cases like this, the recommended technique of tracking a loose plane then linking a tight roto spline to the track doesn’t work well because the track falls off too easily.

I seem to get better results tracking a tight roto spline then adjusting the shape to fit. However, I keep running into problems.

Is this the best method for rotoing organic non-planar shapes?

What is the best practice for when to set keyframes when adjusting a spline when the roto shape falls off?

Is it better to stop tracking, correct the spline, then track backward, or just correct and continue forward?

How do you best determine where to make a spline correction and add a keyframe if the goal is to have a few keyframes as possible?

How do you prevent the spline from falling off a tracked shape between keyframes (which seems to frequently happen)?

I find the relationship between track data and roto shape keyframe data in Mocha very hard to understand in this situation.

I can’t figure out if I’m using the wrong technique, if key framing is buggy (I’m pretty sure in some cases it is), or what’s going on.

The video tutorials and manual don’t discuss these details.

I’d really appreciate clarity on this topic so any advice much appreciated.

Many thanks!

OK, get ready fo a novel. :slight_smile:

For faces or other organic body parts that are in motion or are turning in space, and with almost ALL organic roto (roto not on rigid planar surfaces) make sure you cut perspective tracking off. You may need to switch to translation scale and rotation only, or translation only, in some cases. In these cases, more manual keyframes will be needed.

For fast moving objects that are so quick that they are moving non-planar shapes with motion blur, manually keyframe those frames, and continue tracking when you are more likely to get a track. You can also increase angle, zoom, and minimum % of pixels tracked for better results.

For moving shapes that can’t be broken down into a planar surface, look for the bulk similar motion and Mocha should be able to ignore the irrelevant data. Again, more manual keyframing may be necessary.

Tighter roto shapes can help roto organic, non-planar shapes but most objects can actually be broken down into what is essentially “low poly” models for the purposes of roto. Make sure to paper doll everything, split up shapes by motion groups (like ribcage, soft belly tissue, and pelvis) for difficult roto shots. Things like cloth flapping in the breeze are just hard, period, but using translation only to get the bulk motion and then keyframing the flutters can help.

The best practice for when to set keyframes when adjusting a spline when the roto shape falls off is to keyframe the last frame where the roto shape is onscreen, and the very next frame, keyframe it completely offscreen until it needs to be back onscreen. Then keyframe the frame right before the shape needs to appear onscreen, and again for the next frame it needs to be on screen for (the next immediate frame).

It can absolutely help to stop tracking, correct the spline, then track backward, and it can also work to just correct and continue forward, depending on the shot and what your previous frames look like. If your previous spline looks fine, no need to track backwards.

In order to determine where to make a spline correction and add a keyframe you should look for the “arcs” of the animation. That is, where the spline is most divergent from where you need it to be. Then scrub between those two keyframes and look for the most divergent parts of the spline, correct them, and keep scrubbing between keyframes. The idea is to evaluate where corrections are needed from a large scale to a small scale. Look up principles of animation re: squashing, stretching, and tweening for a better grasp of what this looks like.

You can prevent the spline from falling off a tracked shape between keyframes by adjusting your tracking parameters like switching to translation scale and rotation only, or translation only, adjusting the rotation and zoom values to increase the search area, increasing the dimensions of the search area, or adjusting the Minimum % of pixels tracked.

The shape is where the track is looking, the surface tool is what the track is doing. In the case of roto, the shape follows the track. Keyframes correct the shape edges while it moves along with the tracking data in order to create roto masks.

You are probably using the wrong techniques, or are not understanding how the tracker works, we have not found any bugs with keyframing roto. Have you watched our Fundamentals of Mocha or Getting Started with Mocha series? They really help and go in depth into these details. But honestly, nothing beats hands on experience for learning once you grasp the concept of planar tracking. That concept is the hard part. Basically, Mocha is a texture tracker, looking for motion groups we call “planes” because planes have pixels in them that move all relative to one another. But thinking about it in motion groups can be more helpful than thinking about it in a rigid, “planar” way. Think of primary and secondary motion, think of how those concepts differ, and think of how something is put together of parts all moving separately, then think of what those parts have in common motion-wise, group relative motion together into one shape, and you’ll have a much more articulated “paper doll” to work with in roto. However, also understand that any tracking has places where it will fail and some parts of roto will always be manual work. For every shape that can be tracked, Mocha can save you time, but not all shapes have enough texture or are visible enough to be tracked throughout the whole shot, in these cases, manual keyframes are necessary. You can also check out our documentation for more info.

It is also vital that you stack objects in the layer pile as closest to camera at the top and furthest from camera at the bottom of the pile. This means shapes will hold out from one another properly while you track, and you won’t get conflicting tracks.

Please let me know if you have any more questions and I will be happy to help you.




A big thank you for this clear and detailed novel! :slightly_smiling_face:

These are great tips and they’ve helped me see a couple places where I’m making mistakes. One is turning perspective on instead of off. I’ve stopped doing that. Also, changing angle and zoom seem to help at a cost of slower tracking.

Re: bugs - The inconsistencies I’m calling bugs are exclusively with stereo 360 tracking. My current workaround for 360 issues is to treat the equirectangular frame as a flat top/bottom stereo file with no lens distortion. This is working well in keeping spline shapes from distorting depending on the view angle.

The keyframe inconsistencies generally happen with the non-hero eye and are probably weird edge cases that might be a result of poor technique on my part. I’ll try to document what’s going on and post a video so your team can evaluate.

I’ve gone thru the fundamentals and getting started series as well as the documentation. I think I have the concept of layer stacking down. Here’s a sample from my current project. How does this look?

A couple things to note:
This is a really dark scene shot in a nightclub. I’ve been using brightness scaling to make it easier to find the shapes, but just learned the tracker doesn’t make use of the brighter raster. This would explain why the tracker is having trouble with certain shapes.

The character in this shot is doing a lot of movement and rotations. There are a lot of non-planar detail shapes (like the horns) that have to be broken into separate shapes because they change so radically depending on orientation.

I’ve found zooming into the timeline and focusing on a few seconds at a time helps a lot.

There are a lot of keyframes from tweaking the splines but even a little bit of slip is noticeable. Is this simply the grunt work of roto I’m dealing with here? :wink:

Can you talk a bit about edges?

Besides edge feather and motion blur, what are the tools and techniques in Mocha to get a good looking edge for comping?

How would you deal with something fuzzy, like fur for example?


Geat! I am glad you’ve already done the training. I never know where folks are at on the forum, experience wise. For the stereo, yeah, those are definitely bugs. We are trying to track them down better, so if you have video, that would be helpful to pass to support & QA: Boris FX | Open a Case

Your layers look pretty good to me.

I think, for this shot, you’re unfortunately dealing with the pain of complex roto work. There’s a reason it gets outsourced a lot, no one wants to do it. It simply takes time. I don’t know if you’ve seen that soccer roto shot I did recently, in our tutorials and webinars, but it took me every bit of five or six hours to do and it’s only a few seconds long. It would have taken me a whole day without Mocha.

For edges, I like to do a one or two pixel edge feather on my final imported mattes regardless. It just makes them blend better. I also like to cut in a little on the subject, rather than risk pulling edge pixels into my matte. And you want to make sure you don’t make too many keyframes side by side on objects that aren’t rapidly moving or you’ll get edge jitter and it’s noticable.

For dealing with fur… that’s an entire post. I will say for most hair, you get what you can in the shape and you composite what you can’t get. So for example, in Alice in Wonderland they did cleanplates behind her hair and painted in new hair for the edges of her hair, because it was so wispy around the edges. For fur, try to get away with what you can by cutting into the fur as much as possible and using per point edge feathering with the Mocha selection tools. All hair and transparent objects are tricky.


1 Like

I did see that soccer shot and if that took a master like you 5-6 hours, it makes me feel a lot better about the amount of time my shot is taking! :wink:

I have a total of about 900 frames of this character as he walks down into the audience (where it’s even darker) turning his back to the camera, then turns around and walks back onstage. Big fun, lol. I wish I could outsource but this is a no-budget indy project so it’s all on me. the good news is I have a couple months to get it done and I’ll be a lot better at roto by the time I’m done.

For edges, I’ve been able to leverage some of the great matte FX tools in AE to choke, feather, and detail the edges. So far my comp is looking great. I’ll start using the edge techniques you described moving forward.

Re: fur - makes sense. Comping with a rotobrushed layer is how I would probably approach it. Splines are great for many things but hair and smoke not so much. A roto edge tutorial someday would be great.

Re: bugs - I’m finding them. Not just non-hero splines, the magnetic detail parameter appears to be broken in 2019.5. I’ll make a separate post in the forum to illustrate.

Thanks for your help!