Tracking and object isolation

Hello there,
I am new to tracking and object isolation, and I have some concerns.
I’ll explain in details what I’ve done so far and what is my objective.

I have a few videos for shoes demonstrations, where the model moves the legs, shows the shoes, and so on. The background is small rocks, cement, grass, little piece of woods and so on (we don’t have a green screen, so we film in a parking lot), and the camera is hand held so it’s not in a fixed position.
As much as my cameraman is holding the position as still as he can, but he camera still slightly moves, it’s inevitable.

Objective: Isolate the model legs and shoes (target of the video) from the background, so that i can replace it with a proper one.

Issues:

  • I have tried placing a x-spline around the areas to keep (legs and shoes), boost up the area of tracking (I tried all values up to 100%), when I hit tracking, the spline gets deformed over time catching also elements in the background (small rocks, small pixels deformation etc).

  • I have tried removing the camera shake with stabilization, trying to track some points in the area, however, as soon as the process is done and I apply the stabilization, in After Effects the video gets deformed badly, scaling up and down, rotating and so on, becoming unusable.

Question:

At this point I tried everything, and rotobrush in after effects is just a pain to edit frame by frame so it’s out of story.

So my question is, how should I approach the task?
Knowing the background interfere with the tracking, how can I track legs and shoes accurately avoiding deformation removing completely the background?

Suggestions, please?

Hello
If i got you right, you want to create masks for legs and shoes. Well, Mocha is a great tool for that task. But without seeing the footage it’s really hard to say why the problem might occur.

But I might guess one scenario. Mocha is a Planar tracker, that means that it tracks a groups of pixels that moves in the same direction. Maybe you having a model, that makes some king of moves like turnarounds, and some groups of pixels become unseen, so Mocha “looses” it and everything get deformed. But that’s only a guess

Can you please provide an example of what shot you’re working on, and the way how you draw your shapes?
I can also recommend you watching this video tutorial, maybe that’ll be handy for you

Hello,
I’ll upload some pictures sample of the videos I’m trying to isolate and remove background from.

Those are some of the videos I’m trying to track and remove the background from.


Those are my attempts to define x-splines for the parts I want to keep and track.


Then I start tracking with the following settings:

  • Input Channel: Luminance
  • Min % pixels used: 90
  • Translation, Scale, Rotation, Shear (I even tried with Perspective as well)
  • Large Motion

And the track gets deformed and won’t follow the defined objects/model as shown below
3


5
8
9

These are just a minor of the deformations and wrong tracking, because in some parts the actual track x-spline gets all deformed that it loses completely the shape of what I drawn.

So I have no idea how to proceed from there.
I really don’t want to having to edit and correct frame by frame, because some videos have about 19000+ frames, so only to correct the x-splines in the correct position it’s gonna take me days to elaborate a 2 minute video.

Suggestions?

@xyorutenx
Hi, MP is a planer tracker, it likes to track flat objects that are on the same plane, a spline has to be created & tracked for each plane, tracking round things such as limbs isn’t easy as a round thing can have hundreds if not thousands of planes that make up the curved surface,
& on top of that you’re tracking different parts of the limb,
you would need to track the top of the foot/shoe separately, (the sides & upper part near the ankle of the foot/shoe are different planes),
track the lower part of the leg separately,
track the thigh separately,
you might even have to track the ankle & different part of the knee separately.
PowerMesh tracking might help but you would prob still have to track the individual parts of the limb separately,

Have a look at this series of videos

@xyorutenx
Thanks for sharing screenshots with me.

Now I see. When you’re trying to rotoscope people, you should think about your model as a “marionette doll”. Every part of the leg (feet, shin, thigh) should be considered as separated objects, i.e. you should create separated shapes for different parts. Please watch the video i mentioned above, there’s an excellent demonstration of this principle.
You wont be able to track all leg as one complete object unfortunately

What do you suggest to do then in order to remove the unwanted background?
Apart having to create hundreds of splines…

I had a look at the video, but, I did try that approach, to split splines already, however, it required to correct/modify every single frame, because it would get deformed instantly capturing the background, enlarging, rotating and so on.

I even tried to track something really small, like the shoe lace, still it got deformed and required correction frame by frame

In general Mocha should handle such type of shots. Some manual corrections may be required, but you definitely don’t need to create keyframe every single frame, unless you have some really crazy motion.
While drawing your shapes, try creating as less points as you can. Something like this.

I can try that, but, using less points doesn’t it lose in quality and accuracy?

Less points means less manual tweaking. You can see how much less I needed, to cover the whole leg (except the laces)

1 Like

Right - as Elizabeth suggests, keeping your roto shapes simpler will assist the roto process. It sounds like you are confused – the individual spline points are not tracking points. Mocha tracks the pixel pattern inside the shape. This can be viewed by showing the SURFACE tool. The spline is a child of the track data – meaning you first try to get an accurate track and then correct the roto by adjusting keyframes. When a track is good you should not have to adjust every frame, instead going through the timeline and adding KF when the motion is largest and then subdividing.

You may want to view the Mocha Essentials series with attention to the roto chapters.

Cheers,
Ross

I think I should switch to another software then, if Mocha is unable to track this kind of situation.
I tried everything, even tracking a small area part, I’m forced to keyframe every single frame almost.

I used as less points as possible

still I’m forced to go frame by frame to match the shape and tracking area.

kk

Just to arrive at 1/5 of the work of a 25 seconds video, took me 2 hours and half, not to mention it isn’t even accurate.

Can you share the clip,?
put it on Google Drive, Dropbox or even on Youtube?

If you can share the shot, I can show how to make it work for your example