How can I parse Layer tracks from mocha files?

Mocha AE
1

Hi,

I’m new to Mocha and enjoying the experience!

I’d like to parse a .mocha text file exported from Mocha for AE so I can visualise the tracking data in a standalone Python script. (Currently I’m simply using OpenCV’s drawing functions).

I’ve written a script that extracts each Layer’s surface corners:

Surface0X
Surface0Y
Surface0B
Surface1X
Surface1Y
Surface1B
Surface2X
Surface2Y
Surface2B
Surface3X
Surface3Y
Surface3B

With a few manual tweaks my conversion function from mocha to absolute pixels (Python/OpenCV) looks like this:

  • flip coorinates vertically (frame_h - y1 )
  • offset vertically by half the bounding box height (+ bbox_height // 2)
  • additionally, when visualising tracking data, I add the Translation_X and Translation_Y values to each surface corner
def convert_mocha_coords(corner1, corner2, frame_h):
    x1, y1 = corner1
    x2, y2 = corner2
    bbox_height = y2 - y1
    y1 = frame_h - y1 + bbox_height // 2
    y2 = frame_h - y2 + bbox_height // 2
    return ((x1, y1), (x2, y2))

def offset_corners_int(corner1, corner2, tx, ty):
    x1, y1 = corner1
    x2, y2 = corner2
    x1 = int(x1 + tx)
    y1 = int(y1 + ty)
    x2 = int(x2 + tx)
    y2 = int(y2 + ty)
    return ((x1, y1), (x2, y2))

While it feels I’m close, I’m stabbing in the dark without documentation and I’m only applying data from the translation tracks (but not from rotation, scale, shear, etc.). Additionally I’m multiplying the translation values by 50 (via random trials). This is all guess work at the moment.

This is what tracking looks like in Mocha for AE:
sensori.al/mocha/mocha_tracking_test.gif

and this is what visualising one layer currently looks like:
sensori.al/mocha/mocha_pyparsing_test.gif

Notice the y position starts to drift downwards.

I’ve tried to find documentation online regarding the .mocha format, Layer and Track in particular (including the Mocha Pro Python Guide), but I couldn’t find enough information.

How can I parse a .mocha file and convert surface and track data to pixel coordinates for visualisation ?

More specifically:

  • in what coordinate system are the surface coordinates ?
  • in which orders should the track transformations be applied ? (e.g. translate first, then rotate, then scale (then shear, then apply perspective (if any) ?)
  • do track transformations use different coordinate systems ? (How should transformations be applied to surface coordinates to calculate the absolute pixel coordinates ?)

Thank you so much,
George

2

Hi George,

The Mocha project file is not an interchange format that’s designed to be consumed by other software. As you’ve realised, there are many subtle details affecting how different parameters are combined together to evaluate the position of a surface corner, mesh point or spline vertex. Without using code from Mocha it will be very difficult to get correct results, especially for more complex projects.

Mocha Pro has a Python API which you can use to extract the information you want from a Mocha project file, it’s what I’d recommend you use for this project. Mocha AE is designed only for a narrow range of use cases directly within AE itself, so it doesn’t include the Python API or any other ways to export data in a format suitable for what you’re trying to do.

Best regards,

J-P

3

Hi @jps,

Many thanks for the feedback.

I will try Mocha Pro. I’m starting with tracking a basic rough bounding box around an object: no fancy VFX tasks in my case.
If you do have hints on how I could compute the bounding rect of each tracked object (without revealing any sensitive information of course) that would extremely useful.

Regarding Mocha Pro Python documentation, unfortunately the links I found here all lead to 404s:

Furtunately I tweaking the URL from the reference from 7.5.0 (which a google search revealed) got me to mocha Python API Reference — mocha documentation

Changing the URL to 9.5.6 did not work, but for what I need (bounding box of each tracked bezier contour), the Python API wouldn’t have changed drammatically between 9.5.0 and 9.5.6 :crossed_fingers:

I suspect I will need to do something like:

for each layer in project.layers:
    for each contour in layer.contours:
        for t in frame_times:
            print(contour.get_bounding_rect(time=t)

Hopefully I’m on the right track ?

Thank you so much for your time and support: very much appreciated!

4

Hi @jps

Many thanks again for the feedback.

I’ve just tried Mocha Pro to do a basic test and even using the Python API it seems there are some transformations that need to be applied ?

I’m using get_bounding_rect() but perhaps that isn’t right approach ? (Maybe I should use control_point(index) instead?):

from mocha.project import *
import json
import os

prj = get_current_project()

num_frames = prj.get_length()

data = {}
data['num_frames'] = num_frames
data['contours'] = {}

for layer in prj.layers:
    for i, contour in enumerate(layer.contours):
        label = f"{layer.name} contour {i}"
        data['contours'][label] = [contour.get_bounding_rect(frame) for frame in range(num_frames)]

with open(os.path.join(prj.get_output_dir(), './mocha_pro_test.json'), 'w') as f:
    f.write(json.dumps(data))

print('done')

This is what I have in mocha:

This how the visualised exported data looks like:

(Notice the green rectangles at the top. It looks like Y still needs to be flipped (as in my original .mocha parse test), but also the points seem scaled/in a different coordinate system)

How do you recommend replicating the boxes I see in mocha externally ?

Thank you so much for your time,
George

6

Hi George,

I think you’re on the right lines. The easiest way to verify your work is by importing a static grid image into Mocha and drawing four-point splines very precisely on grid boundaries. In Mocha the centre of the lower-left pixel is (0,0) and the centre of the top-right pixel will be (width,height). When drawing overlays in your own program, be mindful of pixel aspect ratio. In Mocha, the bounding rect is expressed in square pixels, so if you are drawing it in display space in your own code, you may need to stretch it by the PAR.

Also, unless you need the full frame range of the project, you could restrict it to the layer in/out points using

in_point = layer.parameter(['Basic', 'In_Point']).get()
out_point = layer.parameter(['Basic', 'Out_Point']).get()

A worked example is at Creating a Custom Tracking Data Export.

Oh and sorry about the problem with the broken links. There is a problem on our site where it’s possible to reach the same page at two different URLs, one with a trailing slash and one without. I just checked and the one without a slash is now correct. If you follow the website menu Support | Documentation, then click Mocha Python Documentation, you will get to a working page. Hopefully our web team will be able to find a permanent solution to this problem soon.

Best regards,

J-P