Mocha Pro is broken in stereo equirectangular space

tl:dr - Mocha uses the wrong coordinate system for stereoscopic 360 tracking, spline animation, and key-framing, causing inconsistant and incorrect results. Non-hero view keyframe data is inconsistent and easily broken.

I’ve used Mocha Pro professionally on a number of stereoscopic 360 VR video projects since it was in beta last year. While it’s proven invaluable, I’ve struggled with inconsistent and odd behaviors with tracking, spline animation, and key-framing.

I started this thread below as a possible workaround for difficulties I’ve had with stereo 360 tracking. Based on @jps’s suggestions, I’ve done some tests that I think demonstrate a fundamental problem with how Mocha currently works in stereo equirectangular space.

Here’s a video demonstration of option 1. Please read the full video description for timestamp linked annotations.

This video demonstrates inconsistent keyframe behavior in the right (non-hero) view. Tracking a single frame in all views correctly sets the right view offset. The right view spline shape correctly follows changes in the hero view shape. But the added hero view keyframe breaks the offset in the right view. Additional right view keyframes don’t help.

What’s going on with the right view offset data? Why is the offset data reverting when new hero view spline keyframes change?

Here’s a video demonstration of option 2. Again, please read the full video description for timestamp linked notes.

This video demonstrates a basic problem with how Mocha works in stereo equirectangular space. Tracking a spline in the hero view only, we see the right view spline follows properly, with an incorrect offset. We open the stereo offset tab and try adjusting the X Y values. But instead of moving the shape’s X Y position as expected, the shape warps in an odd way. What’s going on?

Turning on 3D difference view shows the answer.

Adjusting the X Y values again, we see the whole frame twisting oddly. Let’s change the view angle to Front and try again.

In Front view, the X Y position changes move the frame as expected.

Changing back to Left view, we now see that the odd warping is because the X Y position is changing relative to the Front view coordinates.

As far as stereo is concerned, Front is 90 degrees vertical facing forward anywhere in the 360 sphere. The frame of reference for front is the viewport not cardinal Front.

Mocha is using the wrong frame of reference in stereo equirectangular space.

It’s treating compass directions as planes, with cardinal Front the reference point, instead of using the viewport as its frame of reference.

This fundamental error affects all aspects of Mocha’s tools in stereo equirectangular space - tracking, spline creation and animation, keyframes, and modules such as adjust track.

It explains odd/inconsistent behaviors such as adjust track tweak controls working backwards or sideways depending on the position of the tracked object.

The rest of the video again demonstrates inconsistent keyframe behavior - After manually positioning the right view spline at the beginning and end keyframe positions of the hero view, the right view tracks properly. Adding a new hero view keyframe in the middle of the track breaks the right view offset. It appears to add a phantom right view keyframe, even though there’s nothing visible on the timeline.

What’s going on here? Keyframe inconsistencies like this, on top of the problem with equirectangular mapping have made Mocha very difficult to use efficiently for stereo 360 work.

If I’m misunderstanding something or “holding it wrong”, please feel free to correct me. I’m an advanced user but not an expert and am still learning new things about Mocha every day. But I’ve worked with software for very long time and I can tell when something’s broken. It really feels like something’s broken here.

@jps, @rosss, @martinb, @maryp - please watch the videos and let me know your thoughts. I’m happy to answer any questions, share sample projects and help in any way.

As always, thanks for your help!
–David

2 Likes

David, thank you for breaking the bug down and finding a solid reproducible case. I have seen exactly the same behavior and it’s making MochaVR completely unusable for doing stereo 360 roto work. Hoping that it can be fixed soon now that you’ve done such a great job laying the problem out so clearly. :pray::crossed_fingers::pray::crossed_fingers::pray::crossed_fingers:

1 Like

This is very thorough, thank you David.

We will investigate all of this and see what can be done.

The offset by tracking once seems clear: Mocha is not assuming the tracking offset after you track additional frames only in the Hero view. I need to determine if this is a regression from the initial stereo implementation.

The manual stereo offset not using the current camera coordinates is a definite bug, or at the very least an incorrect implementation. I’ll file something for this.

We’ll be in touch

1 Like

Thank you Martin. Please keep me posted and let me know if you need any more info or if I can further help in any way.