Drone Footage VR Sky Replacement

I have footage taken with a drone and 360 camera. The drone does move the camera in the shot. I’d like to do a sky replacement with a stock still I have and for the most part it looks ok. The problem though is in the movement of the drone as the replacement sky isn’t tracked to the vr drone footage. I’ve brought the vr drone footage into Mocha Pro and tracked it and reoriented it then pushed it back into After Effects. Then I comp’ed the replaced footage over the reoriented vr drone footage. Obviously this is a mistake but I’m not quite certain how to go about tracking the sky replacement so that it matches the movement of the vr drone.

Any help or tutorials I might peruse would be welcome.

Have you tried tracking the horizon?

Yes. I’ve tracked the horizon…in fact for vr drone footage that’s usually what I track. I use three different locations.

I’d have to see the problem, that should work. Are there any clouds you can track or other sky features that move slowly?

As luck would have it, the day was crystal clear. I redid the horizon track and as long as the movements of the drone are slight, I can do a sky replacement in AE using the vr drone tracked footage. I’m just sort of feeling my way in the dark with this to be honest. I’ve followed a few tutorials on sky replacement using normal HD footage and tried to extend it to vr, but that didn’t really work.

I would have normally used a clone patch method but the drone was sometimes in front of the sun which made things more difficult.

What if you reorient stabilize the footage to stable, replace the sky, and destabilize?

So reorient the footage in mocha, bring it into AE and sky replace, then back out to mocha?

When you reorient, you can simply copy the Mocha effect in AE, precomp the reoriented footage and sky and move all attributes inside the precomp, paste Mocha onto the new precomp, and then invert the 360 stabilize in the new pasted effect. It’s a thankfully very simple process.

Which drone do you have? Does it matter finally?

I’m going to give that method a try. We’re using a DJI Mavic Pro, but we suspend a GoPro Max camera from it for the 360 footage.

Oh my, I think this works wonderfully. I just want to make sure I have done things properly as I’ve run into one snag.

  1. Drone footage is trimmed to length.
  2. Mocha Applied
    3.Track two areas near or on horizon.
  3. Reorient footage so that the horizon is flat and along the middle of the frame.
  4. Save and close mocha, returning to After Effects
  5. in the mocha app, under Module Renders, click Render and select reorient module
  6. Add sky to newly reoriented footage as a layer
  7. Precomp and move all attributes to precomp
  8. Copy original mocha effect and past into precomp
  9. In mocha app, under module renders, click render and select reorient. Under Tracking data, click invert.

This results in a sky that moves exactly with the drone footage. However, the horizon is no longer flat. Should that be the case? If so, then I gather I would precomp this layer, add mocha and retrack to reorient the horizon?

The horizon can be reoriented again if you need for sure. But make sure that’s over the final, where you have everything how you want it. You can also adjust it in the reorient without retracking, just change how the horizon sits using the controls before you add the sky in.

Ok. Wonderful! This is how I am proceeding. ANd yes, I did add a mocha effect to the final precomp that had the sky and drone footage already tracking together just to level out the horizon.


Happy to help. :slight_smile: