Adjust track problem

Before tracking, I set some keyframes to guide (force) the tracker through some tricky footage. This seemed to work well, the track follows the keyframes as expected. When I turn on the surface however, it tears away from the track as if an adjust track is appled. Upon entering adjust track, there do seem to be reference points there. I didn’t set them.

Here’s a screen snap. The plane tracked is behind the silhoutte, this is the portion of the track I set with key frames.


Can I delete or reset adjust track data? am i seeing a feature or a bug?