Retrack the surface as it goes off screen with translation only.
Use adjust track to correct any other problem frames. That’s where your keyframing will come in. The surface is a child of the track, and can’t be animated except for in MANUAL TRACK or ADJUST TRACK, or by the track itself.
Manual track will allow you to animate the surface by hand, sometimes, for only 5 or fewer bad frames, this is the best option. (This is right in the track tab, it turns the surface tool into something that actually drive the track. Turn it on and then when you are done, turn it right off. Keyframes are automatic and can be done on previously tracked frames.)