4 point tracking

So I have green screen and 4 trackers on it.
I create X-spline around those four points, correct plane and i start to track. But after a few frames, trackin looses trackers and goes on its own.
Foootage iz HDV with almost no grain, very little motion blur, so it shouldn’t have any problems tracking it.

For better understanding, here are pics:

set X-spline:
Image: http://shrani.si/f/h/Bv/2mjOfarw/mochatr01.jpg

after coupla frames:
Image: http://shrani.si/f/3F/F4/1vMhdyl4/mochatr02.jpg
(of course image moves and perspective is different)

any suggestions?