The tracking of features in real-time video streams forms the integral part of many important applications in human-computer interaction and computer vision. Unfortunately tracking is a computationally intensive task, since the video information used by the tracker is usually prepared by applying a series of image processing filters. Thus it is difficult to realize a real-time tracker using only the CPU of a standard PC. Over the last few years, commodity Graphics Processing Units (GPU) have evolved from fixed graphics pipeline processors into more flexible and powerful data-parallel processors. These stream processors are capable of sustaining computation rates of greater than ten times that of a single CPU. GPUs are inexpensive and are becoming ubiquitous (desktops, laptops, PDAs, cell phones). They are now capable to greatly relieve the CPU especially for large-scale parallel processing tasks, which map well to the architecture of the GPU. In this paper, we present a system, which uses a gradient vector field to track features with flexible geometric templates. Our implementation is specifically designed to suit the parallel processing architecture of the GPU. It is capable to achieve realtime performance with framerates of around 30 frames per second.