Since graphics cards have become programmable the recent years, numerous computationally intensive algorithms have been implemented on the now called General Purpose Graphics Processing Units (GPGPUs). While the results show that GPGPUs regularly outperform CPU based implementations, the question arose how optical flow algorithms can be ported to graphics hardware. To answer the question, the optimal algorithm structure to maximize the performance gained by using graphics cards has to be found. In this paper we compare the performance of two algorithms that are highly different in structure, implemented on both CPU and graphics hardware. Analyzing the results of the CPU and GPGPU implementation, we explore the mapping of the algorithms to the graphics hardware and thereof extract information about a preferred structure of optical flow algorithms for GPGPU based implementation.