Dimensionality reduction techniques seek to represent a set of images as a set of points in a low dimensional space. Here we explore a video representation that considers a video as two parts - a space of possible images and a trajectory through that space. The non-linear dimensionality reduction technique of Isomap, gives, for many interesting scenes, a very low dimensional representation of the space of possible images. Analysis of the shape of the video trajectory through these image spaces gives new tools for video analysis. Experiments with natural video sequences illustrate methods for the very different tasts of classifying video clips and temporal super-resolution.