A novel method for view-based recognition of human activity is presented. The basic idea of our method is that activities can be positively identified from a sparsely sampled sequence of few body poses acquired from videos. In our approach, an activity is represented by a set of pose & velocity vectors for the major body parts (hands, legs and torso) and stored in a set of multidimensional hash tables. We show that robust recognition of a sequence of body pose vectors can be achieved by a method of indexing & sequencing and it requires only few vectors (i.e. sampled body poses in video frames). We find that the probability of false alarm drops exponentially with the increased number of sampled body poses. We also achieve speed invariant recognition by eliminating the time factor and replacing it with sequence information. Experiments performed with videos having 8 different activities show robust recognition even for different viewing directions.