Expressive face and body gestures are among the main non-verbal communication channels in human-human interaction. Understanding human emotions through these nonverbal means is one of the necessary skills both for humans and also for the computers to interact intelligently and effectively with their human counterparts. Much progress has been achieved in affect assessment using a single measure type; however, reliable assessment typically requires the concurrent use of multiple modalities. Accordingly in this paper, we present preliminary results of automatic visual recognition of expressive face and upper-body action units (FAUs and BAUs) suitable for use in a vision-based affective multimodal framework.