The complicated user interfaces and complex functionality of nowadays interactive products lead to a new class of failures: People do not understand their products and thus fail to use them successfully; many products are returned for which no detectable errors can be found. These field problems of interactive products cannot be found by traditional testing methods. Industry needs reliable and structured information about the users? behavior to get understanding about the root cause of so called soft product failures. In this paper we present a framework that helps usability and quality experts to derive user models from product observation. This is supported by a novel visual languagefor specification what should be observed and how collected data is represented, and a system architecture for distributed self-observing systems. This approach separates the concern definition of observation from the implementation of observation facilities.