With the increasing complexity of large-scale distributed (LSD) systems, an efficient monitoring mechanism has become an essential service for improving the performance and reliability of such complex applications.This paper presents a {\em scalable, dynamic, flexible} and {\em non-intrusive} monitoring architecture for managing large-scale distributed (LSD) systems. This architecture, which is referred to as the HiFi monitoring system, detects and classifies interesting primitive and composite events and performs either a corrective or steering action. When appropriate, information is also disseminated to management applications, such reactive control tools.The outlined solution offers improvements over related works by supporting new monitoring techniques such as hierarchical filtering-based monitoring and filter incarnation that improve the monitoring scalability and dynamism which are required for managing large-scale distributed systems. The HiFi monitoring system has been implemented and used at the Old Dominion University for monitoring and steering Interactive Remote Instruction (IRI) which is a large-scale distributed multimedia system for distance learning.