This paper breaks with the common practice of using a joint state space representation and performing the joint data association in multi-object tracking. Instead, we present an interactively distributed framework with linear complexity for real-time applications. When objects do not interact on each other, our approach performs like multiple independent trackers. When the objects are in close proximity or present occlusions, we propose a magnetic-inertia potential model to handle the "error merge" and "labelling" problems in a particle filtering framework. Specifically, we propose to model the interactive likelihood densities by a "gravitation" and "magnetic" repulsion scheme and relax the common first-order Markov chain assumption by using an "Inertia" Markov chain. Our model represents the cumulative effect of virtual physical forces that objects undergo while interacting with others. It implicitly handles the "error merge" and "labelling" problems and thus solves the difficult object occlusion and data association problems using an innovative scheme. Our preliminary work has demonstrated that the proposed approach is far superior to existing methods not only in robustness but also in speed.