The High Level Architecture (HLA) for distributed simulation was proposed by the Defense Modeling and Simulation Office of the Department of Defense (DOD) in order to support interoperability among simulations as well as reuse of simulation models. One aspect of reusability is to collect and analyze data generated in simulation exercises, including a record of events that occur during the execution, and the states of simulation objects. In order to improve the performance of existing data collection mechanisms in the HLA simulation system, the paper proposes a multi-agent data collection system. The proposed approach adopts the hierarchical data management/organization mechanism to achieve fast data access which is indispensable to the analysis of simulation exercise. Furthermore, the multi-agent data collection system adopts a formalization expression method to describe the system behavioral characteristics, and implements the hierarchy language supports to the description by combing the XML and Petri net. In addition, we propose an independent reinforcement learning algorithm to generate optimized joint recording program which guarantees that the data collection and query tasks can be rationally distributed among logging agents as well as efficiently utilize computational resource. The testing results indicate that the proposed approach, under the premise of complete collection of simulation data, not only reduces the network load imposed by data collection components, but also provides effective supports to the analysis of simulation exercise.