This paper proposes the design of multi-user authentication in the multi-application based environment and Role-based Access Control by using PKI Authentication and X.509 Privilege Management Infrastructure (PMI). A binding model of RBAC authorization based on attribute certificate (AC) and public key certificate (PKC) is presented. Especially, the way of attribute mapping between PKC, Bridge AC, and Role AC is illustrated. In addition, the activity-based policy enforcement is introduced to make the system respond to malicious activities more appropriately. At a core, the multi agent system approach is applied to automate the flexible and effective management of user authentication, role delegation as well as system accountability. Finally, we reported our ongoing implementation status and demonstrated that our proposed model is a potential solution to support strong authentication and dynamic authorization in the multi-user and multi-application environment.