Traditional authentication and access control are no longer suitable for ubiquitous computing paradigm. They are only effective if the system knows in advance which users are going to access and what their access rights are. Therefore, it calls for a novel security model. In this paper, we outline major security problems in ubiquitous computing and propose a new architecture, TBSI (Trust-based Security Infrastructure). In TBSI, trust and risk management plays a key role to support authentication and authorization to unknown users. Meanwhile, intrusion detection and home firewall are also integrated to make TBSI more robust. This paper is an extension of our previous work to give more detailed description and enhancement of the architecture. TBSI is on-going research project to support our context-aware middleware CAMUS.