Ambient Networks (AN) pose new challenges to the management discipline, and policies are considered to be an adequate solution for providing flexibility, distributed control, and self-management features. However, the current state-of-the art IETF policy framework was not designed for the challenges of new 3G/4G environments such as AN. This paper presents PBMAN, a policy-based architecture and a composition framework that extends the AN architecture, where policies are intrinsically at the underlying layer by design and not as a later ad-on. The use of policies and their interaction with network composition is the main research challenge of PBMAN. The current architecture has been designed based on previous experience, on a design-implement-test development cycle. The framework was used to model a video on demand scenario, whereto composition policies based on an extended version of the XACML policy language have been written.