Voice over IP (VoIP) has had a strong effect on global communications by allowing human voice and fax information to travel over existing packet data networks along with traditional data packets. The convergence of voice and data in the same network brings both benefits and constraints to users. Among the several issues that need to be addressed when deploying this technology, security is one of the most critical. We give an overview of VoIP and provide UML models of some aspects of its infrastructure, including architectures and basic use cases. We present some security patterns that describe mechanisms that can control many of the possible attacks and which could be used to design secure systems.