This work presents a gossip-based protocol, termed QuickPeer, which builds and maintains latency-aware overlay topologies. Such topologies are useful for several distributed applications, like distributed online gaming, context-aware P2P applications and QoS-aware publish/ subscribe systems. The distinctive feature of QuickPeer is that it can manage large scale overlay topologies providing each host in the overlay with its closest or furthest neighbour, according to network distance (RTT), in few gossip rounds. We present experimental results that prove that QuickPeer is a scalable and robust solution for large-scale latency-aware overlay topology management.