For H.264/SVC, different layers or frames have different order of precedence according to their importance on video quality and the decoding process. Network should provide more protection for high priority layers or frames when SVC flow is transmitted over IP network. This paper proposes an active queue management algorithm based on priority dropping and PID algorithm, called PID_PD, which first drops the least important packets when network congestion arises. PID_PD implements differentiated service in IP layer to provide high quality video for end user. In addition, PID_PD do not create a side effect for the performance of control system. Simulation shows this method can protect important video data well, lower the disturbance of background flow, and decrease the packet loss rate of SVC flow.