This paper presents a predictable and quantifiable approach to designing a covert communication system capable of effectively exploiting covert channels found in the various layers of network protocols. Two metrics are developed that characterize the overall system. A measure of probability of detection is derived using statistical inference techniques. A measure of reliability is developed as the bit error rate of the combined noisy channel and an appropriate error-correcting code. To support reliable communication, a family of error-correcting codes are developed that handle the high symbol insertion rates found in these covert channels. The system metrics are each shown to be a function of the covert channel signal-to-noise ratio, and as such the two can be used to perform system level design trade-offs. Validation of the system design methodology is provided by means of an experiment using real network traffic data.