Color-based object recognition is typically concerned with building statistical descriptions from pixels that correspond to an object class and then using these models to detect pixels that belong to previously seen objects. Specific instances of color-based classification occur in a number of computer vision problems including background modeling, image-based retrieval, and multi-view object recognition and tracking. Color-based models are dependent on the intrinsic parameters of the camera(s) used to acquire them. Rather than view this as a problem, we propose to utilize this relationship to control (to a degree) how color models are acquired by modifying camera intrinsics. In particular, we introduce an algorithm that searches for the best set of camera settings that will facilitate class separability for a given set of colored objects. The method searches the space of color settings including white balance, hue and saturation in order to maximize classification accuracy of example objects in the camera's view. In this way, a normal commodity camera is tuned for a specific recognition problem. We demonstrate the method on a variety of objects. Results show that class-specific color calibration can significantly improve recognition rates over manual calibration of color balance.