Coarse-to-fine classification is an efficient way of organizing object recognition in order to accommodate a large number of possible hypotheses and to systematically exploit shared attributes and the hierarchical nature of the visual world. The basic structure is a nested representation of the space of hypotheses and a corresponding hierarchy of (binary) classifiers. In existing work, the representation is manually crafted. Here we introduce a design principle for recursively learning the representation and the classifiers together. This also unifies previous work on cascades and tree-structured search. The criterion for deciding when a group of hypotheses should be "retested" (a cascade) versus partitioned into smaller groups ("divide-and-conquer") is motivated by recent theoretical work on optimal search strategies. The key concept is the cost-to-power ratio of a classifier. The learned hierarchy consists of both linear cascades and branching segments and outperforms manual ones in experiments on face detection.