General information about a class of objects, such as human faces or teeth, can help to solve the otherwise ill-posed problem of reconstructing a complete surface from sparse 3D feature points or 2D projections of points. We present a technique that uses a vector space representation of shape (3D Morphable Model) to infer missing vertex coordinates. Regularization derived from a statistical approach makes the system stable and robust with respect to noise by computing the optimal tradeoff between fitting quality and plausibility. We present a direct, non-iterative algorithm to calculate this optimum efficiently, and a method for simultaneously compensating unknown rigid transformations. The system is applied and evaluated in two different fields: (1) reconstruction of 3D faces at unknown orientations from 2D feature points at interactive rates, and (2) restoration of missing surface regions of teeth for CAD-CAM production of dental inlays and other medical applications.