With the pervasive use of handheld digital devices such as camera phones and PDAs, people have started to capture images as a way of recording information. However, due to the non-planar geometric shapes of some documents such as thick bound book pages, rolled scripts, etc, the captured images are often warped. This causes problems for many document analysis and retrieval tasks. This paper proposes a de-warping approach that first reconstructs the surface using a Shape-from-Shading (SFS) technique and then flattens the warped surface using a physically-based deformable model. The SFS method is based on the viscosity framework and considers perspective projection under oblique light source. The recovered surface is represented as a triangular mesh and is restored to its planar state through a numerical integration process. We have tested the proposed approach on both synthetic and real images. The synthetic images are mainly used to evaluate the SFS method and the results are compared with those of a prior SFS method. The reconstructed surfaces of real document images are also compared with the actual shapes captured using a 3D scanner and the restored images are evaluated with those obtained from range-scanned surface shapes.