The present paper focuses on efficient inverse rendering using a Photometric Stereo technique for realistic surfaces. The technique primarily assumes the Lambertian reflection model only. For non-Lambertian surfaces, application of the technique to real surfaces in order to estimate 3-D shape and spatially varying reflectance from sparse images remains difficult. In the present paper, we propose a new Photometric Stereo technique by which to efficiently recover a full surface model, starting from a small set of photographs. The proposed technique allows diffuse albedo to vary arbitrarily over surfaces while non-diffuse characteristics remain constant for a material. Specifically, the basic approach is to first recover the specular reflectance parameters of the surfaces by a novel optimization procedure. These parameters are then used to estimate the diffuse reflectance and surface normal for each point. As a result, a lighting-independent model of the geometry and reflectance properties of the surface is established using the proposed method, which can be used to re-render the images under novel lighting via traditional rendering methods.