We develop efficient (1 + \varepsilon)-approximation algorithms for generalized facility location problems. Such facilities are not restricted to being points in \mathbb{R}^d, and can represent more complex structures such as linear facilities (lines in \mathbb{R}^d, j-dimensional flats), etc. We introduce coresets for weighted (point) facilities. These prove to be useful for such generalized facility location problems, and provide efficient algorithms for their construction. Applications include: k-mean and k-median generalizations, i.e., find k lines that minimize the sum (or sum of squares) of the distances from each input point to its nearest line. Other applications are generalizations of linear regression problems to multiple regression lines, new SVD/PCA generalizations, and many more. The results significantly improve on previous work, which deals efficiently only with special cases. Open source code for the algorithms in this paper is also available.