One of the most effective ways of modeling vector data for unsupervised pattern classification or coding, is to assume that the observations are the result of picking randomly out of a fixed set of different distributions. In this paper, we propose to perform the unsupervised estimation of the mixture density underlying the data as the problem of separating multi-class sources. Assuming in each class independent components standard linear Independent Component Analysis (ICA) can be adopted in the recently extended mode, which provides signal reconstruction for a multi-class mixture [6, 7, 9]. Unfortunately, in practical problems the class densities necessary to match the experimental distributions must be degenerate or poorly conditioned. In this paper, we approach the problem by assuming from the beginning sources, which either have rank-deficient distributions or show very concentrated eigenvalues. The class membership of each point is based on a distance measure from the hyper-planes and on the likelihood on each hyper-plane. The independent components are then searched within each subspace. We present results of the algorithm on synthetic distributions with various degrees of degeneracy. Our results are promising for feature extraction applications.