In the real world concepts are often not stable but change over time. A typical example of this in the biomedical context is antibiotic resistance, where pathogen sensitivity may change over time as pathogen strains develop resistance to antibiotics that were previously effective. This problem, known as concept drift (CD), complicates the task of learning a robust model. Different Ensemble Learning (EL) approaches (that instead of learning a single classifier try to learn and maintain a set of classifiers over time) have been shown to perform reasonably well in the presence of concept drift. In this paper we study how much local feature selection (FS) can improve ensemble performance for data with concept drift. Our results show that FS may improve the performance of different EL strategies, yet being more important for EL with static integration of classifiers like (weighted) voting. Further, the improvement of EL due to FS can be explained by its effect on the accuracy and diversity of base classifiers. The results also provide some additional evidence that diversity can be better utilized with the dynamic integration of classifiers.