Existing searching schemes in unstructured P2Ps can be categorized as either blind or informed. The quality of query results in blind schemes is low. Informed schemes use simple heuristics that lack the theoretical background to support the simulation results. In this paper, we propose to improve searching by reinforcement learning (RL), which has been proven in artificial intelligence to be able to learn the best sequence of actions in order to achieve a certain goal. Our approach, ISRL (intelligent searching by reinforcement learning), aims at locating the best path to desired files at low cost. It explores new paths by forwarding queries to randomly chosen neighbors. It also exploits the paths that have been discovered to reduce the cumulative query cost. Two models of ISRL are proposed: the basic ISRL for finding one desired file, and MP-ISRL (multipath ISRL) for finding multiple desired files. ISRL outperforms existing searching approaches in unstructured P2Ps by achieving higher query quality with less query traffic. The experimental result supports the performance improvement of ISRL.