Deliberative agents operating in open environments must make complex real-time decisions on scheduling and coordination of domain activities. These decisions are made in the context of limited resources and uncertainty about the outcomes of activities. We describe a reinforcement learning based approach for efficient meta-level reasoning. Empirical results showing the effectiveness of meta-level reasoning in a complex domain are provided.