Distinguishing transient blocks from frequently used blocks enables servicing references to transient blocks from a small fully-associative auxiliary cache structure. By inserting only frequently used blocks into the main cache structure, we can reduce the number of conflict misses, thus achieving higher performance and allowing the use of direct mapped caches which offer lower power consumption and lower access latencies. We suggest using a simple probabilistic filtering mechanism based on random sampling to identify and select the frequently used blocks. Furthermore, by using a small direct-mapped lookup table to cache the most recently accessed blocks in the auxiliary cache, we eliminate the vast majority of the costly fully-associative lookups. Finally, we show that a 16K direct-mapped L1 cache, augmentedwith a fully-associative 2K filter, achieves on average over 10% more instructions per cycle than a regular 16K, 4-way set-associative cache, and even ??5% more IPC than a 32K, 4-way cache, while consuming 70%-80% less dynamic power than either of them.